From cde9112a9d5e7c9964dbd68e8b2a00d689b7d5ae Mon Sep 17 00:00:00 2001 From: Gabriel Arazas Date: Tue, 11 Jan 2022 16:22:02 +0800 Subject: [PATCH] Update literature notes --- notebook/literature.HowDrawAnything2008.org | 3 +- ....packagingconPackagingGrailLudovic2021.org | 47 +++++++++++++++++++ ...ure.packagingconPythonPackagingWhy2021.org | 31 ++++++++++++ 3 files changed, 80 insertions(+), 1 deletion(-) create mode 100644 notebook/literature.packagingconPackagingGrailLudovic2021.org create mode 100644 notebook/literature.packagingconPythonPackagingWhy2021.org diff --git a/notebook/literature.HowDrawAnything2008.org b/notebook/literature.HowDrawAnything2008.org index ea1afd3..72d4a81 100644 --- a/notebook/literature.HowDrawAnything2008.org +++ b/notebook/literature.HowDrawAnything2008.org @@ -1,9 +1,10 @@ :PROPERTIES: :ID: 0c00ac45-109a-499e-b521-01eaef2b03b4 +:ROAM_REFS: @HowDrawAnything2008 :END: #+title: How To Draw Anything #+date: 2021-09-19 20:54:51 +08:00 -#+date_modified: 2021-10-24 19:43:29 +08:00 +#+date_modified: 2022-01-07 11:22:20 +08:00 #+published: 2008-01-16 #+author: #+source: https://www.youtube.com/watch?v=p9TFLQq1Uh4 diff --git a/notebook/literature.packagingconPackagingGrailLudovic2021.org b/notebook/literature.packagingconPackagingGrailLudovic2021.org new file mode 100644 index 0000000..3166929 --- /dev/null +++ b/notebook/literature.packagingconPackagingGrailLudovic2021.org @@ -0,0 +1,47 @@ +:properties: +:id: 30d6a3d2-42f3-4f49-8d4c-bf433dc82350 +:roam_refs: @packagingconPackagingGrailLudovic2021 +:end: +#+title: The Packaging Grail - Ludovic Courtès | PackagingCon 2021 +#+date: 2021-12-28 21:27:18 +08:00 +#+date_modified: 2021-12-28 22:51:08 +08:00> +#+published: 2021-11-23 +#+author: Ludovic Courtès +#+source: https://youtu.be/tcTOkXlE_ac +#+language: en + +* Synopsis +- a primer to [[id:be917383-84c4-4bf5-9ca0-b04bfb778f4f][Guix package manager]] and its goals + - it is free as in [[id:e8c1218f-35a4-4dd3-a174-5d8f848b74b2][Free software]] + - a universal package manager that tries to cover all dependencies from different programming languages similar to [[id:3b3fdcbf-eb40-4c89-81f3-9d937a0be53c][Nix package manager]] + - transparent through the [[id:fe9e21bc-3b38-4d0f-a785-253248a38ed7][Reproducible builds]] initiative integrating it into the package manager (e.g., =guix challenge=) +- focus on isolated builds leading to bit-identical builds and [[roam:Bootstrappable builds]] (building everything from source) enforcing further transparency and more security (e.g., "Trusting trust" attacks, creating backdoors through compiler bugs) +- making efforts to reduce the binary blobs required to boot an operating system from scratch through GNU Mes +- the balance to the right way versus pragmatism; + other solutions may present to be faster and more convenient versus doing things the "right way" (e.g., comparing Pytorch package from Guix built from source versus Pypi containing prebuilt binaries; see [[Potential problems]]); + in this case, Guix is drawing the line by providing tools to make packaging easier (e.g., =guix import=) + +* Potential problems +- binary packages in package repositories (e.g., =torch= package in Pypi) [fn:: There's also =PyQt-Qt6= package from Pypi.] + - developer-uploaded binaries which may contain other modifications and harder to verify + - no indication of sources from random binaries + - licenses issues may pop up +- no standard way to reproduce builds and verify so anyone can upload anything; + this can create supply chain issues + - see the left-pad issue from npm that happened years ago +- with project-specific and language-specific package managers (e.g., npm, cargo, pip), a lot of the overall reach of the software can feel isolated; + Guix is intending to at least provide a way to unify them (as it is one of the goals, after all) +- mostly related to software testing — e.g., missing test dependencies, +- hosted source can be different from upstream; + [fn:: I personally think the binaries from Visual Studio Code is another example.] + +* Presenter's wish +- *move non-free software*; + at the very least, move it to another repository to let people make informed decisions; + not only this cannot be reproduced well but also impedes security +- *disallow developer-uploaded binaries*; + the talk presented an example with the previous case with Debian allowing it some time ago; + if it is allowed, at least provide a way to reproduce it; +- *accurate licensing info* +- *accurate package data from repositories* (e.g., dependencies, license) +- *good description/synopsis* diff --git a/notebook/literature.packagingconPythonPackagingWhy2021.org b/notebook/literature.packagingconPythonPackagingWhy2021.org new file mode 100644 index 0000000..9e81f69 --- /dev/null +++ b/notebook/literature.packagingconPythonPackagingWhy2021.org @@ -0,0 +1,31 @@ +:properties: +:id: 2ba59995-1fc7-46a7-aa0b-cceca4134a7f +:roam_refs: @packagingconPythonPackagingWhy2021 +:end: +#+title: Python Packaging: Why don't you just? +#+date: 2021-12-21 17:31:18 +08:00 +#+date_modified: 2021-12-21 20:52:58 +08:00> +#+published: 2021-11-22 +#+author: Tzu-ping Chung (TP) +#+source: https://www.youtube.com/watch?v=jpP07nZiicE +#+language: en + +* The state of Python packaging +- Python predates the common packaging ecosystem we all interact today thus it has some artifacts from history +- back in the day, for someone to use a dependency, they have to download the source; + in Python's case, they chose with the most common way to download a file: simple tarballs +- as more platforms grow and being supported, there are more incentives to make delivering dependencies faster but each output may contain different metadata; + this results in big packages to [[https://pypi.org/project/Pillow/#files][create huge lists of possible outputs]] +- for the package manager to know the metadata, it has to download the whole package and run the build script +- however, this simplicity also brings advantage that you can simply create a Python package repository by simply building the wheels and dropping them along with the source in a directory +- another artifact to consider is Python supports a lot of platforms; + it is considered supported when Python is confirmed to run on a platform; + it cannot support building all wheels for all platforms so at some point when doing a =pip install= of a big package, it has to build the package from source +- metadata are just placed in files which is a good thing since they can be placed next to the sources + +* The answer to the question +- third system effect; + the first few efforts went well but some people want to create a better system with more ambitious goals and a grander vision; + except it didn't work out and the attempts will inspire another group to try a better version +- the quick succession of attempts at standardizing Python packaging fragmented the packaging process by offering multiple ways to do the goal; + as more attempts have made, there's lesser central driving force