Update literature notes

This commit is contained in:
Gabriel Arazas 2022-01-11 16:22:02 +08:00
parent 8d88983e21
commit cde9112a9d
3 changed files with 80 additions and 1 deletions

View File

@ -1,9 +1,10 @@
:PROPERTIES:
:ID: 0c00ac45-109a-499e-b521-01eaef2b03b4
:ROAM_REFS: @HowDrawAnything2008
:END:
#+title: How To Draw Anything
#+date: 2021-09-19 20:54:51 +08:00
#+date_modified: 2021-10-24 19:43:29 +08:00
#+date_modified: 2022-01-07 11:22:20 +08:00
#+published: 2008-01-16
#+author:
#+source: https://www.youtube.com/watch?v=p9TFLQq1Uh4

View File

@ -0,0 +1,47 @@
:properties:
:id: 30d6a3d2-42f3-4f49-8d4c-bf433dc82350
:roam_refs: @packagingconPackagingGrailLudovic2021
:end:
#+title: The Packaging Grail - Ludovic Courtès | PackagingCon 2021
#+date: 2021-12-28 21:27:18 +08:00
#+date_modified: 2021-12-28 22:51:08 +08:00>
#+published: 2021-11-23
#+author: Ludovic Courtès
#+source: https://youtu.be/tcTOkXlE_ac
#+language: en
* Synopsis
- a primer to [[id:be917383-84c4-4bf5-9ca0-b04bfb778f4f][Guix package manager]] and its goals
- it is free as in [[id:e8c1218f-35a4-4dd3-a174-5d8f848b74b2][Free software]]
- a universal package manager that tries to cover all dependencies from different programming languages similar to [[id:3b3fdcbf-eb40-4c89-81f3-9d937a0be53c][Nix package manager]]
- transparent through the [[id:fe9e21bc-3b38-4d0f-a785-253248a38ed7][Reproducible builds]] initiative integrating it into the package manager (e.g., =guix challenge=)
- focus on isolated builds leading to bit-identical builds and [[roam:Bootstrappable builds]] (building everything from source) enforcing further transparency and more security (e.g., "Trusting trust" attacks, creating backdoors through compiler bugs)
- making efforts to reduce the binary blobs required to boot an operating system from scratch through GNU Mes
- the balance to the right way versus pragmatism;
other solutions may present to be faster and more convenient versus doing things the "right way" (e.g., comparing Pytorch package from Guix built from source versus Pypi containing prebuilt binaries; see [[Potential problems]]);
in this case, Guix is drawing the line by providing tools to make packaging easier (e.g., =guix import=)
* Potential problems
- binary packages in package repositories (e.g., =torch= package in Pypi) [fn:: There's also =PyQt-Qt6= package from Pypi.]
- developer-uploaded binaries which may contain other modifications and harder to verify
- no indication of sources from random binaries
- licenses issues may pop up
- no standard way to reproduce builds and verify so anyone can upload anything;
this can create supply chain issues
- see the left-pad issue from npm that happened years ago
- with project-specific and language-specific package managers (e.g., npm, cargo, pip), a lot of the overall reach of the software can feel isolated;
Guix is intending to at least provide a way to unify them (as it is one of the goals, after all)
- mostly related to software testing — e.g., missing test dependencies,
- hosted source can be different from upstream;
[fn:: I personally think the binaries from Visual Studio Code is another example.]
* Presenter's wish
- *move non-free software*;
at the very least, move it to another repository to let people make informed decisions;
not only this cannot be reproduced well but also impedes security
- *disallow developer-uploaded binaries*;
the talk presented an example with the previous case with Debian allowing it some time ago;
if it is allowed, at least provide a way to reproduce it;
- *accurate licensing info*
- *accurate package data from repositories* (e.g., dependencies, license)
- *good description/synopsis*

View File

@ -0,0 +1,31 @@
:properties:
:id: 2ba59995-1fc7-46a7-aa0b-cceca4134a7f
:roam_refs: @packagingconPythonPackagingWhy2021
:end:
#+title: Python Packaging: Why don't you just?
#+date: 2021-12-21 17:31:18 +08:00
#+date_modified: 2021-12-21 20:52:58 +08:00>
#+published: 2021-11-22
#+author: Tzu-ping Chung (TP)
#+source: https://www.youtube.com/watch?v=jpP07nZiicE
#+language: en
* The state of Python packaging
- Python predates the common packaging ecosystem we all interact today thus it has some artifacts from history
- back in the day, for someone to use a dependency, they have to download the source;
in Python's case, they chose with the most common way to download a file: simple tarballs
- as more platforms grow and being supported, there are more incentives to make delivering dependencies faster but each output may contain different metadata;
this results in big packages to [[https://pypi.org/project/Pillow/#files][create huge lists of possible outputs]]
- for the package manager to know the metadata, it has to download the whole package and run the build script
- however, this simplicity also brings advantage that you can simply create a Python package repository by simply building the wheels and dropping them along with the source in a directory
- another artifact to consider is Python supports a lot of platforms;
it is considered supported when Python is confirmed to run on a platform;
it cannot support building all wheels for all platforms so at some point when doing a =pip install= of a big package, it has to build the package from source
- metadata are just placed in files which is a good thing since they can be placed next to the sources
* The answer to the question
- third system effect;
the first few efforts went well but some people want to create a better system with more ambitious goals and a grander vision;
except it didn't work out and the attempts will inspire another group to try a better version
- the quick succession of attempts at standardizing Python packaging fragmented the packaging process by offering multiple ways to do the goal;
as more attempts have made, there's lesser central driving force