logo

drewdevault.com

[mirror] blog and personal website of Drew DeVault git clone https://hacktivis.me/git/mirror/drewdevault.com.git

Python-stop-screwing-distros-over.md (4309B)

    

  1. ---
  2. title: "Python: Please stop screwing over Linux distros"
  3. date: 2021-11-16
  4. ---
  6. > Linux distributions? Oh, those things we use to bootstrap our Docker
  7. > containers? Yeah, those are annoying. What were you complaining about again?
  9. The Python community is obsessed with reinventing the wheel, over and over and
  10. over and over and over and over again. distutils, setuptools, pip, pipenv, tox,
  11. flit, conda, poetry, virtualenv, requirements.txt, setup.py, setup.cfg,
  12. pyproject.toml... I honestly can't even list all of the things you have to deal
  13. with. It's a disaster.
  15. [![An xkcd comic showing a convoluted graph of competing Python environments](https://imgs.xkcd.com/comics/python_environment.png)](https://xkcd.com/1987/)
  17. This comic is almost 4 years old and it has become much worse since. Python is a
  18. mess. I really want to like Python. I have used it for many years and in many
  19. projects, including SourceHut, which was predominantly developed in Python. But
  20. I simply can't handle it anymore, and I have been hard at work removing Python
  21. from my stack.
  23. This has always been a problem with Python, but in the past few years everyone
  24. and their cousin decided to "solve" it by building another mess which is totally
  25. incompatible with all of the others, all of the "solutions" enjoying varying
  26. levels of success in the community and none of them blessed as the official
  27. answer.
  29. I manage my Python packages in the only way which I think is sane: installing
  30. them from my Linux distribution's package manager. I maintain a few dozen Python
  31. packages for Alpine Linux myself. It's from this perspective that, throughout
  32. all of this turmoil in Python's packaging world, I have found myself feeling
  33. especially put out.
  35. Every one of these package managers is designed for a reckless world in which
  36. programmers chuck packages wholesale into ~/.pip, set up virtualenvs and pin
  37. their dependencies to 10 versions and 6 vulnerabilities ago, and ship their
  38. computers directly into production in Docker containers which aim to do the
  39. minimum amount necessary to make their user's private data as insecure as
  40. possible.
  42. None of these newfangled solutions addresses the needs of any of the distros,
  43. despite our repeated pleas. They all break backwards compatibility with our
  44. use-case and send our complaints to /dev/null. I have seen representatives from
  45. every Linux distro making repeated, desperate pleas to Python to address their
  46. concerns, from Debian to Arch to Alpine to NixOS, plus non-Linux distros like
  47. FreeBSD and Illumos. Everyone is frustrated. We are all struggling to deal with
  48. Python right now, and Python is not listening to us.
  50. What is it about Linux distros that makes our use-case unimportant? Have we
  51. offered no value to Python over the past 30 years? Do you just feel that it's
  52. time to shrug off the "legacy" systems we represent and embrace the brave new
  53. world of serverless cloud-scale regulation-arbitrage move-fast-and-break-things 
  54. culture of the techbro startup?
  56. Distros are feeling especially frustrated right now, but I don't think we're
  57. alone. Everyone is frustrated with Python packaging. I call on the PSF to sit
  58. down for some serious, sober engineering work to fix this problem. Draw up a
  59. list of the use-cases you need to support, pick the most promising initiative,
  60. and put in the hours to make it work properly, today and tomorrow. Design
  61. something you can stick with and make stable for the next 30 years. If you have
  62. to break some hearts, fine. Not all of these solutions can win. Right now,
  63. upstream neglect is destroying the Python ecosystem. The situation is grave, and
  64. we need strong upstream leadership right now.
  66. ---
  68. P.S. PEP-517 and 518 are a start, but are very disappointing in how little they
  69. address distro problems. These PEPs are designed to tolerate the proliferation
  70. of build systems, which is exactly what needs to stop. Python ought to stop
  71. trying to avoid hurting anyone's feelings and pick one. Maybe their
  72. decision-making framework prevents this, if so, the framework needs to be
  73. changed.
  75. ---
  77. P.P.S. There are a lot of relevant xkcds that I wanted to add. Here's the ones I
  78. left out:
  80. - https://xkcd.com/1988/
  81. - https://xkcd.com/927/
  83. Further reading: [Developers: Let distros do their job](https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html)