A plea not to use uv-build for Python

2 days ago by bignose to c/python

After seeing yet another dozen packages lose keywords on Gentoo because of a single key dependency switching to uv-build, I would like to plea on behalf of Gentoo users: please don’t use uv-b…

@mgorny@treehouse.systems:

After seeing yet another dozen packages lose keywords on Gentoo because of a single key dependency switching to uv-build, I would like to plea on behalf of Gentoo users: please don’t use uv-build as a build backend.

Just to be clear, this is not about uv itself or any other Astral tools; these are good tools with a nice and responsive upstream. The uv-build backend is one thing we have to disagree on. I’m not saying it’s bad; I’m saying that its benefits don’t outweigh the drawbacks.

Voroxpete 26 points 2 days ago

I'm going to be completely real with you, I'm really not sure how many developers have the Gentoo user base that high on their list of priorities.

path: 0 26320600, hotness: undefined, score: 26, children: 0
CameronDev 21 points 2 days ago

It’s a huge dependency (as of 0.13.0, its wheels are between 1.2 MiB and 1.5 MiB

What world are they living in that 1.5MiB is huge?

SPARC

Right.

Its cool you want to keep your old system going, but thats a cost you have to pay. You can't expect the rest of the world to make their lives harder to support your niche.

path: 0 26320860, hotness: undefined, score: 21, children: 18
bignose 12 points 2 days ago

What world are they living in that 1.5MiB is huge?

That depends. Am I buying the SSD last year, or today?

The RAMpocalypse is going to make “this doesn't run on a low-powered low-capacity device” a much more relevant issue.

path: 0 26320860 26321447, hotness: undefined, score: 12, children: 15
eager_eagle 9 points 2 days ago

1.5MiB wasn't much 20 years ago, even with a rampocalypse that's not changing soon.

path: 0 26320860 26321447 26323331, hotness: undefined, score: 9, children: 14
undefined 11 points 2 days ago

This line of thinking is why binaries today are like 800 MB. If you can cut cruft as you go during development you shouldn’t have to go back and say “it’s only 1.5 MB”. You should’ve been in the habit of writing DRY, clean code and not importing libraries that do nothing but add bloat.

I’m sorry but I’m sick and tired of this attitude where I work (web development). We are blowing thousands/millions of dollars per year on resources because people no longer believe in optimizing anything. I’ve lost count where I’ve pushed back and said “we need to figure out how to do this with less memory” rather than simply handing shit more and more and more.

It might seem nitpicky as fuck at first, but if you start designing with good processes these wins become more automatic.

path: 0 26320860 26321447 26323331 26323683, hotness: undefined, score: 11, children: 9
CameronDev 8 points a day ago

There is a valid discussion to be had for software bloat, but 1.5mb isnt huge by any measure. Plenty of binaries can and do exceed that without being "poorly coded".

path: 0 26320860 26321447 26323331 26323683 26324876, hotness: undefined, score: 8, children: 6
eager_eagle 3 points 21 hours ago

That's slippery slope. If you reason about performance in absolute terms, nobody will ever get things done. The reason we have electron apps is very much not because people are not shaving off 1.5MB off their binaries. This request to not use uv-build because of 1.5MB is just insane and out of touch for normal consumer hardware, and it only makes sense in the context of SPARC.

path: 0 26320860 26321447 26323331 26323683 26334934, hotness: undefined, score: 3, children: 0
Senal 3 points a day ago

I mostly agree, but there is a tradeoff here between effort and resource usage

Do you spend the extra x months dev time to go from 800Mb to 200Mb?

how about another x months to go from 200MB to 100MB?

100MB -> 80Mb? 80Mb -> 67Mb ?

and that's only memory footprint, not performance, test coverage, infrastructure etc.

We've swung way in to the "800Mb binary ?, meh, they all have RAM to spare" (looking at you specifically fucking electron shells) and i fully agree we should be way more on the side of resource usage optimisation.

Optimising for deadlines and effort costs is also a consideration, if we don't acknowledge and address that as part of the conversation, it'll be seen as gold-plating or "perfect is the enemy of good".

path: 0 26320860 26321447 26323331 26323683 26326042, hotness: undefined, score: 3, children: 0
mapto 3 points 2 days ago

Multiple this by the number of package dependencies which with virtual environments easily become high thousands.

path: 0 26320860 26321447 26323331 26323585, hotness: undefined, score: 3, children: 3
Senal 5 points a day ago

path: 0 26320860 26321447 26323331 26323585 26326049, hotness: undefined, score: 5, children: 0
eager_eagle 3 points 2 days ago

uv uses hardlinks, you'll only need more space if you need different versions of uv-build

path: 0 26320860 26321447 26323331 26323585 26323646, hotness: undefined, score: 3, children: 1
canaryWart -4 points a day ago

You can't expect the rest of the world to make their lives harder to support your niche.

I bet you wouldn't say that shit to your boss, or a billionaire

path: 0 26320860 26327646, hotness: undefined, score: -4, children: 1
CameronDev 7 points a day ago

I absolutely would say that to a billionaire, because they aren't paying me.

My boss, have said it a number of times. I've almost verbatim told them that their use case doesn't line up with our broader user bases' use case, and that I wasn't going to support it. Part of a healthy working relationship is being able to push back.

Ignoring my specific case though, what is your point? That people with power shouldn't ever face push back? Surely that's not your point?

path: 0 26320860 26327646 26328764, hotness: undefined, score: 7, children: 0
eager_eagle 12 points 2 days ago

It’s a huge dependency (as of 0.13.0, its wheels are between 1.2 MiB and 1.5 MiB, depending on the platform)

🤣

path: 0 26323268, hotness: undefined, score: 12, children: 0
tal 11 points 2 days ago

I don't use Gentoo or uv-build, but I don't get the problem.

Gentoo builds packages from source for each user. Okay.

But I assume that it can't be building the entire toolchain from scratch for each package. It can't be rebuilding, say, GNU autotools for each package that uses it during its build. It's presumably either packaging the toolchain dependencies and not updating them between packages or caching the build.

Why can't Gentoo just do whatever it does there for this uv-build thing?

EDIT: Here's a Hacker News thread on this same article, which I turned up in trying to get some context:

https://news.ycombinator.com/item?id=50076048

I discovered partway through that I don't care as much about Python or Gentoo build expectations as some of the people involved, enough to fully understand it, but it sounds like, from a quick skim, that some people feel that there are unspecified expectations from the Python build environment. Some people feel that an unspecified dependency version means that one must use the latest verson as of time of build, and since Gentoo typically builds packages for a given user at different times, this entails using different versions of the build tool. Others do not feel that way.

Some snippets:

UV maintainer asking why Gentoo doesn't just pick a version and freeze there and then some responses

(I am one of uv’s maintainers.)

...I don’t understand why Gentoo needs to rebuild every single release of uv-build. It doesn’t change as much as uv itself, and it’s intentionally compatible across uv versions. Why not set a policy of only updating once weekly, or even less frequently, with exceptions for important breaking changes or bug fixes?

Because if you don't pin a version of uv-build in your project, that means "use the latest version available". Gentoo can't second guess you--even if it were able to verify somehow that the new version of uv-build that just got released will end up producing an identical wheel for your project as the previous version, without actually using the new version of uv-build.

Other distros don't necessarily have quite the same problem because, if they include packages of Python projects, they don't normally track the build backend as a dependency. So if uv-build pushes an update, the distro would push an updated package for uv-build, but they wouldn't look at every Python project they package to see if it used uv-build as a build backend and rebuild it if it did. It's the "rebuild from source every time" aspect of Gentoo that forces that connection.

i guess gentoo is rolling-release, vs something like debian where every package is frozen in time (+ backports), and debian does seem to largely solve the coordination problem by making everything on the system use a fixed version of some package; with rare situations allowing >1 version of the package at a time.

my take on linux distros being annoyed by their own policies is that perhaps they should change the policy. in this case, perhaps they should pretend uv-build updates once a month or something.

path: 0 26320794, hotness: undefined, score: 11, children: 1
AnyOldName3 1 point a day ago

It does seem hypocritical for Gentoo to complain about wasted time building things from source over and over because of another project's policies when the whole point of Gentoo is building things from source over and over.

Also, assuming any project that doesn't pin a specific version of a dependency requires the latest possible version seems backwards, as that's typically how things indicate that they're independent of any particular version.

path: 0 26320794 26326960, hotness: undefined, score: 1, children: 0
psyq 1 point 10 hours ago

Somewhat of a side discussion, but these aren't the only issues you can have when using uv-build.

While it's stable now, it expects certain directory layouts and can only be used on pure Python projects. If you need anything else, you are better off using hatchling anyway, which used to be the default even for uv.

This is also why some of your old uv-based projects might build fine and the same code won't build in a modern uv project; they switched the default.

Here's a thing directly from the authors:

While the backend supports a number of options for configuring your project structure, when build scripts or a more flexible project layout are required, consider using the hatchling build backend instead.

Source

path: 0 26342533, hotness: undefined, score: 1, children: 0
python
python

@programming.dev

login for more options
8074
744
278

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events
Past

November 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
  • Pythörhead: a Python library for interacting with Lemmy
  • Plemmy: a Python package for accessing the Lemmy API
  • pylemmy pylemmy enables simple access to Lemmy's API with Python
  • mastodon.py, a Python wrapper for the Mastodon API
Feeds

go to feed...