Leebra python
login register

New to Python packaging? Avoid hatch

24 days ago by dan_p_the_dev to c/python

save

share

report

I recently built a Python library for the first time in a while, and to make it available in PyPI I followed the official Python packaging guide. But it turns out there’s some rough edges in the suggested tools. Namely, the build backend hatch puts all the files in the project directory into the built sdist, except from some files it chooses to exclude, which led to a gitignored file as well as VCS metadata getting packaged and uploaded to PyPI. This really surprised me, and I think most Python users new to packaging would be better served by choosing a conservative build backend like uv_build, which puts only the Python source modules and some specific project-level files into the sdist.
eager_eagle 6 points 24 days ago

hatch has hardcoded awareness of Git and Mercurial and specifically searches for project-level .gitignore and .hgignore to read those files directly. But it doesn’t handle user-level or system-level config, so anything you have ignored from config outside of your project, like .env in my example, will not be ignored by hatch, and will wind up in your sdist.

ah that probably explains why I didn't have this problem, same reason why I don't like global gitignores

save

path: 0 25940000, hotness: undefined, score: 6, children: 10
esc 3 points 24 days ago 
[tool.hatch.build.targets.sdist]
include = [
  "pkg/*.py",
  "/tests",
]
exclude = [
  "*.json",
  "pkg/_compat.py",
]

It takes about 30 seconds of reading the documentation to find this.

save

path: 0 25940000 25945139, hotness: undefined, score: 3, children: 6
dan_p_the_dev 3 points 24 days ago

Actually working out the right configuration for your project is likely going to take you more than 30 seconds. And finding the documentation for how to do this is only quick if you know exactly where to look and what to look for. When you invoke pyproject-build it's not clear that hatch is the piece of the puzzle that's causing unexpected files to wind up in your sdist.

And this is still more config and overhead than simply choosing a build backend with more sensible default behavior like uv_build.

save

path: 0 25940000 25945139 25945757, hotness: undefined, score: 3, children: 5
esc 1 point 23 days ago

Please, you've wasted a lot of time writing the article without bothering to read documentation (it's mentioned in both python guide and on the hatchling page), which takes one click and like three paragraphs down, and uv_build requires to use uv and not everyone likes it or wants to use it. Anyway reading documentation is far more important for beginners than automagical tools that supposedly solve all the problems in the world.

save

path: 0 25940000 25945139 25945757 25951255, hotness: undefined, score: 1, children: 4
dan_p_the_dev 2 points 23 days ago

Please, you’ve wasted a lot of time writing the article without bothering to read documentation

? I linked directly to the documentation I read, and that's how I wound up using hatch in the first place. Did you read my article...? Reading the official Python documentation is what led to me uploading my .jj folder and a gitignored .env file to PyPI and writing this post. And writing the post was not a waste of time.

uv_build requires to use uv and not everyone likes it or wants to use it

That's fine. I'm sure there are other build backends that work similarly to uv_build and only grab what's needed to distribute your library. The point isn't that everyone needs to use uv_build, the point is that hatch has surprising and imo dangerous default behavior.

save

path: 0 25940000 25945139 25945757 25951255 25956275, hotness: undefined, score: 2, children: 3
dan_p_the_dev 3 points 24 days ago

Even if you're using git or hg for version control you'll probably wind up with more in your sdist than you need: things like uv.lock, Justfile, or the contents of your tests/ dir might not be harmful, but they bloat your sdist and don't really belong there, and hatch will package them up with your source code unless you explicitly configure it not to.

save

path: 0 25940000 25941971, hotness: undefined, score: 3, children: 2
eager_eagle 7 points 24 days ago

save

path: 0 25940000 25941971 25942837, hotness: undefined, score: 7, children: 1
dan_p_the_dev 2 points 24 days ago

Interesting discussion, thanks for the link! I side with pf_moore there; when I think of distributing a library I think of shipping only the library code itself and none of the supporting files, although I can understand the argument to include things like tests/ and docs/.

That said, if you haven't already tailored your project for hatch, configuring only-include is more work than switching to a build backend that just bundles source modules by default. With the right config hatch can do what you like, and I can understand existing users sticking with it, but for newbies I think it's really risky to suggest a build backend that grabs everything it finds in the working directory and packages it up for PyPI. It's just surprising and dangerous default behavior imo.

save

path: 0 25940000 25941971 25942837 25943545, hotness: undefined, score: 2, children: 0
foenkyfjutschah 6 points 24 days ago

i think the conservative choice would be to use setuptools as build backend.

save

path: 0 25940608, hotness: undefined, score: 6, children: 3
eager_eagle 5 points 24 days ago

It's the conservative choice, which is maybe not ideal for new projects as it predates many PEPs. Hatch, uv, poetry, pdm, are probably all better modern choices.

save

path: 0 25940608 25941084, hotness: undefined, score: 5, children: 2
foenkyfjutschah 1 point 23 days ago

how so?

save

path: 0 25940608 25941084 25952953, hotness: undefined, score: 1, children: 1
eager_eagle 3 points 23 days ago

Caution

If you use setuptools, please be aware that some features that predate standardisation efforts are now deprecated and only temporarily kept for compatibility.

In particular, do not use direct python setup.py invocations. On the other hand, configuring setuptools with a setup.py file is still fully supported, although it is recommended to use the modern [project] table in pyproject.toml (or setup.cfg) whenever possible and keep setup.py only if programmatic configuration is needed. See Is setup.py deprecated?.

Other examples of deprecated features you should not use include the setup_requires argument to setup() (use the [build-system] table in pyproject.toml instead), and the easy_install command (cf. pip vs easy_install).

https://packaging.python.org/...

it's just easier to have a modern hassle-free setup with the other tools

save

path: 0 25940608 25941084 25952953 25955352, hotness: undefined, score: 3, children: 0
onlinepersona 2 points 23 days ago

Thank you. That seems quite problematic. I would expect the build backend only to include what I tell it to. IIRC pyproject has a field to tell which packages exist in the project and I'd expect only that to be included + what was specified e.g resources like images.

save

path: 0 25952333, hotness: undefined, score: 2, children: 2
foenkyfjutschah 2 points 23 days ago

the article doesn't mention the many ways you can define what (not) to include that hatch offers.

save

path: 0 25952333 25952950, hotness: undefined, score: 2, children: 1
onlinepersona 3 points 23 days ago

The problem is the default. It's seemingly opt-out, not opt-in. That leads to experiences like these.

save

path: 0 25952333 25952950 25952964, hotness: undefined, score: 3, children: 0
OliMoli2137 -2 points 24 days ago

Let me say it

N I X F I X E S T H I S

save

path: 0 25946041, hotness: undefined, score: -2, children: 5
onlinepersona 4 points 23 days ago

Uh how? It doesn't have a lockfile for python projects and uv2nix triggers a rebuild everytime you garbage collect nix.

save

path: 0 25946041 25952263, hotness: undefined, score: 4, children: 2
OliMoli2137 1 point 23 days ago

ig nix isn't the best not for every use case then

save

path: 0 25946041 25952263 25955702, hotness: undefined, score: 1, children: 1
onlinepersona 3 points 23 days ago

Then why did you claim nix fixed it? 🤨

save

path: 0 25946041 25952263 25955702 25957139, hotness: undefined, score: 3, children: 0
sukhmel 3 points 24 days ago

But takes forever to set up (at least for me, when I set it up for cross compilation)

save

path: 0 25946041 25947562, hotness: undefined, score: 3, children: 1
OliMoli2137 2 points 23 days ago

yeah cross compilation is very slow

save

path: 0 25946041 25947562 25951627, hotness: undefined, score: 2, children: 0
logging_strict -6 points 23 days ago

There is no way for xyz to be either included or excluded from a package cuz everyone should be running python -m build and checking the tarball and .whl before even considering a commit or a release.

So although the OP chose hitch that doesn't change the workflow of manually checking the tarball.

I use setuptools, wreck (requirements management), and drain-swamp (choose versioning by either "tag" or "current" rather than specifying exact semantic version). What i use has no bearing on the opinion above.

"modern" means you are a little bitch. I'm very tired of that term. A function with multiple return statements is considered "modern". Whereas i see that as wrong. So now that "modern" word is being thrown around for packaging?! Please stop using that term. It's a cope for not being able to code your own build backend (drain-swamp), frame stack free locals inspection (logging-strict), and requirements management (wreck)!

Use whatever you are familiar with without peer pressuring others.

save

path: 0 25952937, hotness: undefined, score: -6, children: 1
eager_eagle 3 points 23 days ago

save

path: 0 25952937 25957705, hotness: undefined, score: 3, children: 0
python
python

@programming.dev

login for more options
8074
744
278

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events
Past

November 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
  • Pythörhead: a Python library for interacting with Lemmy
  • Plemmy: a Python package for accessing the Lemmy API
  • pylemmy pylemmy enables simple access to Lemmy's API with Python
  • mastodon.py, a Python wrapper for the Mastodon API
Feeds

go to feed...