Closed Bug 914506 Opened 11 years ago Closed 11 years ago

Support wheels for installing Python packages

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: gps, Assigned: gps)

References

Details

Attachments

(1 file)

Python wheels are the new packaging hotness. I propose we start using them instead of importing Python package sources into the tree. Well, at least we support installing from wheels.
(In reply to Gregory Szorc [:gps] from comment #0)
> I propose we start using them instead of importing Python package sources into the tree.

Not a good idea. At all.
(In reply to Mike Hommey [:glandium] from comment #1)
> (In reply to Gregory Szorc [:gps] from comment #0)
> > I propose we start using them instead of importing Python package sources into the tree.
> 
> Not a good idea. At all.

Why? To me, this represents an easy solution to the .pyc in srcdir problem. It also solves the psutil binary package problem. The .whl archives contain the original source, so we don't lose that.
Given your last comment, I'm guessing this will be r-. We can retain the
source files and not install from .whl if you want. But we'll likely
want wheel support for installing psutil from a wheel.
Attachment #802091 - Flags: review?(mh+mozilla)
Assignee: nobody → gps
(In reply to Gregory Szorc [:gps] from comment #2)
> Why? To me, this represents an easy solution to the .pyc in srcdir problem.
> It also solves the psutil binary package problem. The .whl archives contain
> the original source, so we don't lose that.

Painful to patch (and we have a great history of patching the python modules we import), and painful for tier-3 platforms.
Comment on attachment 802091 [details] [diff] [review]
Install 3rd party Python packages via wheels

Review of attachment 802091 [details] [diff] [review]:
-----------------------------------------------------------------

If you really want to use wheels, then create the wheel files from source at build time.

That being said, if the only foreseen advantage for doing that for all the imported packages is for the .pyc problem, istr there was a symlink based solution for that. As for prebuilt binaries, if you want to ship some, they could be in the tree, and copied appropriately. We don't need wheels for that either.

Wheels is maybe good for python packages developers to distribute their stuff, but to me it's not meant to use the way we consume python packages. Ask yourself this: would you rm -rf python/mozbuild and replace it with a .whl package?
Attachment #802091 - Flags: review?(mh+mozilla) → review-
(In reply to Mike Hommey [:glandium] from comment #5)
> That being said, if the only foreseen advantage for doing that for all the
> imported packages is for the .pyc problem, istr there was a symlink based
> solution for that.

Right. We'll need to solve the .pyc problem for all the tree-maintained Python packages (e.g. python/mozbuild). I was thinking wheels would be a clean way to solve distributing 3rd party packages whose source we don't modify. Single .whl file in the source tree is nice and clean.

> As for prebuilt binaries, if you want to ship some, they
> could be in the tree, and copied appropriately. We don't need wheels for
> that either.

Not technically. But the binary packaging story pre-wheels is teh crap. Wheels gets us sanity there. Drop a file in a directory and Python will select the one for your current platform and Python distribution automatically.

> Wheels is maybe good for python packages developers to distribute their
> stuff, but to me it's not meant to use the way we consume python packages.
> Ask yourself this: would you rm -rf python/mozbuild and replace it with a
> .whl package?

Of course not - because python/mozbuild is developed in tree! Again, I was thinking wheels would be used for 3rd party packages only.
You convinced me I'm wrong.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: