Closed Bug 1356101 Opened 2 years ago Closed 2 years ago

virtual_packages.txt misses many modules that appear in mach_bootstrap.py

Categories

(Firefox Build System :: General, enhancement)

enhancement
Not set

Tracking

(firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox55 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(1 file)

Greg, is there a technical reason not to parse virtualenv_packages.txt from mach_bootstrap.py? Or should we remove all the .pth entries from virtual_packages.txt and import mach_bootstrap to use the SEARCH_PATHS from there to feed the virtualenv preparation?
Flags: needinfo?(gps)
(A third option is to keep the status quo, add the missing things in virtualenv_packages.txt, and have a unit test ensuring both lists are in sync)
Initially, virtualenv_packages.txt was for the virtualenv, which was used by the build system and only the build system. mach was its own no-virtualenv (just a bunch of sys.path hacks). They each listed just the packages they needed.

Over time, mach commands gained the ability to activate the virtualenv. This is slower than sys.path hacks, but allows e.g. C extension compilation, entrypoint install, etc. And, people started adding new packages to both mach and virtualenv_packages.txt (myself included).

At this point, yes, we can consolidate things. The only major trouble should be for the "special" conditions in virtualenv_packages.txt, namely the setup.py hacks to compile C extensions, support for optional packages, and single file copies. For the bulk of the "just install a .pth file / modify sys.path" packages, we can unify that list somewhere.
Flags: needinfo?(gps)
Assignee: nobody → mh+mozilla
Blocks: 1343718
Comment on attachment 8857823 [details]
Bug 1356101 - Derive the mach_bootstrap search path from build/virtualenv_packages.txt.

https://reviewboard.mozilla.org/r/129838/#review133048

This looks good except for the failure in automation.
Attachment #8857823 - Flags: review?(gps) → review-
Comment on attachment 8857823 [details]
Bug 1356101 - Derive the mach_bootstrap search path from build/virtualenv_packages.txt.

The failure was on one of the multiple previous iterations. But mozreview doesn't tell that.
Attachment #8857823 - Flags: review- → review?(gps)
In fact, let me rebase this on top of the now-landed bug 1343718, too.
Comment on attachment 8857823 [details]
Bug 1356101 - Derive the mach_bootstrap search path from build/virtualenv_packages.txt.

https://reviewboard.mozilla.org/r/129838/#review133082
Attachment #8857823 - Flags: review?(gps) → review+
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/54f9c9d1a094
Derive the mach_bootstrap search path from build/virtualenv_packages.txt. r=gps
https://hg.mozilla.org/mozilla-central/rev/54f9c9d1a094
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.