Closed Bug 1656614 Opened 6 months ago Closed 5 months ago

Virtualenv `.pth` files don't get recreated on virtualenv "re-creation"

Categories

(Firefox Build System :: General, defect)

defect

Tracking

(firefox81 fixed)

RESOLVED FIXED
81 Branch
Tracking Status
firefox81 --- fixed

People

(Reporter: rstewart, Assigned: rstewart)

References

Details

Attachments

(1 file)

No description provided.

I wrote this patch because I noticed that the .pth files in my objdir virtualenvs were extremely repetitive, containing multiple references to the same directories. This happens because we append to the in-virtualenv .pth files when calling populate(), but we don't ever clean up the old ones, meaning that whenever we determine that the virtualenvs are out of date and need to be recreated, we actually leave A LOT of state lying around on-disk that is going to go on to impact further uses of the virtualenv. Concretely, how this manifested is that when I erroneously removed an entry from virtualenv_packages.txt, the build actually succeeded because that entry was still in the .pth file in the virtualenv; instead of "creating" a new virtualenv with the correct .pth files, it just appended the new .pth data to the old, stale data.

I've chosen to address this by completely deleting the entire virtualenv when we try to re-create it. Another way you might solve this problem is by doing a find $VIRTUALENV -name '*.pth' | xargs rm before doing the virtualenv re-creation, but I'm suggesting we do it this way because we have had a long history of difficulty with virtualenv persistence. Bug 1628498 is an obvious example; note that we would never have encountered that bug if we always unconditionally deleted the virtualenv before creating a new one, as in this patch. A patch that is laser-targeted at handling the issue with .pth files might be fine for now but this is more foolproof and future-proof.

Assignee: nobody → rstewart
Status: NEW → ASSIGNED
Pushed by rstewart@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/951942b47734
Delete existing `virtualenv` before re-creating it r=mhentges,ahal
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 81 Branch
See Also: → 1662381
You need to log in before you can comment on or make changes to this bug.