Closed
Bug 1290516
Opened 8 years ago
Closed 8 years ago
Decision and action tasks should share workspace cache across {{project}}s
Categories
(Firefox Build System :: Task Configuration, task)
Firefox Build System
Task Configuration
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: gps, Assigned: gps)
References
Details
Attachments
(1 file)
Explanation forthcoming in a commit message.
Assignee | ||
Comment 1•8 years ago
|
||
Currently, the gecko-decision and action tasks maintain a separate cache for each "project" (Firefox repo) source checkout. In my opinion, this is not necessary. Separate caches/source checkouts mean a lower chance for a cache hit: a lower chance that any given worker will have a source checkout readily available for the "project" being built. They also mean workers store more data and files in caches. By using a shared cache for source checkouts, workers will ideally only have a single source checkout cache and will be pretty much guaranteed to have a checkout readily available (the main exception being the first task on a freshly-provisioned worker). A downside of a shared cache is that workers may spend more time doing `hg update` operations. e.g. if the last task was from mozilla-release and the current task is from mozilla-central, thousands of files will be updated. However, I believe we will still be better off because separate caches *increase* the probability for a full checkout, which involves writing ~150,000 files! So I believe this change will lower the overall amount of VCS work done by workers while possibly sacrificing the lowest possible execution time (which is achieved by minimizing the `hg update` work). It's worth noting that `hg robustcheckout` (which is used by these tasks) does the equivalent of an `hg purge --all`. This ensures that all untracked and ignored files from previous tasks are deleted. So there should be no contamination from previous tasks. Review commit: https://reviewboard.mozilla.org/r/67984/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/67984/
Attachment #8776053 -
Flags: review?(dustin)
Comment 2•8 years ago
|
||
Comment on attachment 8776053 [details] Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; https://reviewboard.mozilla.org/r/67984/#review65110
Attachment #8776053 -
Flags: review?(dustin) → review+
Pushed by dmitchell@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/f351e9f3e564 Remove {{project}} from cache path for decision and action tasks; r=dustin
Comment 4•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/f351e9f3e564
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Updated•6 years ago
|
Product: TaskCluster → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•