Closed Bug 1354625 Opened 2 years ago Closed 2 years ago

Not all VCS interactions using mozilla-unified upstream with robustcheckout

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(firefox-esr45 fixed, firefox-esr52 fixed, firefox53 fixed, firefox54 fixed, firefox55 fixed)

RESOLVED FIXED
Tracking Status
firefox-esr45 --- fixed
firefox-esr52 --- fixed
firefox53 --- fixed
firefox54 --- fixed
firefox55 --- fixed

People

(Reporter: gps, Assigned: gps)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

Automation is currently seeing lots of failures like those in https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-linux64-add-on-devel/1491563674/mozilla-beta-linux64-add-on-devel-bm74-build1-build65.txt.gz.

I'm still not sure why the Mercurial operations are failing and I'm committed to getting to the root cause. But I've identified a mitigation.

Currently, some mozharness jobs that perform a Firefox VCS clone aren't using the --upstream feature of robustcheckout. This means that the Mercurial repo in the shared store may initially be populated with e.g. mozilla-release - a repo thousands of changesets behind beta, aurora, central, etc. When the machine processes an e.g. beta job, it has to `hg pull` thousands of changesets for the beta repo. This takes extra time and adds significant load to the server.

--upstream with the mozilla-unified repo causes the initial clone to be seeded with a superset of changesets from all the repos. This means that any subsequent `hg pull` should be minimal. A minimal pull maximizes the changes that it will succeed.

We should add --upstream https://hg.mozilla.org/mozilla-unified to all of the jobs and clobber all the builders to force a mozilla-unified clone to be present.
Comment on attachment 8855890 [details]
Bug 1354625 - Use mozilla-unified for upstream for all Firefox builds;

https://reviewboard.mozilla.org/r/127766/#review130472
Attachment #8855890 - Flags: review?(aki) → review+
Comment on attachment 8855889 [details]
Bug 1354625 - Use mozilla-unified for single_local try builds upstream;

https://reviewboard.mozilla.org/r/127764/#review130468
Attachment #8855889 - Flags: review?(aki) → review+
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c92bc314c18b
Use mozilla-unified for single_local try builds upstream; r=aki
https://hg.mozilla.org/integration/autoland/rev/508dca13e963
Use mozilla-unified for upstream for all Firefox builds; r=aki
Blocks: 1291926
No longer depends on: 1348637
A spot checked some logs and this seems to have the desired effect! I love paving over problems. That leaves bug 1291926 for me to address the root cause...
See Also: → 1355404
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.