Closed Bug 637833 Opened 9 years ago Closed 9 years ago
Split up final verification
I am not sure if by platform or by a number of locales but it takes too much time right now if this builder has any value on finishing before we release. Feel free to close this or modify if I don't understand this correctly.
I started a couple of jobs and each one took 20 hours and 17 hours.
Could you look up the slavenames for those jobs ? Did they hit VMs ? How does the elapsed time compare to previous releases for the same branch ? It's possible that something in the chain is much slower now. We * query AUS for an update * do a HEAD request on the mar url (download.m.o) * get redirected to a mirror and do a HEAD there We know that AUS changed to be MPT and PHX prior to the release, previously it was just MPT. The PHX webheads may be slower than MPT, and DNS is now 3 steps to get to the colo-specific load balancer: aus2.mozilla.org -> aus2.3crowd.mozilla.net (CNAME) -> aus01.zlb.phx.mozilla.net or aus2.acelb.sj.mozilla.com (CNAME) -> IP where previously we had aus2.m.o -> aus2-mozilla-org.geo.mozilla.com (CNAME) -> IP There's lots of other candidates - download.m.o may have been slow, or the mirrors themselves, or ESX buried by load.
The slaves for the first two were moz2-linux-slave26 and moz2-linux-slave19. There is no previous logs for that builders on pm01 and pm03. The run of today on IX machine took 5 hrs, 13 mins, 21 secs.
OS: Mac OS X → All
Priority: -- → P5
Hardware: x86 → All
This patch splits final verification per platform. ReleaseFinalVerification class stays backward compatible. The builders tested in staging and worked as expected. master-dump also shows expected changes for the list of builders triggered by mirror uptake monitoring poller.
Attachment #521323 - Flags: review?(catlee)
Comment on attachment 521323 [details] [diff] [review] split per platform http://hg.mozilla.org/build/buildbotcustom/rev/76a6cfcddf47
Attachment #521323 - Flags: checked-in+
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.