perform staging releases in preparation for 2019-01-29
Categories
(Release Engineering :: General, enhancement)
Tracking
(Not tracked)
People
(Reporter: mtabara, Assigned: mtabara)
References
Details
This is to track potential issues for the staging releases that I'm currently running as part of the 2019-01-29 upcoming mergeduty.
|
Assignee | |
Comment 1•6 years ago
|
||
There's two staging releases that I attempted to run:
the future 66.0b1
Still has issues in Ship-ip with setting up partials. Seems like even though I trim the entries to just one, it still includes the default list which is auto-completed by Ship-it UI. There could be a bug in Ship-it side. I'll try again with Devedeition as initially I attempted a Firefox.
The treeherder tracking job is here while action task failing is here
the future 65.0RC
This one worked slightly smoother. Except two things:
-
bouncer submission job is failing. More here but the corrupt entries that it's been complaining about is because the bouncer submission job submits
"paths_per_bouncer_platform": { "win": "/firefox/releases/65.0/win32/:lang/Firefox%20Setup%2065.0.msi", "win64": "/firefox/releases/65.0/win64/:lang/Firefox%20Setup%2065.0.msi" }
while bouncer returns
/firefox/releases/65.0/win32/:lang/Firefox%20Installer.msi
/firefox/releases/65.0/win64/:lang/Firefox%20Installer.msi
There seems to be an inconsistency here with respect to the naming. I remember Callek pushing a patch to fix this a while ago so my first guess is that the naming was done correctly for beta but potentially not done for RC? Doubtful though since we have a single kind in taskcluster for all but I'll investigate further.
- UV is failing completely in the graph. Not sure if this is expected.
|
|
Assignee | |
Comment 2•6 years ago
|
||
@tom: I haven't touched UV for a long time. Are these expected to fail for staging releases or could it be because of the partials?
|
||
Comment 3•6 years ago
|
||
The UV failing is https://hg.mozilla.org/mozilla-central/rev/4192b6560bf7 (which only affects staging and wasn't uplifted to beta). If you look at the errors, they are https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/common/updatererrors.h#38 (at least the one I checked was).
It looks like the bouncer entries already exists (from a previous staging run?) and have the old values.
|
|
Assignee | |
Comment 4•6 years ago
|
||
(In reply to Tom Prince [:tomprince] from comment #3)
The UV failing is https://hg.mozilla.org/mozilla-central/rev/4192b6560bf7 (which only affects staging and wasn't uplifted to beta). If you look at the errors, they are https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/common/updatererrors.h#38 (at least the one I checked was).
That makes sense, I'll ignore this then.
It looks like the bouncer entries already exists (from a previous staging run?) and have the old values.
Your scenario makes much more sense. I wiped off the locations and the product, I'll be starting a new release. I expect it to be green this time.
Thanks for your help!
|
|
Assignee | |
Comment 5•6 years ago
|
||
Latest set of staging releases are now:
- 66.0b1 with e5e5176e7fb8a9e69eb058efb8a24d00f0d63a22 on try and treeherder job
- 65.0 with 4ee312fd2454ee1f708cd0e020adb33a7c2e12d1 on try and treeherder job
|
|
Assignee | |
Comment 6•6 years ago
|
||
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #4)
Your scenario makes much more sense. I wiped off the locations and the product, I'll be starting a new release. I expect it to be green this time.
Follow-up 65.0 try staging release's bouncer submission job worked smoothly after having a clean state on bouncer side.
|
|
Assignee | |
Comment 7•6 years ago
|
||
I may do another set tomorrow morning Friday just to make sure things are fine and ready for Monday.
Or maybe just a 66.0b1 which was slightly troublous on windows-side earlier today.
|
|
Assignee | |
Comment 8•6 years ago
|
||
Calling this done. I think we're in good shape for Monday.
I tried to create another 66.0b1 today but failed from Ship-it side when it attempted to "promote_firefox". That's because of bug 1496890 and its https://hg.mozilla.org/integration/autoland/rev/aee9f213f3c7 which changed the .taskcluster structure.
Rail is attempting a fix on Ship-it side (enrich the jsone context there to encompass the newly added clientId) so that when central -> beta on Monday, we won't be hitting this issue on production Ship-it too.
Good thing we have these try staging releases, thanks again Tom for the work in making them available.
|
|
Assignee | |
Comment 9•6 years ago
|
||
Rail fixed the bug on ship-it side, we should have a good state on Monday for upcoming production 66.0b1
Description
•