Closed Bug 1385134 Opened 8 years ago Closed 8 years ago

do a staging release of 56.0b1 once central => nightly migration is complete

Categories

(Release Engineering :: Release Automation, enhancement)

Product:
Release Engineering
Component:
Release Automation
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox56 fixed, firefox57 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox56 --- fixed
firefox57 --- fixed

People

(Reporter: mtabara, Assigned: mtabara)

References

Details

We talked about doing a staging release next Wednesday, after nightly => beta migration is complete. We need to ensure everything works smoothly and tree is stabilized before real beta 1 Gtb on 7th August. This bug is to track that work and any other related testing.
Depends on: 1385248
Assignee: nobody → mtabara
* Past staging release we've done this week were pre-mergeday so were not useful. * We've had some issues with patcher configs file in the fake tools repo but we've mocked that. * we also needed to land some sha384 env var change to fix the signing certs - https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530 * last build so far is build 23 - https://tools.taskcluster.net/groups/hXY4JB_rRa-5HlNeVlrbRQ
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #3) > We grafted this change from central => jamun > https://hg.mozilla.org/projects/jamun/rev/ > 4e3ca84ed0bf652e42a8366acf280594ea383d21. Note to self: * we need to uplift this patch to beta
We eventually fixed half of patcher configs as it now tags correctly the tools-fake repo but we're getting some issues in making it communicate with Balrog. Sounds like this is of broader scope as I couldn't find a working updates builder Rail's past 5-10 staging releases. IIRC he did mention we have troubles with that. Might be worth investigating later this week but this should not block beta1 as nothing changed in that logic during 55 so we should be good.
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2) > Note to self: > * need to pull this on rr-bm85 as well > https://github.com/mozilla-releng/releasetasks/commit/ > 2e4258a4e84356cc7d4707b79c72646cfa353530 Done
We need to decide if we still go with the following: On inbound/beta: (In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2) > * need to graft > https://hg.mozilla.org/projects/jamun/rev/ > e8539ac81ed8a65a13bcc01e17376687c521d51f on central/beta as well Uplift to beta: (In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #3) > We grafted this change from central => jamun > https://hg.mozilla.org/projects/jamun/rev/ > 4e3ca84ed0bf652e42a8366acf280594ea383d21.
Bah, so sad to see this eventually affected 56.0b1-build1. We thought we'd risk a build2 if we land all lzma/sha384 or build2 if we don't land it. We went with latter but got unlucky. Note to self: lesson learned: if jamun is merged from beta, whatever patches we add there and work for jamun, uplift/land on beta without doubts.
Was looking at :aki's releasetasks patches and trying to figure out how come that bit us in build2 and [1] was needed. Turns out on jamun we only got so far with "[funsize] Update generating task {{ platform }} {{ locale }} for {{ partial_version }}" tasks which got solved by [2] but didn't get to "[funsize] Publish to Balrog {{ platform }} {{ locale }} for {{ partial_version }}" for Balrog reasons or whatsoever so we missed seeing that _SHA1 + SHA384 needed for "[funsize] Publish to Balrog {{ platform }} {{ locale }} for {{ partial_version }}" jobs as well. [1]: ttps://github.com/mozilla-releng/releasetasks/commit/85f8b39206c8ad00359bc47159e8d36841d00dee [2]: https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530
Note to self: why we needed build2: RyanVM> https://hg.mozilla.org/mozilla-central/rev/ef9a0f01e4f6 went in before the uplift 00:29:57 <RyanVM> https://hg.mozilla.org/mozilla-central/rev/e1b93d2abdb2 after 00:30:04 <RyanVM> wait, no 00:30:06 <RyanVM> oh yeah 00:30:13 <aki> we might want both those patches then 00:30:13 <RyanVM> want me to uplift ^ 00:30:30 <aki> yes please, then we can try a build 2. if that doesn't work, we may want to back out all the sha384 stuff 00:30:37 <RyanVM> we already did I thought 00:31:08 <RyanVM> bug 1387231 00:31:36 <aki> i don't think that included simon's funsize patches referenced above 00:31:46 <RyanVM> ah no, it did not 00:31:49 <RyanVM> just 3 bugs 00:32:47 <aki> i'm not that familiar with these -- i think these changes are supposed to work with both sha1 and sha384, but i don't know if it's better to fail forward or roll back 00:32:55 <RyanVM> ok, I pushed it 00:32:57 <RyanVM> we'll see what happens 00:33:34 <@nthomas> so even though we're not using lzma for b1, we have support in funsize but didn't bring everything up to beta ? 00:33:53 <aki> RyanVM: did you get the https://hg.mozilla.org/projects/jamun/rev/e8539ac81ed8 fix? 00:34:54 <aki> nthomas: i think so. we landed support, then did a partial rollback and additional patches in jamun to try to get staging b1 working 00:35:05 <@nthomas> fun 00:35:13 <RyanVM> aki: ugh, that never landed on m-c AFAICT 00:35:28 <aki> RyanVM: yeah. but it fixes the import error 00:35:29 — @nthomas adds deved to rw in the meantime 00:35:47 <RyanVM> aki: i'll push it to Beta, but yeah, it'll need pushing to inbound at some point too 00:36:14 <aki> ok 00:36:31 <RyanVM> heh, mtabara covered this in bug 1385134 00:37:14 <aki> yeah, we were thinking we were safe since https://hg.mozilla.org/projects/jamun/rev/4e3ca84ed0bf didn't land in beta 00:37:28 <aki> but the first half of the patches landed, so we needed to either clean up fully or fix fully 00:38:26 <RyanVM> done and done :)
Note to self, why we needed a 56.0b1 Firefox build3: @nthomas> I don't see an error in https://mozilla-release-logs.s3.amazonaws.com/mozilla-beta/firefox-56.0b1/build2/%5Bfunsize%5D_Publish_to_Balrog_linux64_en-US_for_55.0b11-linux64-1Z3wokpiRZu1PjI2oVzk-w-0 03:14:41 <@nthomas> other than a lack of work being done 03:15:09 <@nthomas> unless '+ test' is supposed to have a value there 03:15:49 <@nthomas> https://hg.mozilla.org/releases/mozilla-beta/file/default/taskcluster/docker/funsize-balrog-submitter/runme.sh#l7 perhaps 03:17:28 <@nthomas> and bm85:~cltbld/releasetasks is on the tip of the repo 03:17:38 <@nthomas> so it has https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530 03:18:53 <@nthomas> but H-W3huzcSbGmyyK-Zuq4yw doesn't have SHA1_SIGNING_CERT or SHA384_SIGNING_CERT set, just SIGNING_CERT 03:22:36 <@nthomas> can't really explain that, unless someone else updated the releasetasks checkout after the releases started off 03:23:58 <@nthomas> naturally enough, I need to head out soon 03:34:35 <aki> i think we're missing the same env var maybe 03:34:56 <aki> we may need https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530 type stuff for the balrog tasks 03:41:43 <aki> which means we need a build3 03:43:17 <@nthomas> ah yes, that makes sense. would have hit this in staging if there wasn't issues talking to an instance 03:43:50 <aki> got time for an r? https://github.com/mozilla-releng/releasetasks/pull/261 03:43:54 <aki> yeah 03:45:50 <@nthomas> r+
https://hg.mozilla.org/mozilla-central/rev/a489d77d03d6
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
See Also: → 1388357
Blocks: 1388357
See Also: 1388357 →
You need to log in before you can comment on or make changes to this bug.