do a staging release of 56.0b1 once central => nightly migration is complete

RESOLVED FIXED

Status

Release Engineering
Release Automation
RESOLVED FIXED
6 months ago
5 months ago

People

(Reporter: mtabara, Assigned: mtabara)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox56 fixed, firefox57 fixed)

Details

(Assignee)

Description

6 months ago
We talked about doing a staging release next Wednesday, after nightly => beta migration is complete. We need to ensure everything works smoothly and tree is stabilized before real beta 1 Gtb on 7th August.

This bug is to track that work and any other related testing.
(Assignee)

Updated

6 months ago
Depends on: 1385248
(Assignee)

Updated

6 months ago
Assignee: nobody → mtabara
(Assignee)

Comment 1

6 months ago
* Past staging release we've done this week were pre-mergeday so were not useful.
* We've had some issues with patcher configs file in the fake tools repo but we've mocked that.
* we also needed to land some sha384 env var change to fix the signing certs - https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530
* last build so far is build 23 - https://tools.taskcluster.net/groups/hXY4JB_rRa-5HlNeVlrbRQ
(Assignee)

Comment 4

5 months ago
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #3)
> We grafted this change from central => jamun
> https://hg.mozilla.org/projects/jamun/rev/
> 4e3ca84ed0bf652e42a8366acf280594ea383d21.

Note to self:
* we need to uplift this patch to beta
(Assignee)

Comment 5

5 months ago
Pushed a bunch of patcher configs amends hoping to fix the updater builder. No results so far:
* https://hg.mozilla.org/users/jlorenzo_mozilla.com/tools-fake/rev/933236787936
* https://hg.mozilla.org/users/jlorenzo_mozilla.com/tools-fake/rev/5d11ae3ea03b
(Assignee)

Comment 6

5 months ago
We eventually fixed half of patcher configs as it now tags correctly the tools-fake repo but we're getting some issues in making it communicate with Balrog. Sounds like this is of broader scope as I couldn't find a working updates builder Rail's past 5-10 staging releases. IIRC he did mention we have troubles with that. Might be worth investigating later this week but this should not block beta1 as nothing changed in that logic during 55 so we should be good.
(Assignee)

Comment 7

5 months ago
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)
> Note to self:
> * need to pull this on rr-bm85 as well
> https://github.com/mozilla-releng/releasetasks/commit/
> 2e4258a4e84356cc7d4707b79c72646cfa353530

Done
(Assignee)

Comment 8

5 months ago
We need to decide if we still go with the following:

On inbound/beta:

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)
> * need to graft
> https://hg.mozilla.org/projects/jamun/rev/
> e8539ac81ed8a65a13bcc01e17376687c521d51f on central/beta as well

Uplift to beta:

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #3)
> We grafted this change from central => jamun
> https://hg.mozilla.org/projects/jamun/rev/
> 4e3ca84ed0bf652e42a8366acf280594ea383d21.

Comment 10

5 months ago
uplift
https://hg.mozilla.org/releases/mozilla-beta/rev/e88f8b00ea43
status-firefox56: --- → fixed
status-firefox57: --- → fixed
(Assignee)

Comment 11

5 months ago
Bah, so sad to see this eventually affected 56.0b1-build1. We thought we'd risk a build2 if we land all lzma/sha384 or build2 if we don't land it. We went with latter but got unlucky. 

Note to self: lesson learned: if jamun is merged from beta, whatever patches we add there and work for jamun, uplift/land on beta without doubts.
(Assignee)

Comment 12

5 months ago
Was looking at :aki's releasetasks patches and trying to figure out how come that bit us in build2 and [1] was needed.

Turns out on jamun we only got so far with "[funsize] Update generating task {{ platform }} {{ locale }} for {{ partial_version }}" tasks which got solved by [2] but didn't get to "[funsize] Publish to Balrog {{ platform }} {{ locale }} for {{ partial_version }}" for Balrog reasons or whatsoever so we missed seeing that _SHA1 + SHA384 needed for "[funsize] Publish to Balrog {{ platform }} {{ locale }} for {{ partial_version }}" jobs as well.

[1]: ttps://github.com/mozilla-releng/releasetasks/commit/85f8b39206c8ad00359bc47159e8d36841d00dee
[2]: https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530
(Assignee)

Comment 13

5 months ago
Note to self: why we needed build2:

RyanVM> https://hg.mozilla.org/mozilla-central/rev/ef9a0f01e4f6 went in before the uplift
00:29:57 <RyanVM> https://hg.mozilla.org/mozilla-central/rev/e1b93d2abdb2 after
00:30:04 <RyanVM> wait, no
00:30:06 <RyanVM> oh yeah
00:30:13 <aki> we might want both those patches then
00:30:13 <RyanVM> want me to uplift ^
00:30:30 <aki> yes please, then we can try a build 2. if that doesn't work, we may want to back out all the sha384 stuff
00:30:37 <RyanVM> we already did I thought
00:31:08 <RyanVM> bug 1387231
00:31:36 <aki> i don't think that included simon's funsize patches referenced above
00:31:46 <RyanVM> ah no, it did not
00:31:49 <RyanVM> just 3 bugs
00:32:47 <aki> i'm not that familiar with these -- i think these changes are supposed to work with both sha1 and sha384, but i don't know if it's better to fail forward or roll back
00:32:55 <RyanVM> ok, I pushed it
00:32:57 <RyanVM> we'll see what happens
00:33:34 <@nthomas> so even though we're not using lzma for b1, we have support in funsize but didn't bring everything up to beta ?
00:33:53 <aki> RyanVM: did you get the https://hg.mozilla.org/projects/jamun/rev/e8539ac81ed8 fix?
00:34:54 <aki> nthomas: i think so. we landed support, then did a partial rollback and additional patches in jamun to try to get staging b1 working
00:35:05 <@nthomas> fun
00:35:13 <RyanVM> aki: ugh, that never landed on m-c AFAICT
00:35:28 <aki> RyanVM: yeah. but it fixes the import error
00:35:29 — @nthomas adds deved to rw in the meantime
00:35:47 <RyanVM> aki: i'll push it to Beta, but yeah, it'll need pushing to inbound at some point too
00:36:14 <aki> ok
00:36:31 <RyanVM> heh, mtabara covered this in bug 1385134
00:37:14 <aki> yeah, we were thinking we were safe since https://hg.mozilla.org/projects/jamun/rev/4e3ca84ed0bf didn't land in beta
00:37:28 <aki> but the first half of the patches landed, so we needed to either clean up fully or fix fully
00:38:26 <RyanVM> done and done :)
(Assignee)

Comment 14

5 months ago
Note to self, why we needed a 56.0b1 Firefox build3:

@nthomas> I don't see an error in https://mozilla-release-logs.s3.amazonaws.com/mozilla-beta/firefox-56.0b1/build2/%5Bfunsize%5D_Publish_to_Balrog_linux64_en-US_for_55.0b11-linux64-1Z3wokpiRZu1PjI2oVzk-w-0
03:14:41 <@nthomas> other than a lack of work being done
03:15:09 <@nthomas> unless '+ test' is supposed to have a value there
03:15:49 <@nthomas> https://hg.mozilla.org/releases/mozilla-beta/file/default/taskcluster/docker/funsize-balrog-submitter/runme.sh#l7 perhaps
03:17:28 <@nthomas> and bm85:~cltbld/releasetasks is on the tip of the repo
03:17:38 <@nthomas> so it has https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530
03:18:53 <@nthomas> but H-W3huzcSbGmyyK-Zuq4yw doesn't have SHA1_SIGNING_CERT or SHA384_SIGNING_CERT set, just SIGNING_CERT
03:22:36 <@nthomas> can't really explain that, unless someone else updated the releasetasks checkout after the releases started off
03:23:58 <@nthomas> naturally enough, I need to head out soon
03:34:35 <aki> i think we're missing the same env var maybe
03:34:56 <aki> we may need https://github.com/mozilla-releng/releasetasks/commit/2e4258a4e84356cc7d4707b79c72646cfa353530 type stuff for the balrog tasks
03:41:43 <aki> which means we need a build3
03:43:17 <@nthomas> ah yes, that makes sense. would have hit this in staging if there wasn't issues talking to an instance
03:43:50 <aki> got time for an r? https://github.com/mozilla-releng/releasetasks/pull/261
03:43:54 <aki> yeah
03:45:50 <@nthomas> r+

Comment 15

5 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/a489d77d03d6
Status: NEW → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED
(Assignee)

Updated

5 months ago
See Also: → bug 1388357
(Assignee)

Updated

5 months ago
Blocks: 1388357
See Also: bug 1388357
You need to log in before you can comment on or make changes to this bug.