1397679 - [push-apk] Fennec beta: Use staged rollout only on the first betas of the cycle

Status: RESOLVED WORKSFORME

(Release Engineering :: Release Automation: Other, enhancement, P1)

Description

[Johan Lorenzo [:jlorenzo]]

After aurora was turned off, we decided to gradually ship beta thanks to Google Play's rollout track (bug 1358420). At that time, the plan was to switch staged rollout always on, on central so that each merge m-c -> m-b reset the flag back to on. Then, a human had to land an "opposite patch" to the one in bug 1358420.

It worked fine in the 55 cycle (when the system was first set up). However, in 56, we encountered bug 1395279: 56.0b{1,3,5,7} were pushed as staged rollout. Release management had manually fully rolled out before the next release came. Release engineering hadn't landed the "opposite patch". 

Thus, the plan in bug 1358420 doesn't work well. :mtabara suggested to let the taskgraph determine that the Google Play track depending on the beta number. For instance:
> * b1 => rollout track, 10%. Release management manually moves b1 to the production track, before b3 is out.
> * b3 => Same as b1
> * b5 and beyond => production track (which means all users will get it). No further action required from release management, until next b1. 

For reference, today, we ship Fennec betas once a week (unlike desktop, which is twice a week), which means there is no even beta numbers.

Liz, Julien, does this plan sound good to you?

Comment 1

[Julien Cristau [:jcristau]]

I'm not convinced basing this on beta number is flexible enough.

Comment 2

[Johan Lorenzo [:jlorenzo]]

I just chatted with :jcristau IRL. We both agreed this is not a good idea to implement this for the 57 cycle. Reasons:
1. Dawn happened 1 cycle ago. This means 56 is the second beta crafted fresh out central. Release management doesn't have a clear estimate about what beta is sane enough to be pushed without any rollback possible[1]. 
2. 57 has a special schedule. 57.0b1 will be desktop-only. Hence, the first Fennec beta will be b3, which blows the example (in comment 0) up.

We might want to implement the logic after 57 is out, but that's a call to make later. In the meantime, both Julien and I will update RelMan/RelEng documentations. On my end, I will add special requirements in release warrior and file corresponding bugs.


[1] For the record, there is no way to take an APK out once it's on the production track https://johanlorenzo.github.io/blog/2017/06/12/part-3-how-mozilla-publishes-apks-onto-google-play-store-in-a-reasonably-secure-and-automated-way.html#there-is-no-way-to-rollback-to-previous-apks-even-if-you-ask-a-human

Comment 3

[Johan Lorenzo [:jlorenzo]]

Documentation left in bug 1397684 and bug 1397688. Reminders left in releasewarrior at https://github.com/mozilla/releasewarrior/commit/0a0bf1e1b6c9946254cd8dbad02db90dc478eb40

Comment 4

[Liz Henry (:lizzard) (relman/hg->git project)]

Sounds ok to me, we can stick with how we're doing this for 57 and re-visit the process for 58.

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

I think work for this happened in other bugs. 
Please feel free to reopen if I missed anything.