1178302 - figure out what to do with beta numbers in a release promotion world

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[[e-mail] (:bhearsum)]

Currently we have beta numbers embedded into the version of each beta release. Eg, 39.0b1, b2, b3, b4, etc. We never ship two Betas and it's rare that we skip a number.

As of bug 1145175 these are embedded into the product, which is going to make the current numbering system problematic for release promotion. Because we can't change the numbers after the build is completed, we have to make sure that we have a unique beta number when we build. This becomes tricky when we have multiple changes landing in short succession, and we don't know which will be shipping. For example:
* Change A lands at 8am, beta number is already set to 1 in the repo
* Change A builds as 40.0b1
* Change B lands at 11am, beta number is still set to 1
* Change B builds as 40.0b1

It is now impossible to ship both of these builds as unique betas.

Rail, Nick and I talked about a few possible mitigations:
* Bump beta number as part of postrelease. This would mean any changeset that is landed after the previous beta _ships_ would be shippable. Anything that lands prior to the previous beta shipping (even if they land after the previous beta's release builders are complete) are unshippable unless we rebuild with a new beta number.

* Bump the beta number once a day. This would let us ship up to one Beta each day but has the side effect of potentially skipping beta numbers if we don't ship every day.

* Drop the current beta number concept and use buildid instead. This is a bit less user friendly to show in the about dialog, but is the most flexible plan from a technical standpoint. We already have unique, always increasing buildids that are embedded into builds, so any changeset becomes shippable under this plan. We may still need a version like "40.0b1" to stay compatible with ship-it and some update generation stuff, but it would be strictly internal-only (like it was prior to bug 1145175).


It's worth noting that this problem can in theory affect release builds too, but because we so rarely do them in rapid succession we're much much less likely to hit it there. If we do, we can rebuild with a new version number to get a shippable build.

Comment 1

[Sylvestre Ledru [:Sylvestre]]

As a requirement for this, having a simple version name is important. Dealing with buildid to know if a bug is fixed in a specific version is hard.

What about increasing the beta number after a build has been shipped (as part of the post release process)?

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Sylvestre Ledru [:sylvestre] PTO => July 10th from comment #1)
> As a requirement for this, having a simple version name is important.
> Dealing with buildid to know if a bug is fixed in a specific version is hard.
> 
> What about increasing the beta number after a build has been shipped (as
> part of the post release process)?

That is absolutely a possibility, but it means that any changes that land between bumps become unshippable. So, if you start the release promotion for eg, 40.0b5, and then a critical bugfix lands, it would build as 40.0b5. You'd have to bump to 40.0b6 (spend time rebuilding) to ship the critical bugfix.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

One way to mitigate the window of unshippable builds is to bump as soon as a release is started in ship-it. I have a half memory that there is a downside too, but can't recall what it is.

Comment 4

[Rail Aliiev [:rail]]

We talked about possible scenarios with Sylvestre last week and came to following conclusions:

* We want to keep existing versioning schema as is
* Version will be bumped as a part of post release
* We are aware of the scenario mentioned in comment #0 and possible "unshipable" changesets. There is a work around: manually bump the version and deal with possible burnt version
* There will be initial version bump for *beta* as part of merge day automation: version_display will be set to XX.0b1.