846335 - Add check to prevent missing platforms in l10n-changesets_mobile-*.json

Status: RESOLVED DUPLICATE of bug 1280730

(Release Engineering :: Release Automation, defect, P3)

Description

[John Hopkins (:jhopkins)]

It is too easy to omit android-multilocale from the platforms list in l10n-changesets_mobile-*.json, like what happened here:

 http://hg.mozilla.org/build/buildbot-configs/rev/9b12096e7580

This almost happened during the previous beta release as well but was caught during review.

We should add a check to ship-it or release_sanity that ensures the required platforms are present in these l10n changesets files to avoid broken release builds.

Comment 1

[[e-mail] (:bhearsum)]

The tricky part is doing this without making assumptions that will break in ways that hard to deal with in the future. Eg, beta releases shipping with a significantly different arrangement of platforms than non-betas.

Comment 2

[Rail Aliiev [:rail]]

...or we can load current and previous json files and compare them. Expected changes should be changeset changes and locale additions. Not sure what to do when we add drop new platforms though.

Comment 3

[Rail Aliiev [:rail]]

Another idea is to eliminate human intervention here and use l10n dashboard API (which doesn't exist yet).

Comment 4

[u429623]

An immediate workaround is to add a manual step to the review process to manually open the two links to changesets:
 - from the release under review, and
 - from the prior matching release

Both are clickable links on the review page.

The good news is that this class of failure is very obvious to the pattern matching abilities of the human eye.

The short term automation support could be a "yes no" javascript alert on the "on clicked" for the "reviewed" checkbox for "Have you manually compared the l10n changesets?"

Comment 5

[u429623]

Updated wiki https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It to include manual L10N check

Comment 6

[[e-mail] (:bhearsum)]

Wanted, but not really high priority. It takes two humans missing this to cause a problem, and Hal's doc edit should reduce the possibility of that.