650305 - release sanity should check that l10n-changesets from L10n dashboard match

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

If we are doing a normal release we should check that we have taken the L10n changes and if they don't match notify us so we can take action. If it is a chemspill we could pass a parameter to ignore l10n changes.

For example, for 3.6.17 these should match:
https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx3.6.17
http://hg.mozilla.org/build/buildbot-configs/raw-file/FIREFOX_3_6_17_BUILD2/mozilla/l10n-changesets_mozilla-1.9.2
Notice I included the tag as the revision.

Comment 1

[Axel Hecht [:Pike]]

Note, in the future, the milestone to ship will not exist unless releng actually triggered the data generation piece through "ship it" or "chemspill" (names to change). Also, the data format might convert over to json proper, merging l10n-changesets and shipped-locales.

To go back to https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial/Known_Issues:

Forget about taking L10n changes

    Preempt: We should not allow reviews for releases without L10n changes
    Recover: Cancel every job that is currently running for your release and start with another build #. 

I wonder if there's an api you want on the dashboard for this, or, how to phrase this in a way that works across systems (releng/l10n).

I wouldn't side track chemspills, but try to let them follow the same scheme. I.e., verify that the milestone exists, and that it has the data you expect.

Not sure if you want to check for a difference between the revision in l10n-changesets and the actual built revision in the builds?

Comment 2

[Chris AtLee [:catlee]]

Axel, what we're trying to do right now is to have a simple check that whoever is driving the release has synced up the l10n changesets from the dashboard.

As the dashboard is currently implemented, we can fetch the list from, e.g. https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx3.6.17 and compare against our l10n changesets, right?

It will be up to the releng person doing the release whether to ignore the error if the changesets differ (e.g. for chemspills).

Is "https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx%(version)s" the correct place to look currently?

Comment 3

[Anamaria Stoica [:anamarias]]

Attachment: Checks the l10n-changesets against the l10n dashboard

@catlee: let me know if anything needs changing

Comment 4

[Axel Hecht [:Pike]]

For context, the current dashboard code is not going to work for the new release model, we need to make some pretty involved changes. I hope to get some progress made on that next week, we're going to have a face-to-face. Those changes are drafted for the data model, but what that means for URLs, not sure yet. That's covered in bug 650816.

https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?av=fx4.0 might be more stable, though that should turn out wrong for chemspills. Not committing to that URL being stable yet, though.

When milestones move to "create on demand", releng is going to determine the name of the milestone, and you may want to include the build number, to work with "we need to respin for bs" or so.

Verifying that the changesets match doesn't imply that the dashboard got the right knobs hit to not break the next release, too. In particular, there have been a few releases that got the data, but not the final button press.

PS: You should write this code such that it deals gracefully with network errors, to deal with situations like last night, when IT takes down the current setup, and doesn't bring it back up in time.

Comment 5

[Chris AtLee [:catlee]]

Comment on attachment 527666 [details] [diff] [review]
Checks the l10n-changesets against the l10n dashboard

Looks like there are two nearly identical patches here? The first set looks fine.

Comment 6

[Anamaria Stoica [:anamarias]]

Attachment: cleaned up previous patch to contain just one diff