Last Comment Bug 1049550 - create balrog rules to properly support OS blocking and watershed updates on beta (and equivalent test) channel
: create balrog rules to properly support OS blocking and watershed updates on ...
Status: RESOLVED FIXED
:
Product: Release Engineering
Classification: Other
Component: Balrog: Backend (show other bugs)
: unspecified
: All All
-- normal (vote)
: ---
Assigned To: Nick Thomas [:nthomas]
: Ben Hearsum (:bhearsum)
: Chris AtLee [:catlee]
Mentors:
Depends on: 1021022 1065857
Blocks:
  Show dependency treegraph
 
Reported: 2014-08-06 07:22 PDT by Ben Hearsum (:bhearsum)
Modified: 2014-09-17 16:01 PDT (History)
0 users
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
9.0 beta urls for testing (289.98 KB, text/plain)
2014-09-03 02:14 PDT, Nick Thomas [:nthomas]
no flags Details
9.0 beta urls for testing (279.90 KB, text/plain)
2014-09-03 02:17 PDT, Nick Thomas [:nthomas]
no flags Details

Description User image Ben Hearsum (:bhearsum) 2014-08-06 07:22:52 PDT
Per the post mortem yesterday, filing this for you Nick.

If we go with the channel names in bug 986900, I think we can get away with one rule for beta and its test channels by using "beta*" as the channel in the rule.
Comment 1 User image Nick Thomas [:nthomas] 2014-08-26 06:06:43 PDT
Watersheds should be at 29.0b8 and 11.0b5. Also present (but of less interest) is assorted mess below 4.0b7, and in the case of more then one build (ie build1 points to build2 rather than what build2 points). More info at http://people.mozilla.org/~nthomas/update-watch/release/.

There's a few OS deprecations in http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/inc/config-dist.php#601 to catch too.

(In reply to Ben Hearsum [:bhearsum] from comment #0)
> If we go with the channel names in bug 986900, I think we can get away with
> one rule for beta and its test channels by using "beta*" as the channel in
> the rule.

Bug 986990 ? I forget how we were going to handle test channels pointing at a new release in this scenario. Probably still need two rules for that, ie beta -> latest, beta-*test -> in progress, and some priority to make it work.
Comment 2 User image Ben Hearsum (:bhearsum) 2014-09-02 09:55:58 PDT
(In reply to Nick Thomas [:nthomas] from comment #1)
> (In reply to Ben Hearsum [:bhearsum] from comment #0)
> > If we go with the channel names in bug 986900, I think we can get away with
> > one rule for beta and its test channels by using "beta*" as the channel in
> > the rule.
> 
> Bug 986990 ? I forget how we were going to handle test channels pointing at
> a new release in this scenario. Probably still need two rules for that, ie
> beta -> latest, beta-*test -> in progress, and some priority to make it work.

I think we might be talking about different things...I was thinking that we only need a single rule per watershed update on all beta + beta test channels. We definitely need separate rules for the non-watershed (ie, latest) updates.

We already have 3 rules for beta -> latest updates (beta, betatest, releasetest). The releasetest one should get changed to use "beta-cdntest" for a channel (not strictly part of this bug, but we could do it here I suppose), but these are otherwise okay AFAICT.

As part of this bug I _think_ we need two new rules:
1) product=Firefox, channel=beta*, version<11.0b5, priority=95, mapping=Firefox-11.0b5-build1
2) product=Firefox, channel=beta*, version<29.0b8, priority=93, mapping=Firefox-29.0b8-build1

The priorities are important here. Eg, 10.0b2 will get caught by the first rule but 15.0b1 will fall through to the second. The priority on the existing "beta" rule is 90, so 29.0b8+ will fall through to it.
Comment 3 User image Nick Thomas [:nthomas] 2014-09-03 02:14:10 PDT
Created attachment 8483346 [details]
9.0 beta urls for testing

Ah, that's a good idea. Made a slight tweak on the rules when adding them
1) product=Firefox, channel=beta*, version<11.0, priority=95, mapping=Firefox-11.0b5-build1
2) product=Firefox, channel=beta*, version<29.0, priority=93, mapping=Firefox-29.0b8-build1
to match what we have on aus3.

The attached file is the six 9.0b releases, all platforms & locales, and beta channel. I switched from betatest, because betatest != beta on aus3 (booo).

I've submitted a Firefox-11.0b5-build1, and ran the comparions but got 
* some capitalization of Firefox wrong in bouncer product, & betatest fileUrl
* detailsUrl has en-US instead of %LOCALE%
* ?force=1 isn't being passed on by balrog
* there's a bunch of 32.0 rc1 differences (displayVersion, URL), which will go away once 32.0b1 is in both update servers

aus4-admin isn't letting me delete the 11.0b5 release to re-add it, so need to track the bug with that down.
Comment 4 User image Nick Thomas [:nthomas] 2014-09-03 02:17:29 PDT
Created attachment 8483349 [details]
9.0 beta urls for testing

The correct file.
Comment 5 User image Ben Hearsum (:bhearsum) 2014-09-03 07:45:29 PDT
(In reply to Nick Thomas [:nthomas] from comment #3)
> Created attachment 8483346 [details]
> 9.0 beta urls for testing
> 
> Ah, that's a good idea. Made a slight tweak on the rules when adding them
> 1) product=Firefox, channel=beta*, version<11.0, priority=95,
> mapping=Firefox-11.0b5-build1
> 2) product=Firefox, channel=beta*, version<29.0, priority=93,
> mapping=Firefox-29.0b8-build1
> to match what we have on aus3.
> 
> The attached file is the six 9.0b releases, all platforms & locales, and
> beta channel. I switched from betatest, because betatest != beta on aus3
> (booo).
> 
> I've submitted a Firefox-11.0b5-build1, and ran the comparions but got 
> * some capitalization of Firefox wrong in bouncer product, & betatest fileUrl
> * detailsUrl has en-US instead of %LOCALE%
> * ?force=1 isn't being passed on by balrog
> * there's a bunch of 32.0 rc1 differences (displayVersion, URL), which will
> go away once 32.0b1 is in both update servers
> 
> aus4-admin isn't letting me delete the 11.0b5 release to re-add it, so need
> to track the bug with that down.

This is bug 1035293 :(. We might need IT to delete this one by hand for us in the meantime...
Comment 6 User image Ben Hearsum (:bhearsum) 2014-09-03 07:50:35 PDT
(In reply to Ben Hearsum [:bhearsum] from comment #5)
> (In reply to Nick Thomas [:nthomas] from comment #3)
> > Created attachment 8483346 [details]
> > 9.0 beta urls for testing
> > 
> > Ah, that's a good idea. Made a slight tweak on the rules when adding them
> > 1) product=Firefox, channel=beta*, version<11.0, priority=95,
> > mapping=Firefox-11.0b5-build1
> > 2) product=Firefox, channel=beta*, version<29.0, priority=93,
> > mapping=Firefox-29.0b8-build1
> > to match what we have on aus3.
> > 
> > The attached file is the six 9.0b releases, all platforms & locales, and
> > beta channel. I switched from betatest, because betatest != beta on aus3
> > (booo).
> > 
> > I've submitted a Firefox-11.0b5-build1, and ran the comparions but got 
> > * some capitalization of Firefox wrong in bouncer product, & betatest fileUrl
> > * detailsUrl has en-US instead of %LOCALE%
> > * ?force=1 isn't being passed on by balrog
> > * there's a bunch of 32.0 rc1 differences (displayVersion, URL), which will
> > go away once 32.0b1 is in both update servers
> > 
> > aus4-admin isn't letting me delete the 11.0b5 release to re-add it, so need
> > to track the bug with that down.
> 
> This is bug 1035293 :(. We might need IT to delete this one by hand for us
> in the meantime...

I actually managed to delete it with a DELETE request to https://aus4-admin.mozilla.org/releases/Firefox-11.0b5-build1?data_version=1&csrf_token=(token stolen from web page source)
Comment 7 User image Nick Thomas [:nthomas] 2014-09-08 03:26:37 PDT
Bug 1064143 for force=1 not being passed on for download.m.o, which is the only thing holding 9.0 tests back, I think. 

For builds pointed at 29.0b8, I also need to bump the blob to version 2 with adjusted version parameters, mozilla.com for details url, lower case the bouncer product name, action and openURL (just for consistency). Will whip a little script to do a POST against what's already there.
Comment 8 User image Nick Thomas [:nthomas] 2014-09-08 19:57:17 PDT
With bug 1064143 fixed we have this for the 9.0 urls:

Tested 2580 paths.
Pass count: 2580
Fail count: 0
Error count: 0
Comment 9 User image Nick Thomas [:nthomas] 2014-09-10 03:18:56 PDT
I uploaded a Firefox-29.0b8-build1-schema2 release, which is Firefox-29.0b8-build1 converted to schema_version 2 and assorted other fixes. There's still quite a lot of fail due to differences in actions and openURL, but it turns out bug 1057460 is a convenient reason to modify the aus3 snippets to squash that.
Comment 10 User image Nick Thomas [:nthomas] 2014-09-17 16:01:50 PDT
Bug 1027608 has the latest test results, which all pass apart from some OK exceptions for Thunderbird around detailsUrl and capitalisation of bouncer products.

Comment #8 tests 9.0b6, and I don't need to test betas in the range 4.0b7 to 9.0b5 because bug 717214 set up symlinks so that the updates are the same.

For 4.0b6 and older, they're pointing at 5.0b1 at the moment. This looks to be left over from the very beginning of rapid release (when mozBeta-branch-patcher2.cfg was created) rather than anything else. When we switch they'll start pointing 11.0b5.

Note You need to log in before you can comment on or make changes to this bug.