1309130 - Provide a partial mar file to update from Firefox 43.0.1

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect)

Description

[Robert Strong (they/them - no direct email)]

One of the things that will help move clients from Firefox 43.0.1 is a partial mar since the vast majority of issues the clients on 43.0.1 are experiencing are with downloading the mar file.

Comment 1

[Chris AtLee [:catlee]]

Is this Windows only?

Comment 2

[Robert Strong (they/them - no direct email)]

This affects all platforms though Windows of course has the largest number of users.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

I was curious about the size of a partial from 43.0.1 to 49.0.1. For win32 en-US it's ~38M, vs a complete of 51M, so that's a moderate saving of 25%. About 32M is patches for xul.dll, omni.ja, and browser.omni.ja, as we already knew.

Comment 4

[Rail Aliiev [:rail]]

BTW, we won't be able to server 43.0.1-to-49.0.x partial, because we have a watershed (at 48.0.2) to enable SSE detection. Also, we need to account the websense situation.. :/

Comment 5

[Robert Strong (they/them - no direct email)]

What is the status on getting this bug fixed?

Comment 6

[Rail Aliiev [:rail]]

I haven't started yet. I hope to poke it sometime this week.

Comment 7

[Romain Testard [:RT]]

Hi rail, a heartbeat roll-out decision depends on the timing for this, can you please clarify how soon this could be rolled out to help us there?

Comment 8

[Rail Aliiev [:rail]]

Sorry, nothing firm yet. This is something that we have never done before.

Comment 9

[Rail Aliiev [:rail]]

I actually looked at this today and chatted with bhearsum to sanity check my plan.

This is what we need to do:

0) copy Firefox-47.0.2-build2 blob to Firefox-47.0.2-build2-prod and use it for release rules. Firefox-47.0.2-build2 will be used by automation. (Done)

1) Add a new top-level partial info for 43.0.1 in Firefox-47.0.2-build2

2) generate a TC graph with the following tasks (pseudo code)

  for platofrm in (win32, win64):
     for locale in common_locales:
       - generate MAR task
       - sign MAR task
       - publish to balrog task
       - copy MARs to the candidates directory task
   - copy to the releases directory task (probably with a human decision task)

3) create a new bouncer entry

4) QA should tests update from 43.0.1 and other versions

5) it would be great to schedule update verify tests as well. This part is a bit tricky, we may need to create a new branch in the tools repo with updated patcher config and update verify configs.

I believe QA is busy with releases this week and aurora next week. Also Releang has a work week next week. I don't really want to rush with this and test the generated MARs and updates.

Let's plan deploying this in a couple of weeks.

Comment 10

[Robert Strong (they/them - no direct email)]

Since 47.0.2 is also a watershed it would be a good thing to also create a partial for it as well.

Comment 11

[Rail Aliiev [:rail]]

I looked at this today again.

We have 4 different paths for updates from 43.0.1:

1) windows: to 47.0.2 (sse+websense)
2) linux: 45.0.2 (GTK watershed)
3) osx 10.6-10.8: 48.0.2 (desupport)
4) osx 10.9+: latest

Let's concentrate on 1) for now, and possibly ignore the other options.

Comment 12

[Rail Aliiev [:rail]]

Attachment: Bug 1309130 - Provide a partial mar file to update from Firefox 43.0.1

Review commit: https://reviewboard.mozilla.org/r/94434/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/94434/

Comment 13

[Rail Aliiev [:rail]]

(In reply to Rail Aliiev [:rail] ⌚️ET from comment #12)
> Created attachment 8812884 [details]
> Bug 1309130 - Provide a partial mar file to update from Firefox 43.0.1
> 
> Review commit: https://reviewboard.mozilla.org/r/94434/diff/#index_header
> See other reviews: https://reviewboard.mozilla.org/r/94434/

This is how it was generated:

wget https://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_47_0_2_RELEASE/browser/locales/shipped-locales
hg clone https://hg.mozilla.org/build/tools
hg -R tools up FIREFOX_47_0_2_RELEASE_RUNTIME
PERL5LIB=tools/lib/perl perl tools/release/patcher-config-bump.pl \
  -c tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  -p firefox -r Firefox -v 47.0.2 -a 47.0.2 -o 43.0.1 -b 2 \
  -f archive.mozilla.org -d download.mozilla.org -l shipped-locales --partial-version 43.0.1 \
  --platform linux --platform linux64 --platform macosx64 --platform win32 --platform win64
  
 
python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win32 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win32.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 2

python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win64 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win64.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 2
  
grep -F 43.0.1 tools/release/updates/release-firefox-win32.cfg  > tools/release/updates/release-firefox-win32.cfg.tmp
mv tools/release/updates/release-firefox-win32.cfg.tmp tools/release/updates/release-firefox-win32.cfg
grep -F 43.0.1 tools/release/updates/release-firefox-win64.cfg  > tools/release/updates/release-firefox-win64.cfg.tmp
mv tools/release/updates/release-firefox-win64.cfg.tmp tools/release/updates/release-firefox-win64.cfg

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8812884 [details]
Bug 1309130 - Provide a partial mar file to update from Firefox 43.0.1

nm, reviewboard is weird, I'll submit a patch

Comment 15

[Rail Aliiev [:rail]]

Attachment: 43.0.1-47.0.2-win.diff

see comment #13

Comment 16

[Rail Aliiev [:rail]]

Comment on attachment 8812885 [details] [diff] [review]
43.0.1-47.0.2-win.diff

Also re-tagged using FIREFOX_47_0_2_RELEASE_RUNTIME

remote:   https://hg.mozilla.org/build/tools/rev/b0d2cfd4caca8d11f6b8d06f96f70d088e73555c
remote:   https://hg.mozilla.org/build/tools/rev/f46ca23ad709ee8d10b1632adf1d483bf4ea8195

Comment 17

[Rail Aliiev [:rail]]

My checklist:

[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[ ] Schedule the actual automation to generate, sign, publish and run update verify tests
[ ] Get QE sign off on release-localtest
[ ] Push to cdns
[ ] Setup release-cdntest similar to release-localtest
[ ] Get QE sign off on release-cdntest
[ ] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!

We can probably skip the cdntest channel and go to release after localtest.

Comment 18

[Rail Aliiev [:rail]]

I actually went ahead and pushed the updates to the release-cdntest channel after update verify tests passed on the release-localtest channel. The TC graph is https://tools.taskcluster.net/task-group-inspector/#/TrUxMZgHThKbpekwX4JESw

Updating the checklist:


[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[x] Schedule the actual automation to generate, sign, publish and run update verify tests
[skip] Get QE sign off on release-localtest
[x] Push to cdns
[x] Setup release-cdntest similar to release-localtest
[ ] Get QE sign off on release-cdntest
[ ] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!

Comment 19

[Rail Aliiev [:rail]]

I tried to update en-US win64 43.0.1 and it got a partial update to 47.0.2 as expected (tested on release-localtest channel).


Andrei, can you test the following scenario:

Platforms: win32, win64
Version: 43.0.1
Channel: release-cdntest

Expected result: should a partial update to 47.0.2 (with prompt), then to latest

Comment 20

[Camelia Badau [:cbadau], Desktop Test Engineering]

I've tested on Windows 7, Windows 8.1, Windows 10 and Windows XP the following scenario: 
* started 43.0.1 (win32, win64 en-US) → waited for about 2 mins → update prompt correctly displayed -> → updated to 47.0.2

* triggered an update check manually → updated to 50.0 

Everything looks good. 

Detailed test results can be found in the following etherpad: https://public.etherpad-mozilla.org/p/bug1309130 



(In reply to Rail Aliiev [:rail] ⌚️ET from comment #19)
> I tried to update en-US win64 43.0.1 and it got a partial update to 47.0.2
> as expected (tested on release-localtest channel).
> 
> 
> Andrei, can you test the following scenario:
> 
> Platforms: win32, win64
> Version: 43.0.1
> Channel: release-cdntest
> 
> Expected result: should a partial update to 47.0.2 (with prompt), then to
> latest

Comment 21

[Rail Aliiev [:rail]]

Thank you, Camelia!

I'll set up the release channel accordingly in a bit.

Comment 22

[Rail Aliiev [:rail]]

Camelia, can you repeat the same scenario (comment #20) against the release channel?

Comment 23

[Rail Aliiev [:rail]]

Updating the checklist:


[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[x] Schedule the actual automation to generate, sign, publish and run update verify tests
[skip] Get QE sign off on release-localtest
[x] Push to cdns
[x] Setup release-cdntest similar to release-localtest
[x] Get QE sign off on release-cdntest
[x] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!

Only 1 step left, then Champagne! :)

Comment 24

[Camelia Badau [:cbadau], Desktop Test Engineering]

I've retested the same scenario on the release channel and everything looks good.

Please see the detailed test results here: https://public.etherpad-mozilla.org/p/bug1309130



(In reply to Rail Aliiev [:rail] ⌚️ET from comment #22)
> Camelia, can you repeat the same scenario (comment #20) against the release
> channel?

Comment 25

[Rail Aliiev [:rail]]

Woot! Thanks a lot for the testing!

I'll see what we can do for mac and linux. Not sure if it's worth it though. Robert, what you say?

Comment 26

[Rail Aliiev [:rail]]

The only pending task is Champagne! Too early though. :)

Comment 27

[Robert Strong (they/them - no direct email)]

(In reply to Rail Aliiev [:rail] ⌚️ET from comment #25)
> Woot! Thanks a lot for the testing!
> 
> I'll see what we can do for mac and linux. Not sure if it's worth it though.
> Robert, what you say?
I'm fine with only a Windows partial.

Comment 28

[Rail Aliiev [:rail]]

Sweet! Let's resolve it then!

Comment 29

[Rail Aliiev [:rail]]

Attachment: partials-43.0.1-47.0.2.tar.gz

FTR, this is a Q&D way to schedule required automation. A lot of room for improvements :)

Comment 30

[Robert Strong (they/them - no direct email)]

Thank you Rail!

Comment 31

[Robert Strong (they/them - no direct email)]

Filed bug 1319905 for doing the same for 47.0.2

Comment 32

[Chris AtLee [:catlee]]

(In reply to Nick Thomas [:nthomas] from comment #3)
> I was curious about the size of a partial from 43.0.1 to 49.0.1. For win32
> en-US it's ~38M, vs a complete of 51M, so that's a moderate saving of 25%.
> About 32M is patches for xul.dll, omni.ja, and browser.omni.ja, as we
> already knew.

I was curious about the potential impact of bug 1231379 (disabling compression of omni.ja) on bugs like this, so I repacked the en-US win32 complete mars for 43.0.1 and 47.0.2 to repack their omni.ja's without compression, and then generated a partial update. The new partial was 25MB, compared to 33MB for the partial we actually shipped. The complete mar for 47.0.2 was 52MB.