Closed Bug 1309130 Opened 3 years ago Closed 3 years ago

Provide a partial mar file to update from Firefox 43.0.1

Categories

(Release Engineering :: Release Requests, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: robert.strong.bugs, Assigned: rail)

Details

Attachments

(2 files, 1 obsolete file)

One of the things that will help move clients from Firefox 43.0.1 is a partial mar since the vast majority of issues the clients on 43.0.1 are experiencing are with downloading the mar file.
Is this Windows only?
Assignee: nobody → rail
This affects all platforms though Windows of course has the largest number of users.
I was curious about the size of a partial from 43.0.1 to 49.0.1. For win32 en-US it's ~38M, vs a complete of 51M, so that's a moderate saving of 25%. About 32M is patches for xul.dll, omni.ja, and browser.omni.ja, as we already knew.
BTW, we won't be able to server 43.0.1-to-49.0.x partial, because we have a watershed (at 48.0.2) to enable SSE detection. Also, we need to account the websense situation.. :/
What is the status on getting this bug fixed?
Flags: needinfo?(rail)
I haven't started yet. I hope to poke it sometime this week.
Flags: needinfo?(rail)
Hi rail, a heartbeat roll-out decision depends on the timing for this, can you please clarify how soon this could be rolled out to help us there?
Flags: needinfo?(rail)
Sorry, nothing firm yet. This is something that we have never done before.
Flags: needinfo?(rail)
I actually looked at this today and chatted with bhearsum to sanity check my plan.

This is what we need to do:

0) copy Firefox-47.0.2-build2 blob to Firefox-47.0.2-build2-prod and use it for release rules. Firefox-47.0.2-build2 will be used by automation. (Done)

1) Add a new top-level partial info for 43.0.1 in Firefox-47.0.2-build2

2) generate a TC graph with the following tasks (pseudo code)

  for platofrm in (win32, win64):
     for locale in common_locales:
       - generate MAR task
       - sign MAR task
       - publish to balrog task
       - copy MARs to the candidates directory task
   - copy to the releases directory task (probably with a human decision task)

3) create a new bouncer entry

4) QA should tests update from 43.0.1 and other versions

5) it would be great to schedule update verify tests as well. This part is a bit tricky, we may need to create a new branch in the tools repo with updated patcher config and update verify configs.

I believe QA is busy with releases this week and aurora next week. Also Releang has a work week next week. I don't really want to rush with this and test the generated MARs and updates.

Let's plan deploying this in a couple of weeks.
Since 47.0.2 is also a watershed it would be a good thing to also create a partial for it as well.
I looked at this today again.

We have 4 different paths for updates from 43.0.1:

1) windows: to 47.0.2 (sse+websense)
2) linux: 45.0.2 (GTK watershed)
3) osx 10.6-10.8: 48.0.2 (desupport)
4) osx 10.9+: latest

Let's concentrate on 1) for now, and possibly ignore the other options.
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #12)
> Created attachment 8812884 [details]
> Bug 1309130 - Provide a partial mar file to update from Firefox 43.0.1
> 
> Review commit: https://reviewboard.mozilla.org/r/94434/diff/#index_header
> See other reviews: https://reviewboard.mozilla.org/r/94434/

This is how it was generated:

wget https://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_47_0_2_RELEASE/browser/locales/shipped-locales
hg clone https://hg.mozilla.org/build/tools
hg -R tools up FIREFOX_47_0_2_RELEASE_RUNTIME
PERL5LIB=tools/lib/perl perl tools/release/patcher-config-bump.pl \
  -c tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  -p firefox -r Firefox -v 47.0.2 -a 47.0.2 -o 43.0.1 -b 2 \
  -f archive.mozilla.org -d download.mozilla.org -l shipped-locales --partial-version 43.0.1 \
  --platform linux --platform linux64 --platform macosx64 --platform win32 --platform win64
  
 
python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win32 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win32.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 2

python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win64 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win64.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 2
  
grep -F 43.0.1 tools/release/updates/release-firefox-win32.cfg  > tools/release/updates/release-firefox-win32.cfg.tmp
mv tools/release/updates/release-firefox-win32.cfg.tmp tools/release/updates/release-firefox-win32.cfg
grep -F 43.0.1 tools/release/updates/release-firefox-win64.cfg  > tools/release/updates/release-firefox-win64.cfg.tmp
mv tools/release/updates/release-firefox-win64.cfg.tmp tools/release/updates/release-firefox-win64.cfg
Comment on attachment 8812884 [details]
Bug 1309130 - Provide a partial mar file to update from Firefox 43.0.1

nm, reviewboard is weird, I'll submit a patch
Attachment #8812884 - Attachment is obsolete: true
Attachment #8812884 - Flags: review?(bhearsum)
see comment #13
Attachment #8812885 - Flags: review?(bhearsum)
Attachment #8812885 - Flags: review?(bhearsum) → review+
My checklist:

[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[ ] Schedule the actual automation to generate, sign, publish and run update verify tests
[ ] Get QE sign off on release-localtest
[ ] Push to cdns
[ ] Setup release-cdntest similar to release-localtest
[ ] Get QE sign off on release-cdntest
[ ] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!

We can probably skip the cdntest channel and go to release after localtest.
I actually went ahead and pushed the updates to the release-cdntest channel after update verify tests passed on the release-localtest channel. The TC graph is https://tools.taskcluster.net/task-group-inspector/#/TrUxMZgHThKbpekwX4JESw

Updating the checklist:


[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[x] Schedule the actual automation to generate, sign, publish and run update verify tests
[skip] Get QE sign off on release-localtest
[x] Push to cdns
[x] Setup release-cdntest similar to release-localtest
[ ] Get QE sign off on release-cdntest
[ ] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!
I tried to update en-US win64 43.0.1 and it got a partial update to 47.0.2 as expected (tested on release-localtest channel).


Andrei, can you test the following scenario:

Platforms: win32, win64
Version: 43.0.1
Channel: release-cdntest

Expected result: should a partial update to 47.0.2 (with prompt), then to latest
Flags: needinfo?(andrei.vaida)
I've tested on Windows 7, Windows 8.1, Windows 10 and Windows XP the following scenario: 
* started 43.0.1 (win32, win64 en-US) → waited for about 2 mins → update prompt correctly displayed -> → updated to 47.0.2

* triggered an update check manually → updated to 50.0 

Everything looks good. 

Detailed test results can be found in the following etherpad: https://public.etherpad-mozilla.org/p/bug1309130 



(In reply to Rail Aliiev [:rail] ⌚️ET from comment #19)
> I tried to update en-US win64 43.0.1 and it got a partial update to 47.0.2
> as expected (tested on release-localtest channel).
> 
> 
> Andrei, can you test the following scenario:
> 
> Platforms: win32, win64
> Version: 43.0.1
> Channel: release-cdntest
> 
> Expected result: should a partial update to 47.0.2 (with prompt), then to
> latest
Flags: needinfo?(andrei.vaida)
Thank you, Camelia!

I'll set up the release channel accordingly in a bit.
Camelia, can you repeat the same scenario (comment #20) against the release channel?
Flags: needinfo?(camelia.badau)
Updating the checklist:


[x] Add Firefox-47.0.2-Partial-43.0.1 product to bouncer
[x] Add win and win64 locations for Firefox-47.0.2-Partial-43.0.1
[x] Land the tools patch (comment #16)
[x] Add top level partial Firefox-43.0.1-build1 to Firefox-47.0.7-build2 release blob in Balrog, https://gist.github.com/rail/f7a4bc8b289749e25a0981a8bf74520f
[x] Setup release-localtest so it points to the original Firefox-47.0.2-build2 blob before any other modifications of that blob (showPrompt and prod)
[x] Schedule the actual automation to generate, sign, publish and run update verify tests
[skip] Get QE sign off on release-localtest
[x] Push to cdns
[x] Setup release-cdntest similar to release-localtest
[x] Get QE sign off on release-cdntest
[x] Setup release similar to release-cdntest
[ ] Get QE sign off on release
[ ] Champagne!

Only 1 step left, then Champagne! :)
I've retested the same scenario on the release channel and everything looks good.

Please see the detailed test results here: https://public.etherpad-mozilla.org/p/bug1309130



(In reply to Rail Aliiev [:rail] ⌚️ET from comment #22)
> Camelia, can you repeat the same scenario (comment #20) against the release
> channel?
Flags: needinfo?(camelia.badau)
Woot! Thanks a lot for the testing!

I'll see what we can do for mac and linux. Not sure if it's worth it though. Robert, what you say?
Flags: needinfo?(robert.strong.bugs)
The only pending task is Champagne! Too early though. :)
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #25)
> Woot! Thanks a lot for the testing!
> 
> I'll see what we can do for mac and linux. Not sure if it's worth it though.
> Robert, what you say?
I'm fine with only a Windows partial.
Flags: needinfo?(robert.strong.bugs)
Sweet! Let's resolve it then!
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
FTR, this is a Q&D way to schedule required automation. A lot of room for improvements :)
(In reply to Nick Thomas [:nthomas] from comment #3)
> I was curious about the size of a partial from 43.0.1 to 49.0.1. For win32
> en-US it's ~38M, vs a complete of 51M, so that's a moderate saving of 25%.
> About 32M is patches for xul.dll, omni.ja, and browser.omni.ja, as we
> already knew.

I was curious about the potential impact of bug 1231379 (disabling compression of omni.ja) on bugs like this, so I repacked the en-US win32 complete mars for 43.0.1 and 47.0.2 to repack their omni.ja's without compression, and then generated a partial update. The new partial was 25MB, compared to 33MB for the partial we actually shipped. The complete mar for 47.0.2 was 52MB.
You need to log in before you can comment on or make changes to this bug.