Closed Bug 1319905 Opened 8 years ago Closed 7 years ago

Provide a partial mar file to update from Firefox 47.0.2

Categories

(Release Engineering :: Release Requests, defect)

Product:
Release Engineering
Component:
Release Requests
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: robert.strong.bugs, Assigned: jlund)

References

Details

Attachments

(3 files)

Bug 1319905 - Provide a partial mar file to update from Firefox 47.0.2.
58 bytes, text/x-review-board-request
jlund
: review+
rail
: checked-in+
Details
partials-47.0.2-50.1.0.tar.gz
6.43 KB, application/gzip
Details
47_0_2_to_50_1_0-tools.patch
48.11 KB, patch
rail
: review+
jlund
: checked-in+
Details | Diff | Splinter Review 
Spinoff of bug 1309130 comment #10

One of the things that will help move clients from Firefox 47.0.2 is a partial mar since 47.0.2 is a watershed and we've seen how watersheds tend to cause a significant rise in clients on older versions. It would be best to wait until 50.0.1 is released.
We can include 47.0.2 in the list of partials for 50.0.1 as a part of ship-it submission, so we generate them as a part of release automation. ATM we can't generate more than 3 partials there (some TC limitations), so we may need to repeat the steps we've done for 43.0.1.
Assignee: nobody → rail
the patch above was generated by running the following:

$ wget https://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_50_0_1_RELBRANCH/browser/locales/shipped-locales                                                                

$ PERL5LIB=tools/lib/perl perl tools/release/patcher-config-bump.pl \                                                                                                     
  -c tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \                                                                                                                   
  -p firefox -r Firefox -v 50.0.1 -a 50.0.1 -o 47.0.2 -b 1 \                                                                                                                          
  -f archive.mozilla.org -d download.mozilla.org -l shipped-locales --partial-version 47.0.2 \                                                                                        
  --platform linux --platform linux64 --platform macosx64 --platform win32 --platform win64  

Then I edited release/patcher-configs/mozRelease-branch-patcher2.cfg to include 47.0.2 build information (it was commited in a separate branch), and adjusted "past-update" to include 47.0.2 update path.

Update verify configs are generated using:

python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win32 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win32.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 1

python tools/scripts/build-promotion/create-update-verify-config.py \
  --config tools/release/patcher-configs/mozRelease-branch-patcher2.cfg \
  --platform win64 --update-verify-channel release-localtest \
  --output tools/release/updates/release-firefox-win64.cfg \
  --archive-prefix https://archive.mozilla.org/pub \
  --previous-archive-prefix https://archive.mozilla.org/pub \
  --product firefox --balrog-url https://aus5.mozilla.org --build-number 1


After we have regular update verify tests passed, we need to land this patch on the FIREFOX_50_0_1 branch in the tools repo, tag the commit using FIREFOX_50_0_1_BUILD1_RUNTIME, FIREFOX_50_0_1_RELEASE_RUNTIME.

Once these are landed, we can start generating partials.
Comment on attachment 8814055 [details]
Bug 1319905 - Provide a partial mar file to update from Firefox 47.0.2.

https://reviewboard.mozilla.org/r/95376/#review95562

not the confident of a r+. Pretty unfamiliar. But what could go wrong! If the tests fail we can tweak :)
Attachment #8814055 - Flags: review?(jlund) → review+
I messed up partials and hardcoded the *to* locale, instead of using {{ locale }}. After the files are removed in bug 1320187, I can repeat this again.
Depends on: 1320187
Looks like it worked better this time and the update verify tests are passing 47.0.2 (still in progress). I think we are good to go. The update will be available in sync with other partials generated by release automation, except that these ones are win32/win64 only.
This worked fine in Firefox 51.0b4.

Next step will be separating integrity verification and balrog verification logic into 2 separate tests. In this case we could test MAR file even before we upload them to the candidates directory.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Disregard the comment above, it supposed to be for bug 692501
With 50.0.2 release there is no longer a partial mar file to update from 47.0.2.

Looks like 50.0 to 50.0.2 is also a complete mar file.

Rail, shall I file a new bug or would you prefer that I reopen this one.
Flags: needinfo?(rail)
Hmm, I'm not sure this is going to scale if we start back filling partials manually. Currently we are limited to 3 partials in release automation and use the top 3 used versions. I'd like to tackle this issue (partials on demand) next year.

In any case, if you feel that we need to generate partials from 50.0, please file a separate bug, we'll handle it.
Flags: needinfo?(rail)
What about from 47.0.2 to latest?
Flags: needinfo?(rail)
(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #13)
> What about from 47.0.2 to latest?

I can help investigate and actioning this. will discuss more offline as to the 'how to'
Assignee: rail → jlund
Status: RESOLVED → REOPENED
Flags: needinfo?(rail)
Resolution: FIXED → ---
The script. Before running this we'll need to adjust the balrog rules to avoid working on live release blobs.
didn't use mozreview bc there is already a tools repo patch attached to this bug.

* this was largely created via running commands like: https://bugzilla.mozilla.org/show_bug.cgi?id=1319905#c3 but with s/50.0.1/50.1.0 and s/1/2 for any build-num places.

* the diff is created against current FIREFOX_50_1_0_RELEASE_RUNTIME tag

* I didn't add "47.0.2 build information" or "past-update" info for 47.0.2 as I don't think we have to since we did for 47.0.2->50.0.1 tools patch so they are there already
Attachment #8820180 - Flags: review?(rail)
Comment on attachment 8820180 [details] [diff] [review]
47_0_2_to_50_1_0-tools.patch

Review of attachment 8820180 [details] [diff] [review]:
-----------------------------------------------------------------

::: release/updates/release-firefox-win32.cfg
@@ +10,5 @@
>  release="48.0.2" product="Firefox" platform="WINNT_x86-msvc-x86" build_id="20160823121617" locales="de en-US ru" channel="release-localtest" patch_types="complete" from="/firefox/releases/48.0.2/win32/%locale%/Firefox Setup 48.0.2.exe" ftp_server_from="https://archive.mozilla.org/pub" ftp_server_to="https://archive.mozilla.org/pub"
>  release="48.0.2" product="Firefox" platform="WINNT_x86-msvc-x86" build_id="20160823121617" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da dsb el en-GB en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
>  release="48.0.1" product="Firefox" platform="WINNT_x86-msvc-x86" build_id="20160817112116" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
>  release="48.0" product="Firefox" platform="WINNT_x86-msvc-x86" build_id="20160726073904" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
> +release="47.0.2" product="Firefox" platform="WINNT_x86-msvc-x86" build_id="20161031133903" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete partial" from="/firefox/releases/47.0.2/win32/%locale%/Firefox Setup 47.0.2.exe" ftp_server_from="https://archive.mozilla.org/pub" ftp_server_to="https://archive.mozilla.org/pub"

You can delete all lines except this one (47.0.2) to reduce the amount of tests.

::: release/updates/release-firefox-win64.cfg
@@ +10,5 @@
>  release="48.0.2" product="Firefox" platform="WINNT_x86_64-msvc-x64" build_id="20160823121617" locales="de en-US ru" channel="release-localtest" patch_types="complete" from="/firefox/releases/48.0.2/win64/%locale%/Firefox Setup 48.0.2.exe" ftp_server_from="https://archive.mozilla.org/pub" ftp_server_to="https://archive.mozilla.org/pub"
>  release="48.0.2" product="Firefox" platform="WINNT_x86_64-msvc-x64" build_id="20160823121617" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da dsb el en-GB en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
>  release="48.0.1" product="Firefox" platform="WINNT_x86_64-msvc-x64" build_id="20160817112116" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
>  release="48.0" product="Firefox" platform="WINNT_x86_64-msvc-x64" build_id="20160726073904" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete"
> +release="47.0.2" product="Firefox" platform="WINNT_x86_64-msvc-x64" build_id="20161031133903" locales="ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja kk km kn ko lij lt lv mai mk ml mr ms nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk uz vi xh zh-CN zh-TW" channel="release-localtest" patch_types="complete partial" from="/firefox/releases/47.0.2/win64/%locale%/Firefox Setup 47.0.2.exe" ftp_server_from="https://archive.mozilla.org/pub" ftp_server_to="https://archive.mozilla.org/pub"

The same here
Attachment #8820180 - Flags: review?(rail) → review+
https://hg.mozilla.org/build/tools/rev/8a4642294cb225beea9770aabff0dd8ea1151bde
Bug 1319905 - Provide a partial update from Firefox 47.0.2 to 50.1.0, r=rail
(In reply to Jordan Lund (:jlund) from comment #20)
> https://tools.taskcluster.net/task-group-inspector/#/
> Fe9dsykmTTyKi3Wl39Tb1Q?_k=r3f1ci

all unblocked tasks are now green.

I had to add:
["fileUrls"]["*"]["partials"].append("Firefox-50.0.1-build1": "http://download.mozilla.org/?product=firefox-50.1.0-partial-50.0.1&os=%OS_BOUNCER%&lang=%LOCALE%")

and:
["fileUrls"]["release-localtest"]["partials"].append("Firefox-47.0.2-build2": "http://archive.mozilla.org/pub/firefox/candidates/50.1.0-candidates/build2/update/%OS_FTP%/%LOCALE%/firefox-47.0.2-50.1.0.partial.mar")

then re-trigger all the update verify tests

rail: okay for me to trigger the push to releases human task, fix whatsnewpage, point official release rules to s/-prod//, and delete temp '-prod' release blob?
comment 21
Flags: needinfo?(rail)
Ship it!
Flags: needinfo?(rail)
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #23)
> Ship it!

this should be done now. release channel users on 47.0.2 should receive partial updates for 50.1.0

closing again.
Status: REOPENED → RESOLVED
Closed: 8 years ago → 7 years ago
Resolution: --- → FIXED
(In reply to Jordan Lund (:jlund) from comment #21)

> ["fileUrls"]["*"]["partials"].append("Firefox-50.0.1-build1":
> "http://download.mozilla.org/?product=firefox-50.1.0-partial-50.0.
> 1&os=%OS_BOUNCER%&lang=%LOCALE%")


I meant http://download.mozilla.org/?product=firefox-50.1.0-partial-47.0.2&os=%OS_BOUNCER%&lang=%LOCALE here.

And I forgot to make the bouncer entries for these locations. For those with access, I added it under https://bounceradmin.mozilla.com/admin/mirror/location/?product__id__exact=6528
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: