1180494 - l10n nightlies don't offer incremental updates

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Axel Hecht [:Pike]]

The update from the 4th to the 5th only offers a full mar download for the german mac nightly, https://aus4.mozilla.org/update/3/Firefox/41.0a2/20150704004007/Darwin_x86_64-gcc3-u-i386-x86_64/de/aurora/Darwin%2014.3.0/default/default/update.xml.

This also affects central.

Given that the builds had been very intermittent lately, it's hard to nail down a regression range.

[Tracking Requested - why for this release]:

Not having incremental updates would be a significant regression for Firefox adoption.

Comment 1

[Axel Hecht [:Pike]]

Nightly update snippet, https://aus4.mozilla.org/update/3/Firefox/42.0a1/20150704030210/Darwin_x86_64-gcc3-u-i386-x86_64/de/nightly/Darwin%2014.3.0/default/default/update.xml?force=1

Comment 2

[Francesco Lodolo [:flod]]

To be honest I don't remember the last time I saw a partial update on my Mac for Italian, so it might not be a recent regression (guilty of keeping forgetting to file a bug).

Comment 3

[Rail Aliiev [:rail]]

Sounds like the new l10n repacking script submits the update data twice and the second submission contains complete-only data:

from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/07/2015-07-05-03-02-16-mozilla-central-l10n/mozilla-central-linux-l10n-nightly-1-unknown-bm73-build1-build7.txt.gz

06:16:04     INFO -  Balrog request to https://aus4-admin.mozilla.org/api/releases/Firefox-mozilla-central-nightly-20150705030216/builds/Linux_x86-gcc3/ar
06:16:04     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': 'null', 'schema_version': 4, 'data_version': '50', 'copyTo': '["Firefox-mozilla-central-nightly-latest"]', 'version': u'42.0a1', 'data': '{"platformVersion": "42.0a1", "partials": [{"fileUrl": "http://download.cdn.mozilla.net/pub/mozilla.org/firefox/nightly/2015/07/2015-07-05-03-02-16-mozilla-central-l10n/firefox-42.0a1.ar.linux-i686.partial.20150704030210-20150705085107.mar", "hashValue": "399d4b7fe4757ac9c1960920d2bb247ea85ba99645020c1d104a502d347908d1aa651115c9c10976687b1574fa07e393f010e87098abd3567d9917ad3b93752a", "from": "Firefox-mozilla-central-nightly-20150704030210", "filesize": 7419792}], "completes": [{"fileUrl": "http://download.cdn.mozilla.net/pub/mozilla.org/firefox/nightly/2015/07/2015-07-05-03-02-16-mozilla-central-l10n/firefox-42.0a1.ar.linux-i686.complete.mar", "hashValue": "ce801c1eedaa9c04016ebf64ae0fda891ccaf38366a69d953687b8774340b0e6c19a322876113bb313c1687b9c7aa17ea2628c5d06bc90d4772bb17029f7f4f6", "from": "*", "filesize": 57484410}], "buildID": "20150705030216", "displayVersion": "42.0a1", "appVersion": "42.0a1"}'}

06:16:05     INFO -  Balrog request to https://aus4-admin-dev.allizom.org/api/releases/Firefox-mozilla-central-nightly-20150705030216/builds/Linux_x86-gcc3/ar
06:16:05     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': 'null', 'schema_version': 4, 'data_version': '50', 'copyTo': '["Firefox-mozilla-central-nightly-latest"]', 'version': u'42.0a1', 'data': '{"platformVersion": "42.0a1", "partials": [{"fileUrl": "http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/07/2015-07-05-03-02-16-mozilla-central-l10n/firefox-42.0a1.ar.linux-i686.partial.20150704030210-20150705085107.mar", "hashValue": "399d4b7fe4757ac9c1960920d2bb247ea85ba99645020c1d104a502d347908d1aa651115c9c10976687b1574fa07e393f010e87098abd3567d9917ad3b93752a", "from": "Firefox-mozilla-central-nightly-20150704030210", "filesize": 7419792}], "completes": [{"fileUrl": "http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/07/2015-07-05-03-02-16-mozilla-central-l10n/firefox-42.0a1.ar.linux-i686.complete.mar", "hashValue": "ce801c1eedaa9c04016ebf64ae0fda891ccaf38366a69d953687b8774340b0e6c19a322876113bb313c1687b9c7aa17ea2628c5d06bc90d4772bb17029f7f4f6", "from": "*", "filesize": 57484410}], "buildID": "20150705030216", "displayVersion": "42.0a1", "appVersion": "42.0a1"}'}

Comment 4

[Rail Aliiev [:rail]]

Actually, the log above shows 2 different submissions, one to prod and one to dev balrog.

Comment 5

[Rail Aliiev [:rail]]

So, Linux looks generating partials, but not mac. IIRC, we generate partials using the build system (it's disabled in the script at http://hg.mozilla.org/build/mozharness/file/3ae941c5650c/scripts/desktop_l10n.py#l890). Maybe Mike have some insight. ;)

P.S. Can't wait to see Funsize enabled for nightlies!

Comment 6

[Michael Shal [:mshal]]

Attachment: macosx-l10n-mozconfig

Looks like the l10n-mozconfig isn't including the build/macosx/mozconfig.common file, which is what sets MOZ_AUTOMATION_UPDATE_PACKAGING to tell the build to create the partials. Here's a staging log:

http://dev-master2.bb.releng.use1.mozilla.com:8064/builders/Firefox%20mozilla-central%20macosx64%20l10n%20nightly-1/builds/7/steps/run_script/logs/stdio

10:42:43     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': '["Darwin_x86-gcc3-u-i386-x86_64", "Darwin_x86-gcc3", "Darwin_x86_64-gcc3"]', 'schema_version': 4, 'data_version': '204', 'copyTo': '["Firefox-mozilla-central-nightly-latest"]', 'version': u'42.0a1', 'data': '{"platformVersion": "42.0a1", "partials": [{"fileUrl": "http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/nightly/2015/07/2015-07-06-03-02-06-mozilla-central-l10n/firefox-42.0a1.ar.mac.partial.20150706030206-20150706102459.mar", "hashValue": "ffe3f85d64be54020ddc8dd7b2491d04c7c4b385325cd2cab4d58ced2b3b1266756e920b492ade1a7211eb83cd22ebab63a8cd472c68f802fc6b1739b5604732", "from": "Firefox-mozilla-central-nightly-20150706030206", "filesize": 42890}], "completes": [{"fileUrl": "http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/nightly/2015/07/2015-07-06-03-02-06-mozilla-central-l10n/firefox-42.0a1.ar.mac.complete.mar", "hashValue": "058f8ae9bcbc435d69a02fb1cf8601b94df07da196d14b228bc0d4a3821eb41257d1f636f4881762801944110172801c035c73aa8424108202a618a5f29b7e3b", "from": "*", "filesize": 101786145}], "buildID": "20150706030206", "displayVersion": "42.0a1", "appVersion": "42.0a1"}'}

10:42:44     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': '["Darwin_x86-gcc3-u-i386-x86_64", "Darwin_x86-gcc3", "Darwin_x86_64-gcc3"]', 'schema_version': 4, 'data_version': '205', 'copyTo': '["Firefox-mozilla-central-nightly-latest"]', 'version': u'42.0a1', 'data': '{"platformVersion": "42.0a1", "partials": [{"fileUrl": "http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/nightly/2015/07/2015-07-06-03-02-06-mozilla-central-l10n/firefox-42.0a1.ast.mac.partial.20150706030206-20150706102459.mar", "hashValue": "7f5592b1c7cb64e3d20fc7a879a66eb4448f5d6b4235739bd6f370f7e59bc645cfcce4effac7445fc644a6146636199a14093310686435af00c20bb914c97a18", "from": "Firefox-mozilla-central-nightly-20150706030206", "filesize": 42910}], "completes": [{"fileUrl": "http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/nightly/2015/07/2015-07-06-03-02-06-mozilla-central-l10n/firefox-42.0a1.ast.mac.complete.mar", "hashValue": "57eee7f2fd00303d796e6adf5620ec2cd28fe2b4e996488e8d6665287ca966764598843b7a3663b8dcfc7e421abd898f47c793a55bea625638d628ba3f9d2a56", "from": "*", "filesize": 101691672}], "buildID": "20150706030206", "displayVersion": "42.0a1", "appVersion": "42.0a1"}'}

Comment 7

[Rail Aliiev [:rail]]

Comment on attachment 8630061 [details] [diff] [review]
macosx-l10n-mozconfig

Sweet! Thanks a lot, Mike!

Comment 8

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b95be83cb468

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/b95be83cb468

Comment 10

[Kate Glazko]

Is this also an issue for 41 Aurora? If so, do you plan to request uplift? If not, I can untrack the bug.

Comment 11

[Axel Hecht [:Pike]]

Per initial comment, yes it is.

The landing on central missed today's nightly, so we wasted another 24 hours on verification of the fix to request uplift.

Comment 12

[Kate Glazko]

Thank you, Axel! Tracking for 41. Will keep informed about the uplift.

Comment 13

[Axel Hecht [:Pike]]

Got an incremental update today, same size as en-US.

Next up, uplift to aurora?

Comment 14

[Michael Shal [:mshal]]

Comment on attachment 8630061 [details] [diff] [review]
macosx-l10n-mozconfig

Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: See #c0
[Describe test coverage new/current, TreeHerder]: See #c13
[Risks and why]: 
[String/UUID change made/needed]: None

Comment 15

[Lawrence Mandel [:lmandel] (use needinfo)]

Comment on attachment 8630061 [details] [diff] [review]
macosx-l10n-mozconfig

Aurora+

Comment 16

[Michael Shal [:mshal]]

I'm ready to check in as soon as aurora is open.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/2d009108b099