1827319 - Perma bouncer-check(Firefox and Devedition) FAIL: https://download.mozilla.org/?product=Firefox/Devedition-113.0b1-stub&os=win&lang=de, status: 404

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Uploading, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: ctuns [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=411911305&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cV6trskRTx-dWW_Q5ZrZyw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-04-11T00:53:16.483Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=en-US
[task 2023-04-11T00:53:16.485Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=de
[task 2023-04-11T00:53:16.488Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=it
[task 2023-04-11T00:53:16.489Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=zh-TW
[task 2023-04-11T00:53:16.493Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=en-US
[task 2023-04-11T00:53:16.497Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=de
[task 2023-04-11T00:53:16.499Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=de, status: 404
[task 2023-04-11T00:53:16.501Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=it
[task 2023-04-11T00:53:16.502Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=zh-TW
[task 2023-04-11T00:53:16.505Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.509Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=it, status: 404
[task 2023-04-11T00:53:16.516Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=en-US
[task 2023-04-11T00:53:16.519Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=de
[task 2023-04-11T00:53:16.521Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=it, status: 404
[task 2023-04-11T00:53:16.523Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=de, status: 404
[task 2023-04-11T00:53:16.528Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.531Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=de, status: 404
[task 2023-04-11T00:53:16.534Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=it
[task 2023-04-11T00:53:16.535Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=it, status: 404
[task 2023-04-11T00:53:16.536Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=zh-TW
[task 2023-04-11T00:53:16.538Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=en-US
[task 2023-04-11T00:53:16.539Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.540Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=de
[task 2023-04-11T00:53:16.543Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=it
[task 2023-04-11T00:53:16.546Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=zh-TW
[task 2023-04-11T00:53:16.548Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=en-US
[task 2023-04-11T00:53:16.549Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=de
[task 2023-04-11T00:53:16.551Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=it
[task 2023-04-11T00:53:16.553Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=zh-TW
[task 2023-04-11T00:53:16.567Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=osx&lang=en-US

Comment 1

[Gabriel Bustamante [:gabriel]]

I still haven't gotten to the bottom of it. But, this is what I found.

The Firefox Installer.exe file in the pub/firefox/releases/113.0b1/win32/ l10n paths for 113.0b1 is not there.

See for example 113.0b1 win32 de vs. 112.0b9 win32 de

The file has been missing since the candidate build. That is to say, the file is not in the candidate directories for the 113.0b1 release and that is why the firefox-push-to-release task did not copy it into the release folder.

See for example 113.0b1 candidates win32 de vs. 113.0b1 candidates win32 de

The file is coming from the attribution kinds. See taskcluster/gecko_taskgraph/manifests/firefox_candidates.yml#355-366

The patch on revision D170246 touches taskcluster/gecko_taskgraph/manifests/firefox_candidates.yml and switches the upstream task for Firefox Installer.exe from repackage-signing to attribution and attribution-l10n (the en-US folders did end up with a Firefox Installer.exe file. See 113.0b1 win32 en-US)

This bug might be a regression related this stub attribution patch from Bug 1814727

Comment 2

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1827319 - Beetmove localized stub installers again after the attribution split r=jcristau a=bustage

Fixes up bug 1814727.

Comment 3

[Johan Lorenzo [:jlorenzo]]

Thank you very much :gabriel for your thorough analysis in comment 1! This made the bug much easier to narrow down 😃

Comment 4

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1827319 - Beetmove localized stub installers again after the attribution split r=jcristau a=bustage

Fixes up bug 1814727.

Original Revision: https://phabricator.services.mozilla.com/D175106

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/10242bba12ac
Beetmove localized stub installers again after the attribution split r=jcristau DONTBUILD

Comment 6

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 9327890 [details]
Bug 1827319 - Beetmove localized stub installers again after the attribution split r=jcristau a=bustage

Beta/Release Uplift Approval Request

• User impact if declined: Can't shipped localized Windows Stub installers anymore.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Not risky. The bug was identified by our end-to-end tests in the release pipeline. We'll be able to iterate if the issue remains.
• String changes made/needed: None
• Is Android affected?: No

Comment 7

[Pascal Chevrel:pascalc]

https://hg.mozilla.org/releases/mozilla-beta/rev/3638d3b37957

Comment 8

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/10242bba12ac

Comment 9

[Intermittent Failures Robot]

10 failures in 3206 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 10

Platform and build breakdown:

• devedition-release: 5
  • opt: 5
• firefox-release: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1827319&startday=2023-04-10&endday=2023-04-16&tree=all