Complete updates to DevEdition 70.0b2 failing
Categories
(Release Engineering :: Release Automation: Updates, defect, P1)
Tracking
(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox68 unaffected, firefox69 unaffected, firefox70 fixed)
| Tracking | Status | |
|---|---|---|
| firefox-esr60 | --- | unaffected |
| firefox-esr68 | --- | unaffected |
| firefox68 | --- | unaffected |
| firefox69 | --- | unaffected |
| firefox70 | --- | fixed |
People
(Reporter: nthomas, Assigned: nthomas)
References
Details
Attachments
(1 file, 1 obsolete file)
Eg https://taskcluster-artifacts.net/cJnuTObASSWiaafUIqImLg/0/public/logs/live_backing.log
[task 2019-08-29T20:43:27.546Z] + /builds/worker/checkouts/gecko/tools/update-verify/release/updates/updater/firefox/updater /builds/worker/checkouts/gecko/tools/update-verify/release/updates/update /builds/worker/checkouts/gecko/tools/update-verify/release/updates/source/firefox /builds/worker/checkouts/gecko/tools/update-verify/release/updates/source/firefox 0
[task 2019-08-29T20:43:27.556Z] Unable to init server: Could not connect: Connection refused
[task 2019-08-29T20:43:27.741Z] + set +x
[task 2019-08-29T20:43:27.742Z] PATCH DIRECTORY /builds/worker/checkouts/gecko/tools/update-verify/release/updates/update
[task 2019-08-29T20:43:27.742Z] INSTALLATION DIRECTORY /builds/worker/checkouts/gecko/tools/update-verify/release/updates/source/firefox
[task 2019-08-29T20:43:27.742Z] WORKING DIRECTORY /builds/worker/checkouts/gecko/tools/update-verify/release/updates/source/firefox
[task 2019-08-29T20:43:27.742Z] failed: 23
[task 2019-08-29T20:43:27.742Z] calling QuitProgressUI
[task 2019-08-29T20:43:27.743Z] TEST-UNEXPECTED-FAIL: update status was not successful: failed: 23
[task 2019-08-29T20:43:27.743Z] TEST-UNEXPECTED-FAIL: [70.0 ach complete] check_updates returned failure for Linux_x86-gcc3 downloads/firefox-70.0b1.tar.bz2 vs. downloads/firefox-70.0b2.tar.bz2: 1
Error 23 is a VERSION_DOWNGRADE_ERROR. This is all platforms, all locales, completes but not partials (bug 1577634).
The complete mar files have this encoded metadata:
$ mar -T firefox-70.0b2.complete.mar | head
Signature block found with 1 signature
1 additional block found:
- Product Information Block:
- MAR channel name: firefox-mozilla-aurora
- Product version: 70.0a1
So 70.0a1 < 70.0 used in 70.0b1, so that's why we get the downgrade error. This only occurs on the complete updates, the partials have 70.0 set as expected.
|
Assignee | |
Comment 1•5 years ago
•
|
||
It turns out the repackage tasks (which create the complete mar files) depend on an autoland job (KPYGHHbpQKC-oZaThL01FQ) to generate the mar utility. That has the milestone built into it as a default version. I think we should rebuild the mar-tools toolchain based on milestone.txt.
There's a second issue where we're not passing the actual version we want when calling mar, due to some mismatched variables.
|
|
Assignee | |
Comment 2•5 years ago
|
||
The mar utility stores MOZ_APP_VERSION as the default Product Version for mar creation. So mar should be rebuilt whenever the milestone changes.
|
|
Assignee | |
Comment 3•5 years ago
|
||
MOZ_FULL_PRODUCT_VERSION is only used here, while tools/update-packaging/make_full_update.sh expects MOZ_PRODUCT_VERSION.
Depends on D44089
Pushed by nthomas@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/464a2ec59a58 mar-tools toolchain should depend on milestone.txt, r=Callek
|
||
Comment 5•5 years ago
|
||
Backed out changeset 464a2ec59a58 (Bug 1577664) for causing gecko decision task failure
Backout: https://hg.mozilla.org/integration/autoland/rev/d9a8ec2b7cd8d8d6ee4476db5e78ab872ccd7bac
Pushed by nthomas@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ca2208744242 pass product version to the complete mar generation script, r=Callek
|
|
Assignee | |
Comment 7•5 years ago
•
|
||
And pushed to env var fix to beta too:
https://hg.mozilla.org/releases/mozilla-beta/rev/fc4326204da468558cfd3ad27bf420a0012b5c70
The backout of the toolchain fix seems to be related to resources being able to depend on directories but not files.
|
|
Assignee | |
Updated•5 years ago
|
|
|
||
Comment 8•5 years ago
|
||
Backed out changeset b25884891abc (Bug 1577664) for causing Gecko decision task failure CLOSED TREE
Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=264177810&resultStatus=testfailed%2Cbusted%2Cexception&revision=b25884891abc435e5b79e3a00087018847d697fa
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=264177810&repo=autoland&lineNumber=475
Backout: https://hg.mozilla.org/integration/autoland/rev/a8aa92121117d7c312b9f7f3680866d15e3661c3
|
|
Assignee | |
Comment 9•5 years ago
|
||
Not sure why the decision task doesn't like the toolchain patch, but it's not blocking deved 70.0b2 now that the MOZ_PRODUCT_VERSION change has landed. We'll use mar from task KPYGHHbpQKC-oZaThL01FQ, which is autoland fb03ac72bc47c9b4bd97e3277ab0cee02026aa7f. There is no diff between that rev and beta in toolkit/mozapps/updater.
|
||
Comment 10•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Updated•5 years ago
|
|
||
Comment 11•5 years ago
|
||
If we aren't using the compiled-in version anywhere, is there any reason to make mar-tools depend on the version?
It might make sense to remove compiling in the version (and the update-channel) into the tool, though.
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 12•5 years ago
|
||
(In reply to Tom Prince [:tomprince] from comment #11)
If we aren't using the compiled-in version anywhere, is there any reason to make mar-tools depend on the version?
It might make sense to remove compiling in the version (and the update-channel) into the tool, though.
If rstrong is happy with that in bug 1577760 then that sounds like a good solution.
How does this regress bug 1458385 ?
|
|
Assignee | |
Comment 13•5 years ago
|
||
There's no need to uplift this to release or esr, because the mar-tools toolchain is new in 70 (bug 1458385 again).
|
||
Comment 14•5 years ago
|
||
| bugherder uplift | ||
Description
•