release-update-verify-config-thunderbird fails in gather-info

RESOLVED FIXED in Thunderbird 64.0

Status

defect
RESOLVED FIXED
11 months ago
9 months ago

People

(Reporter: rjl, Assigned: rjl)

Tracking

Trunk
Thunderbird 64.0
Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird_esr60 affected)

Details

Attachments

(8 attachments, 6 obsolete attachments)

39.03 KB, text/plain
Details
7.35 KB, text/plain
Details
46 bytes, text/x-phabricator-request
bhearsum
: review+
Details | Review
46 bytes, text/x-phabricator-request
dustin
: review+
Details | Review
46 bytes, text/x-phabricator-request
jorgk
: review+
Details | Review
937 bytes, patch
Details | Diff | Splinter Review
3.86 KB, patch
Details | Diff | Splinter Review
5.19 KB, patch
Details | Diff | Splinter Review
As mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1481095#c7, Thunderbird builds releases off comm-esr.* rather than comm-release. This is causing the 404 error seen below.

[task 2018-09-12T09:30:17.477Z] 09:30:17     INFO - retry: Calling _urlopen with args: (), kwargs: {'url': 'https://hg.mozilla.org/releases/comm-release/raw-file/THUNDERBIRD_60_0_RELEASE/mail/locales/shipped-locales'}, attempt #5
[task 2018-09-12T09:30:17.981Z] 09:30:17     INFO - retry: attempt #5 caught HTTPError exception: HTTP Error 404: Not Found
[task 2018-09-12T09:30:17.982Z] 09:30:17     INFO - [mozharness: 2018-09-12 09:30:17.982054Z] Finished gather-info step (failed)
[task 2018-09-12T09:30:17.982Z] 09:30:17    FATAL - Uncaught exception: Traceback (most recent call last):
[task 2018-09-12T09:30:17.982Z] 09:30:17    FATAL -   File "/builds/worker/checkouts/gecko/testing/mozharness/mozharness/base/script.py", line 2076, in run
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -     self.run_action(action)
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -   File "/builds/worker/checkouts/gecko/testing/mozharness/mozharness/base/script.py", line 2015, in run_action
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -   File "/builds/worker/checkouts/gecko/testing/mozharness/mozharness/base/script.py", line 1955, in _possibly_run_method
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -     return getattr(self, method_name)()
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -   File "testing/mozharness/scripts/release/update-verify-config-creator.py", line 315, in gather_info
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -     self._get_update_paths()
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -   File "testing/mozharness/scripts/release/update-verify-config-creator.py", line 292, in _get_update_paths
[task 2018-09-12T09:30:17.983Z] 09:30:17    FATAL -     shipped_locales = ret.read().strip()
[task 2018-09-12T09:30:17.984Z] 09:30:17    FATAL - AttributeError: 'NoneType' object has no attribute 'read'
[task 2018-09-12T09:30:17.984Z] 09:30:17    FATAL - Running post_fatal callback...
[task 2018-09-12T09:30:17.984Z] 09:30:17    FATAL - Exiting -1
Posted patch mozilla-esr60_450118.patch (obsolete) — Splinter Review
Patch so that taskgraph/decision.py can get the correct version of Thunderbird from its  copies of *version.txt.
Assignee: nobody → rob
Posted patch mozilla-esr60_450119.patch (obsolete) — Splinter Review
Updates to update-verify-config-creator.py to set the correct URL for Thunderbird releases when getting information from HG repos.
This is a work in progress, saving here for anyone interested. Tom has suggested a couple of changes that I will be making.

Note that these patches are against mozilla-esr60. I'll port to mozilla-central when they're complete.
Thunderbird releases come from the comm-esr.* repositories rather than the
comm-release repository. This adds a special case for Thunderbird and sets
the branch URL correctly.

This is an updated version of the patch that uses branch_prefix == "comm" to identify
a thunderbird product what releases from a comm-esr.* repository.

Additionally, I updated the sorting of the release information that gets pulled
from product-details.mozilla.org. It now uses MozillaVersion in order to get the
sorting in actual reverse order.
Attachment #9010477 - Attachment is obsolete: true
Posted file log_info.log
Output from running update-verify-config-creator.py locally.
Attachment #9010828 - Attachment mime type: text/x-log → text/plain
Posted file update-verify.cfg
Generated update-verify.cfg after changes.
The UVC task needs correct version information supplied to it from the
Taskcluster decision task. That information is exported here for
decision.py to pick up.
This brings the UVC task configuration in line with what's expected for
corresponding M-C changes on this bug.

Depends on D6505
Thunderbird releases come from the comm-esr.* repositories rather than the
comm-release repository. This adds a special case for Thunderbird and sets
the branch URL correctly.

This initial patch is based on the "product" key, the next update will look
at branch_prefix.
The release-update-verify-config task requires that the versions passed
to it match up with what's been released. The version of Thunderbird
does not necessarily match the Gecko version it's based on.

Depends on D6509
Attachment #9011001 - Attachment description: Bug 1491907 - WIP: Set correct branch URL for Thunderbird based on product-details r?bhearsum → Bug 1491907 - Set correct branch URL for Thunderbird based on product-details r?bhearsum
Comment on attachment 9011001 [details]
Bug 1491907 - Set correct branch URL for Thunderbird based on product-details r?bhearsum

Ben Hearsum (:bhearsum) has approved the revision.
Attachment #9011001 - Flags: review+
Blocks: 1460487
Attachment #9010996 - Attachment is obsolete: true
Attachment #9010997 - Attachment is obsolete: true
This is used by the decision task to locate the correct version.txt and
app_version.txt files for products other than Firefox.
Needs the corresponding change to land in M-C before it can be applied.
No breakage should occur if M-C gets updated before C-C as this is listed
as an Optional field in config.py.
Attachment #9010476 - Attachment is obsolete: true
Attachment #9010827 - Attachment is obsolete: true
A few minor changes have been requested to the latest patch in Phabricator. I'll get those sorted out ASAP and resubmit.
- only read config.yml once
- make product_dir a top level property in config.yml
- adjust get_version and get_app_version in properties.py. Remove the constants from __init__.py and simplify the two functions.
Comment on attachment 9011002 [details]
Bug 1491907 - Get comm/ version from comm/mail/*version.txt r?bhearsum

Dustin J. Mitchell [:dustin] pronoun: he has approved the revision.
Attachment #9011002 - Flags: review+
Thunderbird releases come from the comm-esr.* repositories rather than the
comm-release repository. This adds a special case for Thunderbird and sets
the branch URL correctly.

This patch is based on the branch_prefix (comm or mozilla).

Updated to fix pylint errors.
Attachment #9012549 - Attachment is obsolete: true
There are three patches attached to this bug that are in Phabricator.
Attachment 9011001 [details]: Applies to M-C.
Attachment 9011002 [details]: Applies to M-C
Attachment 9011910 [details]: Applies to C-C

These don't apply to the ESR60 repositories so nicely. I can upload updated patches if necessary.
Hmm, I could land the first patch via Lando, but the second one is awaiting Tom's review and can't be landed. Also, I think it would make sense to land them together, no idea how that works in Lando.

Any yes, patches that apply to m/c-esr60 would be good so I don't mess it up ;-)
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/integration/autoland/rev/f9cf6e98fc4b
Set correct branch URL for Thunderbird based on product-details r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/b96cec26d463
Get comm/ version from comm/mail/*version.txt r=dustin
Attachment #9013113 - Attachment is patch: true
Attachment #9013113 - Attachment mime type: application/octet-stream → text/plain
Just uploaded the patches ported to *-esr60 trees. If there's any problems, please let me know.
https://hg.mozilla.org/mozilla-central/rev/f9cf6e98fc4b
https://hg.mozilla.org/mozilla-central/rev/b96cec26d463
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 64.0
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/390588f44e2b
C-C part: Add product-dir to config.yml. r=jorgk
Blocks: 1495480
Tom, who can grant M-B approval here?
Flags: needinfo?(mozilla)
Attachment #9011910 - Flags: review+
Attachment #9011910 - Flags: approval-comm-esr60+
Attachment #9011910 - Flags: approval-comm-beta+
Comment on attachment 9011910 [details]
Bug 1491907 - Add product-dir to config.yml

Jorg K (GMT+2) has been removed from the revision.
Attachment #9011910 - Flags: review+
Attachment #9011910 - Flags: review+
a=tomprince for these. But it is probably best to direct these to relman going forward.
Flags: needinfo?(mozilla)
Reopening, since we're backing these patches out of m-c/m-b for breaking Firefox update-verify-config tasks.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
No longer blocks: 1460487
The fix applied in bug 1496480 does not affect comm-* code, nothing more needs to be done with this.
Status: REOPENED → RESOLVED
Closed: 11 months ago11 months ago
Resolution: --- → FIXED
Depends on: 1505615
You need to log in before you can comment on or make changes to this bug.