1559199 - Thunderbird Taskcluster Index tasks are consistently failing

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect)

Description

[Rob Lemley [:rjl]]

Attachment: live_backing.log

Index tasks fail in automation. I'm not completely sure what this would or could affect, but this needs to be looked at before something breaks.
See attached log file, looks like it tries to send too many headers in the HTTP request it sends off to Taskcluster.

Comment 1

[Rob Lemley [:rjl]]

I believe I need to add a regex to this structure in M-C:taskcluster/taskgraph/morph.py:

# these regular expressions capture route prefixes for which we have a star
# scope, allowing them to be summarized.  Each should correspond to a star scope
# in each Gecko `assume:repo:hg.mozilla.org/...` role.
SCOPE_SUMMARY_REGEXPS = [
    re.compile(r'(index:insert-task:docker\.images\.v1\.[^.]*\.).*'),
    re.compile(r'(index:insert-task:gecko\.v2\.[^.]*\.).*'),
]

https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/morph.py#106-109

The addition would be:

re.compile(r'(index:insert-task:comm\.v2\.[^.]*\.).*')

Given that this structure is looped over multiple times during the decision process, I'm wondering if it would make sense to do it differently so mozilla projects don't have this extra compare over and over that will never match.

Comment 2

[Rob Lemley [:rjl]]

Attachment: Bug 1559199 - Add 'comm' prefix to list of potential index prefixes. r?dustin

Fixes failing index jobs on Thunderbird builds.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7e7807bb8155
Add 'comm' prefix to list of potential index prefixes. r=dustin

Comment 4

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/7e7807bb8155

Comment 5

[Rob Lemley [:rjl]]

Comment on attachment 9078218 [details]
Bug 1559199 - Add 'comm' prefix to list of potential index prefixes. r?dustin

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: This bug has potential to affect the build process for Thunderbird. The index tasks update information about completed jobs in the Taskcluster index, which is broken for Thunderbird without this fix. (It's not all index updates, only ones that require an index task.) I have not observed any impact to these task failures, but that doesn't mean they won't occur in the future.
• User impact if declined: No user impact, this is strictly within the CI system. Firefox builds are not affected.
• Fix Landed on Version: 70
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): There should be no impact on Firefox with this patch as it only adds an additional regex to check to see if a route is a candidate for summarizing in an index task.
• String or UUID changes made by this patch: N/A

Comment 6

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9078218 [details]
Bug 1559199 - Add 'comm' prefix to list of potential index prefixes. r?dustin

Fixes Thunderbird, NPOTB for Firefox. Approved for ESR68.

Comment 7

[Jorg K (CEST = GMT+2)]

Ryan, we're building from a branch
https://hg.mozilla.org/releases/mozilla-esr68/pushloghtml/?changeset=575925ca8346593f2a2796616eeaae774f2ccf79
based on
https://hg.mozilla.org/releases/mozilla-esr68/rev/61219aed2735afac3edcf590e69edbb463ca7372
so I don't think you need to land that, I can put it onto our branch.

Or if you do, you'd have to put it onto FIREFOX_ESR_68_0_X_RELBRANCH and default.

Let me know.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

If you want to land it on just the TB68 verbranch, that probably makes sense. I don't have a strong opinion on this overall.

Comment 9

[Jorg K (CEST = GMT+2)]

Well, it's a one-liner. If you land it on two branches and then I do a merge, that's more work than putting that line onto our branch. Don't you think? In that case, remove the approval so it doesn't land accidentally.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9078218 [details]
Bug 1559199 - Add 'comm' prefix to list of potential index prefixes. r?dustin

Fine by me!

Comment 11

[Jorg K (CEST = GMT+2)]

https://hg.mozilla.org/releases/mozilla-esr68/rev/199420befddc9501c0b4d688cf3390e3092c4074 on THUNDERBIRD_68_VERBRANCH

Comment 12

[Jorg K (CEST = GMT+2)]

https://hg.mozilla.org/releases/mozilla-esr68/rev/bcb12c4a304f5939103f235433112bb22e3f1055 on THUNDERBIRD_68_VERBRANCH for >= 68.1