Closed Bug 1334810 Opened 3 years ago Closed 3 years ago

No gecko-decision task on ffxbld pushes

Categories

(Taskcluster :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: philor, Assigned: dustin)

References

Details

Attachments

(1 file)

I don't have a wide enough sample to know which possibility is the case, but the possibilities seem to be

* something about the periodic update pushes to central and aurora after bug 1332930 unbroke them following the switch to TC nightlies and then unbroke them so they would actually find builds causes them to get no gecko-decision task run, or to get one run but to have it fail so catastrophically that even the failure isn't reported to treeherder

* sometime during the (US) day on 2017-01-27, running gecko-decision on pushes where the owner isn't an email address broke

What data there is are four periodicupdate pushes to central and aurora, https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=045d8fe30f546ab08466c9586ce298e6459c2069 https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=83336666dfce1ad037f1f1bd881f43bfdc266831 https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=e11c1e5538f15f82188f9ddad113b8bf5e8d68e7 https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=87c8697b2039d816c1531e5a1edd698933b38663 which all failed to get a gecko-decision job, and https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr45&revision=e0f9ac84e5c509b50f5a30f0111bcf8a972f6f48 from the morning of 2017-01-27 showing it still worked at that point.

Inconveniently, fixing periodicupdate for central and aurora broke it for beta and release and esr45, so we don't have any "see, this push by this user worked on Friday and was broken on Saturday" smoking guns to say that all ffxbld pushes are broken, and won't until either those periodicupdate jobs are fixed or the next time we tag a release, since that's the other thing ffxbld pushes, releases.
This definitely seems related to the task owner not being an email address.  Found this in the mozilla-taskcluster logs:

Error calling: createTask NOT retrying!, info: {"code":"InputValidationError","message":"\nSchema Validation Failed!\nRejecting Schema: http://schemas.taskcluster.net/queue/v1/create-task-request.json#\nErrors:\n  * data.metadata.owner should match format \"email\"

Not sure what changed with how we scheduled these decisions tasks that might have caused this.

https://papertrailapp.com/systems/mozilla-taskcluster/events?highlight=762042211708858434&focus=762042211708858434
Bug 1303556 "fixes" it into an email in the decision task itself, fwiw.
https://hg.mozilla.org/mozilla-central/diff/459fd779b235/.taskcluster.yml
 tasks:
   - taskId: '{{#as_slugid}}decision task{{/as_slugid}}'
     task:
       created: '{{now}}'
       deadline: '{{#from_now}}1 day{{/from_now}}'
       expires: '{{#from_now}}365 day{{/from_now}}'
       metadata:
-        owner: mozilla-taskcluster-maintenance@mozilla.com
+        owner: {{owner}}

(change made since I don't think mozilla-taskcluster-maintenance goes anywhere...)

reverting that particular hunk should fix this issue.  Implementing something like bug 1303556 in mozilla-taskcluster would be more robust.

I'll be lucky to read all my bugspam today, so I won't be able to fix this quickly, but hopefully that provides enough context.
Duplicate of this bug: 1337408
This needs to be higher priority now that we're shipping TC-based nightly builds.
Assignee: nobody → dustin
Keywords: leave-open
Comment on attachment 8834443 [details]
Bug 1334810: temporarily revert {{owner}};

https://reviewboard.mozilla.org/r/110390/#review111616
Attachment #8834443 - Flags: review?(garndt) → review+
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/43bf48f50829
temporarily revert {{owner}}; r=garndt
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.