tl;dr: It seems that the bot is dependent on the `exchange/taskcluster-queue/v1/task-group-resolved` pulse queue resolving in a timely fashion. But in the event there are things that never run, the group won't resolve until its deadline. It appears that there were a bunch of tier 3 "Bg" jobs like "build-linux64-gcp/opt" that got scheduled but never ran. I've asked in the channel if there's an easy way for the bot to avoid caring about the tier3 jobs by choosing a different queue. So as a basic timeline of an example of Saturday's very delayed coverage report: - https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=searchfox&revision=b579c12907dc3bb210177454a696ccc181c1ded0 is the Saturday afternoon push to mozilla-central. - `2020-08-08T21:33:50.820Z` - Decision task [RrtxoDFqSUKiPyyKj--Wdw](https://firefox-ci-tc.services.mozilla.com/tasks/RrtxoDFqSUKiPyyKj--Wdw) is created and promptly runs which is what gives us the `primary.RrtxoDFqSUKiPyyKj--Wdw.gecko-level-3._` routing key message that will eventually trigger the bot. - `2020-08-08T21:38:41.869Z` - An M1 mochitest coverage task [W09jrnyITZeo828Ddccg4w](https://firefox-ci-tc.services.mozilla.com/tasks/W09jrnyITZeo828Ddccg4w) is created and promptly runs - `2020-08-08T21:38:41.869Z` created - `2020-08-08T22:01:42.493Z` scheduled - `2020-08-08T22:39:47.382Z` started - `2020-08-08T23:00:17.079Z` resolved - `2020-08-09T21:41:01.832Z` - The coverage bot hook task [Fp7LhNPcTRqebSMoEJOXfA](https://firefox-ci-tc.services.mozilla.com/tasks/Fp7LhNPcTRqebSMoEJOXfA) is created. - `2020-08-09T21:41:01.832Z` created - `2020-08-09T21:41:01.866Z` scheduled - `2020-08-09T21:41:02.650Z` started - `2020-08-09T23:43:06.839Z` resolved Chat log links: - #taskcluster - Initial questions and me thinking aloud on Saturday: https://matrix.to/#/!whDRjjSmICCgrhFHsQ:mozilla.org/$gDd5sweCehsSj-dMOSq2Q3hcc6_M6ptlk-h2ctoD7kw?via=mozilla.org&via=matrix.org&via=t2bot.io - Discussion of the tasks queue issue on Monday: https://matrix.to/#/!whDRjjSmICCgrhFHsQ:mozilla.org/$VOcAKJi_6f3lqKsWuTffs7QvUYaGMFFk6bpXy6Nmx6Q?via=mozilla.org&via=matrix.org&via=t2bot.io It does seem
Bug 1566874 Comment 16 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
tl;dr: It seems that the bot is dependent on the `exchange/taskcluster-queue/v1/task-group-resolved` pulse queue resolving in a timely fashion. But in the event there are things that never run, the group won't resolve until its deadline. It appears that there were a bunch of tier 3 "Bg" jobs like "build-linux64-gcp/opt" that got scheduled but never ran. I've asked in the channel if there's an easy way for the bot to avoid caring about the tier3 jobs by choosing a different queue. So as a basic timeline of an example of Saturday's very delayed coverage report: - https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=searchfox&revision=b579c12907dc3bb210177454a696ccc181c1ded0 is the Saturday afternoon push to mozilla-central. - `2020-08-08T21:33:50.820Z` - Decision task [RrtxoDFqSUKiPyyKj--Wdw](https://firefox-ci-tc.services.mozilla.com/tasks/RrtxoDFqSUKiPyyKj--Wdw) is created and promptly runs which is what gives us the `primary.RrtxoDFqSUKiPyyKj--Wdw.gecko-level-3._` routing key message that will eventually trigger the bot. - `2020-08-08T21:38:41.869Z` - An M1 mochitest coverage task [W09jrnyITZeo828Ddccg4w](https://firefox-ci-tc.services.mozilla.com/tasks/W09jrnyITZeo828Ddccg4w) is created and promptly runs - `2020-08-08T21:38:41.869Z` created - `2020-08-08T22:01:42.493Z` scheduled - `2020-08-08T22:39:47.382Z` started - `2020-08-08T23:00:17.079Z` resolved - `2020-08-09T21:33:50.820Z` The decision task/task group's deadline. - `2020-08-09T21:41:01.832Z` - The coverage bot hook task [Fp7LhNPcTRqebSMoEJOXfA](https://firefox-ci-tc.services.mozilla.com/tasks/Fp7LhNPcTRqebSMoEJOXfA) is created. - `2020-08-09T21:41:01.832Z` created - `2020-08-09T21:41:01.866Z` scheduled - `2020-08-09T21:41:02.650Z` started - `2020-08-09T23:43:06.839Z` resolved Chat log links: - #taskcluster - Initial questions and me thinking aloud on Saturday: https://matrix.to/#/!whDRjjSmICCgrhFHsQ:mozilla.org/$gDd5sweCehsSj-dMOSq2Q3hcc6_M6ptlk-h2ctoD7kw?via=mozilla.org&via=matrix.org&via=t2bot.io - Discussion of the tasks queue issue on Monday: https://matrix.to/#/!whDRjjSmICCgrhFHsQ:mozilla.org/$VOcAKJi_6f3lqKsWuTffs7QvUYaGMFFk6bpXy6Nmx6Q?via=mozilla.org&via=matrix.org&via=t2bot.io It does seem