1758746 - searchfox mozilla-central runs frequently lack coverage data because l10n bumper bot runs are DONTBUILD

Status: RESOLVED FIXED

(Webtools :: Searchfox, defect)

Description

[Andrew Sutherland [:asuth] (he/him)]

:marco pointed out in https://chat.mozilla.org/#/room/#codecoverage:mozilla.org that many of the instances of missing searchfox mozilla-central coverage data stem from the searchfox indexer runs (and the nightly builds) being run against no bug - Bumping Firefox l10n changesets r=release a=l10n-bump DONTBUILD revisions which explicitly do not have coverage data.

It seems like there are a few options:

• Bug 1752111 makes it so the l10n bumper bot pushes to autoland instead of mozilla-central, largely side-stepping this issue.
  • Not perfectly side-stepping it, though, as we do see commits like No bug - tagging b20c5a948aaf5c9b8ca0819ff6d8ab83d0a0af8b with FIREFOX_BETA_99_BASE a=release DONTBUILD CLOSED TREE which are DONTBUILD but not the l10n bumper.
• Make the searchfox job have dependencies on all the coverage data that it now really likes to have available, at least on mozilla-central.
  • Because the coverage jobs are basically all of the test jobs, etc. I presume this would involve invoking some other dependency taskgraph?
• Make searchfox jobs run on every mozilla-central push rather than based on a cron rule (and which would not fire when DONTBUILD is specified, I think). This would make searchfox runs independent of the exact revision that nightly corresponds to, but the deviations would be minor and arguably whoever chose DONTBUILD is asserting that.
  • This should not actually require changes in the AWS logic for build selection, as the "latest" would never include the DONTBUILD in that case.
  • This would slightly increase the number of wasted searchfox jobs on a daily basis, but on average it seems like this would correspond exactly to 1 revision's worth.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Attachment: Bug 1758746 - Make searchfox run on-push for mozilla-central

In order to help ensure that searchfox runs for mozilla-central will
always have code coverage information available, run the searchfox jobs
on-push instead of on a cron schedule. This avoids searchfox jobs
being scheduled against DONTBUILD pushes which will lack the coverage
jobs which are normally scheduled on-push.

A better/more thorough thing to do would be to express soft
dependencies "soft-dependencies" as documented at
https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/task-graphs.html#soft-dependencies
on these coverage tasks. However, for now, we just hope really hard
that the coverage tasks get scheduled (as they should? :).

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b51a7271e5f
Make searchfox run on-push for mozilla-central r=ahal

Comment 3

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/3b51a7271e5f

Comment 4

[Andrew Sutherland [:asuth] (he/him)]

Treeherder confirms the mozsearch job run on the merge, as expected. I'm removing the dependency on the l10n pusher behavior change as we ended up not being dependent on that.