Closed Bug 1643335 Opened 5 years ago Closed 5 years ago

Run hazard-linux64-haz/debug less often

Categories

(Testing :: General, task)

task

Tracking

(firefox79 fixed)

RESOLVED FIXED
mozilla79
Tracking Status
firefox79 --- fixed

People

(Reporter: Sylvestre, Assigned: Sylvestre)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

According to https://console.cloud.google.com/bigquery?project=moz-fx-data-taskclu-prod-8fbf&j=bq:US:bquxjob_4c635c74_1727f35a25d&page=queryresults (thanks Ben), hazard-linux64-haz/debug is one of the most expensive build job.

According to
https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=WkPx8G6cRii1D4o-0C1Gaw-0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunning%2Cpending%2Crunnable&tochange=7516aaa595a0c9c7223e4281185233ca6b87a980&fromchange=72129cde7d7fe8d718ce1836bd5f016dd6808e08 (thanks Aryx), over the last 120 days, only one patch has been backout because this job failed (and the rest of the jobs have been disabled - so, maybe other jobs would have found it).

Discussing with Jonco, it is a very valuable job. So, we need to keep it running but we don't have to run it that often.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE

(In reply to Sylvestre Ledru [:Sylvestre] from comment #0)

maybe other jobs would have found it

Sadly this is not the case - this analysis finds bugs that no other jobs will find.

Sadly this is not the case - this analysis finds bugs that no other jobs will find.

Sorry, my wording was terrible.
What I meant here is that hazard-linux64-haz could have failed because of a stupid build failure. linux64 opt/debug would have failed too.
I have been interested in jobs during which everything was green but only hazard-linux64-haz failed (where it has a great value)

Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---

A couple of things:

  • It makes sense for the hazard build to be the most expensive build job: it has to do a full instrumented compile plus the analysis itself
  • it could be sped up slightly by modifying the build system to stop after compiling (and thus skip linking)
  • the compile cannot be incremental or cached with sccache; the compiler needs to generate its internal data structures for every source file in the build
  • it is expected that most hazard failures will be found on the try server before they make it to autoland -- but this depends on whether people are selecting the hazard build for their try pushes. Try syntax will select it whenever -p linux64 is used, but try fuzzy won't automatically pull it in with anything else.
  • the hazard build catches issues that often turn out to be security flaws

It is theoretically possible to make this build incremental or even cachable, but the intermediate files are huge and it would be a fair amount of work. And if we're going to drop down the frequency, it'll stop helping anyway.

I have no reason to challenge the decision on frequency. If it only had one unique failure ("unique" as in no other job would have caught it) in the last 120 days, it totally makes sense to me to drop its frequency. As long as it ends up running on all of the code committed, it'll find everything it can find. There's no need for it to see every change individually.

Pushed by sledru@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/d51c3c722d65 Run hazard-linux64-haz/debug less often r=jmaher,sfink
Pushed by sledru@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/c5e87f7e82af Run hazard-linux64-haz/debug less often r=jmaher,sfink
Pushed by sledru@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e2ae71a2a691 Run hazard-linux64-haz/debug less often r=jmaher,sfink
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla79

Fixed!

Flags: needinfo?(sledru)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: