Closed Bug 1522223 Opened 10 months ago Closed 10 months ago

Intermittent bustages with error: installation or configuration problem: C++ compiler cannot create executables.

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox66 fixed, firefox67 fixed)

RESOLVED FIXED
mozilla67
Tracking Status
firefox66 --- fixed
firefox67 --- fixed

People

(Reporter: CosminS, Assigned: chmanchester)

References

Details

(Keywords: in-triage, intermittent-failure)

Attachments

(1 file)

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=223513171&repo=autoland&lineNumber=40766

Th: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&searchStr=bo&revision=21cb7caef246d374a8f73a14806d9fd328adbf8a&selectedJob=223505284

16:05:04 INFO - Running post-run listener: _parse_build_tests_ccov
16:05:04 INFO - Running post-run listener: _shutdown_sccache
16:05:04 INFO - Running command: ['z:\build\build\src\sccache2\sccache.exe', '--stop-server'] in z:\build\build\src
16:05:04 INFO - Copy/paste: z:\build\build\src\sccache2\sccache.exe --stop-server
16:05:04 INFO - Stopping sccache server...
16:05:04 INFO - Compile requests 4530
16:05:04 INFO - Compile requests executed 4387
16:05:04 INFO - Cache hits 4291
16:05:04 INFO - Cache misses 64
16:05:04 INFO - Cache timeouts 0
16:05:04 INFO - Cache read errors 0
16:05:04 INFO - Forced recaches 0
16:05:04 INFO - Cache write errors 0
16:05:04 INFO - Compilation failures 1
16:05:04 INFO - Cache errors 31
16:05:04 INFO - Non-cacheable compilations 0
16:05:04 INFO - Non-cacheable calls 125
16:05:04 INFO - Non-compilation calls 18
16:05:04 INFO - Unsupported compiler calls 0
16:05:04 INFO - Successful distributed compilations 0
16:05:04 INFO - Failed distributed compilations 0
16:05:04 INFO - Average cache write 0.179 s
16:05:04 INFO - Average cache read miss 34.917 s
16:05:04 INFO - Average cache read hit 0.096 s
16:05:04 INFO - Non-cacheable reasons:
16:05:04 INFO - -Fi 2
16:05:04 INFO - crate-type 69
16:05:04 INFO - unknown source language 1
16:05:04 INFO - - 10
16:05:04 INFO - -fsyntax-only 38
16:05:04 INFO - multiple input files 5
16:05:04 INFO - Cache location S3, bucket: Bucket(name=taskcluster-level-3-sccache-us-east-1, base_url=http://taskcluster-level-3-sccache-us-east-1.s3.amazonaws.com/)
16:05:04 INFO - Return code: 0
16:05:04 INFO - Running post-run listener: _summarize
16:05:04 ERROR - # TBPL FAILURE #

Summary: Intermittent BO bustages with error: installation or configuration problem: C++ compiler cannot create executables. → Intermittent bustages with error: installation or configuration problem: C++ compiler cannot create executables.

I've got something similar on try today, and the peculiar and notable thing that happens is that configure is being re-run during the check-test phase, which is not supposed to happen.

The log says why:
16:04:42 INFO - z:/build/build/src/obj-firefox/config.status is out of date with respect to z:/build/build/src/obj-firefox/.mozconfig.json

Sounds like a timestamp granularity problem...

Flags: needinfo?(cmanchester)

I can't tell if it's related to this intermittent as filed, the Windows builds are pretty much all re-running configure before make check now. It looks like this is due to bug 1477306: when we run |./mach build| for the build we get "CXXFLAGS= -fcrash-diagnostics-dir=z:/build/public/build" from the mozconfig, when we run it for make check we don't, so the mozconfig is legitimately different and configure re-runs itself.

Flags: needinfo?(cmanchester)

Is it normal that make check doesn't reflect the mozconfig's C[XX]FLAGS?

I guess it's more likely that UPLOAD_PATH isn't set for the make check.

(In reply to David Major [:dmajor] from comment #4)

I guess it's more likely that UPLOAD_PATH isn't set for the make check.

Right... mozharness builds a different environment for make check that doesn't seem to include UPLOAD_PATH.

Blocks: 1477306
Keywords: in-triage
See Also: → 1403607
Assignee: nobody → cmanchester
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/85614e63fa3b
Set UPLOAD_PATH during make check to avoid re-running configure. r=Callek
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
You need to log in before you can comment on or make changes to this bug.