Closed Bug 1376593 Opened 3 years ago Closed 3 years ago

sccache makes compiler error messages useless ("cc1plus: error: to generate dependencies you must specify either -M or -MM")

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox56 fixed)

RESOLVED FIXED
mozilla56
Tracking Status
firefox56 --- fixed

People

(Reporter: emk, Assigned: ted)

References

Details

Attachments

(1 file)

Recently, every time I see a build bustage in automation, I see:
  cc1plus: error: to generate dependencies you must specify either -M or -MM
that is totally useless to find what is wrong. For example, see <https://treeherder.mozilla.org/logviewer.html#?job_id=110023178&repo=try&lineNumber=8000>.

If I'm lucky, clang builds will fail with the same error and I can spot the error:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4b0a70eac8686b5a114f2755854a23f211a5aa35&selectedJob=110023174

If I'm unlucky, it is a really painful guesswork to find the error (for example, bug 1373984 comment #11). It has surprised me that nobody complains about this cryptic error message.

Is this a bug of GCC? If so, we should upgrade GCC in automation. Or is this our fault? If so, what can we do to get more decent messages?
2017-06-28T23:57:08: DEBUG : Starting merge handling...
2017-06-28T23:57:08: DEBUG : Using url: https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?changeset=4e3a5199d4fed80eab8dfd8d06ebe07513dd0587&full=1
2017-06-28T23:57:09: DEBUG : Found commit message:
bug 1357825 - use sccache for caching Rust compilation. r=froydnj

MozReview-Commit-ID: 84PCmiVBlrV

2017-06-28T23:57:09: INFO : The bisection is done.
2017-06-28T23:57:09: INFO : Stopped
Blocks: 1357825
Flags: needinfo?(ted)
Flags: needinfo?(nfroyd)
Summary: GCC (in automation?) generates a useless message for compiler errors → sccache makes compiler error messages useless
We believe this is a regression from https://github.com/mozilla/sccache/commit/0a281848d137bec11322c66b5726d2f508ac8a02 .

Prior to this, if gcc compilation failed with -Werror, we would not retry it. Now we'll retry it, but we're not properly handling some preprocessor arguments (-MP in this specific case), so we're passing gcc a confused set of arguments and it errors.
Assignee: nobody → ted
Flags: needinfo?(ted)
Ted has this under control!
Flags: needinfo?(nfroyd)
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> Prior to this, if gcc compilation failed with -Werror, we would not retry
> it. Now we'll retry it, but we're not properly handling some preprocessor
> arguments (-MP in this specific case), so we're passing gcc a confused set
> of arguments and it errors.

Does it mean the retry is not functioning effectively? (Because the retry will always fail anyway.)
When will this be fixed? If it will take some time, please revert the mozilla/sccache#110 change until the proper retry logic is implemented because the current status is strictly worse than before.
Flags: needinfo?(ted)
I have a patch:
https://github.com/mozilla/sccache/pull/145
Flags: needinfo?(ted)
(In reply to Masatoshi Kimura [:emk] from comment #4)
> (In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> > Prior to this, if gcc compilation failed with -Werror, we would not retry
> > it. Now we'll retry it, but we're not properly handling some preprocessor
> > arguments (-MP in this specific case), so we're passing gcc a confused set
> > of arguments and it errors.
> 
> Does it mean the retry is not functioning effectively? (Because the retry
> will always fail anyway.)

There was a bug in the argument handling that made the retry not use the correct set of arguments. This same bug existed in the clang code, but clang doesn't error in this circumstance.
Duplicate of this bug: 1377211
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #6)
> There was a bug in the argument handling that made the retry not use the
> correct set of arguments. This same bug existed in the clang code, but clang
> doesn't error in this circumstance.

Then please backout <https://hg.mozilla.org/mozilla-central/rev/4e3a5199d4fe> or turn it off at least for gcc ASAP until the sccache bug is fixed.
Flags: needinfo?(ted)
Duplicate of this bug: 1378467
Comment on attachment 8883686 [details]
bug 1376593 - update sccache to 69334a26ba65fc88e3934271a2ce6781c51b445e to fix a regression.

https://reviewboard.mozilla.org/r/154596/#review159676

Stealing review while gps is out, so this can be landed sooner rather than later.
Attachment #8883686 - Flags: review+
My apologies, this was just hard to get done in the confines of the work week, and I neglected to finish it off Monday.
Flags: needinfo?(ted)
[Adding the bogus error message to bug summary to make this bug more discoverable via search / new-bug-form dupe-suggestions]
Summary: sccache makes compiler error messages useless → sccache makes compiler error messages useless ("cc1plus: error: to generate dependencies you must specify either -M or -MM")
I had a few broken builds on that try push, but nothing seemed related to the sccache update. I retriggered all the failing builds and the second attempt was green, so I'm going to land this.
Attachment #8883686 - Flags: review?(gps)
Pushed by tmielczarek@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4f6450b87658
update sccache to 69334a26ba65fc88e3934271a2ce6781c51b445e to fix a regression. r=froydnj
Sorry about all the annoyance here. I added a test to sccache's test suite that will prevent us from breaking things in this particular way again. Obviously you'll need to update your tree to pull in this changeset to fix this issue in your try pushes.
https://hg.mozilla.org/mozilla-central/rev/4f6450b87658
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.