Closed Bug 1666035 Opened 4 years ago Closed 4 years ago

clangd shows many compilation errors when sccache is in use (example: "In included file: 'algorithm' file not found")

Categories

(Developer Infrastructure :: Developer Environment Integration, defect, P4)

x86_64
macOS

Tracking

(firefox83 fixed)

RESOLVED FIXED
83 Branch
Tracking Status
firefox83 --- fixed

People

(Reporter: mstange, Assigned: mossop)

References

(Blocks 1 open bug)

Details

Attachments

(4 files)

I tried out mach ide vscode today on macOS, and it completed successfully, but now VSCode displays a large amount of build errors.

Some of those build errors complain about not finding std headers.

Attached file My mozconfig file

I invoked the command using MOZCONFIG=.mozconfig-opt mach ide vscode from my srcdir. I have a copy of srcdir/mach at ~/bin/mach.
I am using sccache.

Here's an example of a somewhat simple file, gfx/layers/composite/Diagnostics.cpp. I can't seem to find a way to copy the whole list of errors as text.

The "Clang Language Server" section of the Output pane has the following:

[Error - 5:55:58 p.m.] Request textDocument/hover failed.
  Message: Task was cancelled.
  Code: -32001 

Furthermore, in the process manager I can still see clangd chugging away in the background.

Attachment #9176680 - Attachment description: screenshot of build errors on gfx/layers/Diagnostics.cpp → screenshot of build errors on gfx/layers/composite/Diagnostics.cpp
Attachment #9176682 - Attachment mime type: application/octet-stream → text/plain
Attachment #9176679 - Attachment mime type: application/octet-stream → text/plain

I have created a version of my mozconfig file which does not include the sccache configuration line ac_add_options --with-ccache=/Users/mstange/.mozbuild/sccache/sccache, and re-run mach ide vscode with that.
It works! No more errors.

Summary: clangd does not find standard library headers ("In included file: 'algorithm' file not found") → clangd shows many compilation errors when sccache is in use (example: "In included file: 'algorithm' file not found")

Markus and I discussed this a bit in chat. The problem seems to be related to the use of sccache, which shows up at the beginning of the compile commands in compile_commands.json. It confuses clangd into thinking sccache is the compiler, which leads clangd to be unable to successfully determine what the compiler's built-in include paths are (leading to things like <algorithm> not being resolved).

In my setup, I use ccache (rather than sccache), and this works because clang's libTooling special-cases ccache as a compiler wrapper, so it knows to ignore it in the compiler command and look at the next argument instead.

Possible ways to fix this:

  1. Submit a patch to clang libTooling to include sccache in the list of things they recognize as a compiler wrapper.
  2. Modify our Clangd build backend to strip out the sccache from the generated commands in compile_commands.json.
Severity: -- → S4
Priority: -- → P4

(In reply to Botond Ballo [:botond] from comment #6)

  1. Modify our Clangd build backend to strip out the sccache from the generated commands in compile_commands.json.

Seems likely that this would be the fastest route for now. Would a patch be accepted?

Flags: needinfo?(botond)

(In reply to Dave Townsend [:mossop] from comment #7)

(In reply to Botond Ballo [:botond] from comment #6)

  1. Modify our Clangd build backend to strip out the sccache from the generated commands in compile_commands.json.

Seems likely that this would be the fastest route for now.

Agreed

Would a patch be accepted?

Yep -- thanks! Feel free to flag me or Andi for review.

Flags: needinfo?(botond)
Assignee: nobody → dtownsend

(In reply to Botond Ballo [:botond] from comment #8)

(In reply to Dave Townsend [:mossop] from comment #7)

(In reply to Botond Ballo [:botond] from comment #6)

  1. Modify our Clangd build backend to strip out the sccache from the generated commands in compile_commands.json.

Seems likely that this would be the fastest route for now.

Agreed

Would a patch be accepted?

Yep -- thanks! Feel free to flag me or Andi for review.

Could we write two compile_commands.json, one with and one without sccache? I use the output for some things and I'm not 100% certain that there's a simple transformation from one form to the other...

(In reply to Nick Alexander :nalexander [he/him] from comment #9)

Could we write two compile_commands.json, one with and one without sccache? I use the output for some things and I'm not 100% certain that there's a simple transformation from one form to the other...

We already have ./mach build-backend -b CompileDB to write a "regular" compile_commands.json, and ./mach build-backend -b Clangd to write the one for clangd with non-unified commands. I imagined that stripping sccache would only be done for the clangd one.

(In reply to Botond Ballo [:botond] from comment #10)

(In reply to Nick Alexander :nalexander [he/him] from comment #9)

Could we write two compile_commands.json, one with and one without sccache? I use the output for some things and I'm not 100% certain that there's a simple transformation from one form to the other...

We already have ./mach build-backend -b CompileDB to write a "regular" compile_commands.json, and ./mach build-backend -b Clangd to write the one for clangd with non-unified commands. I imagined that stripping sccache would only be done for the clangd one.

That works for me -- thanks for the reminder, botond.

(In reply to Botond Ballo [:botond] from comment #10)

(In reply to Nick Alexander :nalexander [he/him] from comment #9)

Could we write two compile_commands.json, one with and one without sccache? I use the output for some things and I'm not 100% certain that there's a simple transformation from one form to the other...

We already have ./mach build-backend -b CompileDB to write a "regular" compile_commands.json, and ./mach build-backend -b Clangd to write the one for clangd with non-unified commands. I imagined that stripping sccache would only be done for the clangd one.

Looks like the patch I just pushed doesn't differentiate between the two, any tips on how to do that?

(In reply to Dave Townsend [:mossop] from comment #13)

We already have ./mach build-backend -b CompileDB to write a "regular" compile_commands.json, and ./mach build-backend -b Clangd to write the one for clangd with non-unified commands. I imagined that stripping sccache would only be done for the clangd one.

Looks like the patch I just pushed doesn't differentiate between the two, any tips on how to do that?

Answered in Phabricator.

Pushed by dtownsend@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d0e5422a86b2
Strip ccache compilation wrappers from the clangd compilation database. r=botond
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → 83 Branch
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: