Closed Bug 1262735 Opened 8 years ago Closed 8 years ago

build-clang.py -c clang-static-analysis-linux64-centos6.json is busted, failing to apply patches

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox48 fixed)

RESOLVED FIXED
mozilla48
Tracking Status
firefox48 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(6 files)

It blocks on:

Checked out revision 247539.
The text leading up to this was:
--------------------------
|commit 865b9340996f9f9d04b73b187248737dc6fd845e
|Author: Michael Wu <mwu@mozilla.com>
|Date:   Mon Sep 14 17:47:21 2015 -0400
|
|    Add support for querying the visibility of a cursor
|
|diff --git a/llvm/tools/clang/include/clang-c/Index.h b/llvm/tools/clang/include/clang-c/Index.h
|index fad9cfa..311bfcb 100644
|--- a/llvm/tools/clang/include/clang-c/Index.h
|+++ b/llvm/tools/clang/include/clang-c/Index.h
--------------------------
File to patch: 



This was probably caused by bug 1210154
Once the error in comment 0 is fixed, we hit another error due to bug 1042132:

FAILED: "/home/worker/workspace/build/src/gcc/bin/gcc -static-libgcc" -o
  CMakeFiles/cmTryCompileExec3173220583.dir/testCCompiler.c.o -c
  testCCompiler.c

  /bin/sh: /home/worker/workspace/build/src/gcc/bin/gcc -static-libgcc: No
  such file or directory
Blocks: 1042132
And another...

Traceback (most recent call last):
  File "./build-clang.py", line 402, in <module>
    final_stage_inst_dir)
  File "./build-clang.py", line 122, in build_and_use_libgcc
    run_in(gcc_dir, ["./build-gcc.sh", tempdir, "libgcc"])
  File "./build-clang.py", line 45, in run_in
    os.chdir(path)
OSError: [Errno 2] No such file or directory: '/home/worker/workspace/build/src/build/build-clang/../build-gcc'
Assignee: nobody → mh+mozilla
When passing -DCMAKE_C_COMPILER="gcc -flags" to CMake, ninja then tries
to literally call "gcc -flags", not "gcc" "-flags", and a "gcc -flags"
file obviously doesn't exist, so the build fails.

This fixes a regression from bug 1042132. Hopefully, this also doesn't
regress bug 1042132 itself.

Review commit: https://reviewboard.mozilla.org/r/44951/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/44951/
Blocks: 1262781
... and it looks like bug 1203393 is causing problems too. The resulting clang can't find the C++11 headers that it contains when building the clang-plugin.
Blocks: 1203393
Are our local clang patches not things we could upstream?
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #7)
> Are our local clang patches not things we could upstream?

At least one of them comes from upstream (and goes away with bug 1262781).

(In reply to Mike Hommey [:glandium] from comment #6)
> ... and it looks like bug 1203393 is causing problems too. The resulting
> clang can't find the C++11 headers that it contains when building the
> clang-plugin.

Let me mention that I fixed that issue. I got a working clang 3.8 that works for linux and osx cross builds. I'll attach patches when I know they don't break building the osx native clang.
Comment on attachment 8738957 [details]
MozReview Request: Bug 1262735 - Change the clang paths in patches added in bug 1210154. r?ehsan

https://reviewboard.mozilla.org/r/44949/#review42119
Attachment #8738957 - Flags: review?(ehsan) → review+
Comment on attachment 8738958 [details]
MozReview Request: Bug 1262735 - Separate compiler and flags when passing them to CMake. r?ehsan

https://reviewboard.mozilla.org/r/44951/#review42121
Attachment #8738958 - Flags: review?(ehsan) → review+
Comment on attachment 8738959 [details]
MozReview Request: Bug 1262735 - Fix the path to build-gcc used by build-clang. r?ehsan

https://reviewboard.mozilla.org/r/44953/#review42123
Attachment #8738959 - Flags: review?(ehsan) → review+
Since build-clang.py requires a gcc_dir to be set, and we're using GCC
from there to build clang, we might as well copy libgcc from there
instead of building a fresh GCC. On the taskcluster job building clang,
GCC comes from a tooltool package that is already the result of
build-gcc anyways.

Review commit: https://reviewboard.mozilla.org/r/45635/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45635/
Attachment #8740217 - Flags: review?(ehsan)
Attachment #8740218 - Flags: review?(ehsan)
Attachment #8740219 - Flags: review?(ehsan)
Instead, copy libgcc from the GCC used to build clang at each stage.
When passing --gcc-toolchain, the flag ends up appearing in the output
of llvm-config, and completely defeats the purpose of copying libgcc in
clang/lib/gcc.

Review commit: https://reviewboard.mozilla.org/r/45637/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45637/
Both point to the same path for cc and cxx, but not for gcc_dir, which
makes no sense. That's the only significant difference between both, so
just merge them both, and use the merged file in the taskcluster build
script.

Review commit: https://reviewboard.mozilla.org/r/45639/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45639/
Comment on attachment 8738957 [details]
MozReview Request: Bug 1262735 - Change the clang paths in patches added in bug 1210154. r?ehsan

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/44949/diff/1-2/
Comment on attachment 8738958 [details]
MozReview Request: Bug 1262735 - Separate compiler and flags when passing them to CMake. r?ehsan

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/44951/diff/1-2/
Comment on attachment 8738959 [details]
MozReview Request: Bug 1262735 - Fix the path to build-gcc used by build-clang. r?ehsan

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/44953/diff/1-2/
This patch queue doesn't appear to have broken the script on mac.
(In reply to Mike Hommey [:glandium] from comment #18)
> This patch queue doesn't appear to have broken the script on mac.

Awesome!  FWIW I previously had to build on a 10.7 machine since it didn't work on any other OS.  If you need me to, I'd be happy to try to build there again and fix any issues that I possibly find after you land your changes here.
Attachment #8740217 - Flags: review?(ehsan) → review+
Comment on attachment 8740217 [details]
MozReview Request: Bug 1262735 - Copy libgcc from the GCC used to build clang instead of building a new GCC. r?ehsan

https://reviewboard.mozilla.org/r/45635/#review42391

I'm fine with this, can you please answer the question below and edit the commit message accordingly?

::: build/build-clang/build-clang.py:120
(Diff revision 1)
> -        run_in(gcc_dir, ["./build-gcc.sh", tempdir, "libgcc"])
> -        run_in(tempdir, ["tar", "-xf", "gcc.tar.xz"])
> -        libgcc_dir = glob.glob(os.path.join(tempdir,
> -                                            "gcc", "lib", "gcc",
>                                              "x86_64-unknown-linux-gnu",
>                                              "[0-9]*"))[0]

Does this not break when building from source without stuff being pulled down from tooltool?
Comment on attachment 8740218 [details]
MozReview Request: Bug 1262735 - Don't pass --gcc-toolchain to build clang. r?ehsan

https://reviewboard.mozilla.org/r/45637/#review42395

::: build/build-clang/build-clang.py:367
(Diff revision 1)
>          [cxx] + extra_cxxflags,
>          llvm_source_dir, stage1_dir, build_libcxx,
>          build_type, assertions, python_path)
>  
> +    if is_linux():
> +        install_libgcc(gcc_dir, final_stage_dir + '/clang')

I think it's a bit cleaner if you moved this at the end of build_one_stage().  Can you please do that?
Attachment #8740218 - Flags: review?(ehsan) → review+
Comment on attachment 8740219 [details]
MozReview Request: Bug 1262735 - Merge both clang-static-analysis-linux64*.json files. r?ehsan

https://reviewboard.mozilla.org/r/45639/#review42397
Attachment #8740219 - Flags: review?(ehsan) → review+
https://reviewboard.mozilla.org/r/45635/#review42391

> Does this not break when building from source without stuff being pulled down from tooltool?

Actually, it can be made to work with a slight modification. Specifically, if I replace x86_64-unknown-linux-gnu with x86_64-*linux-gnu,  I can then use /usr as gcc_dir, although the glob + [0] might pick the wrong version if you have multiple gcc versions installed. A better way to do it would be to use the output from gcc --print-libgcc-file-name. I'll file a followup bug.
Blocks: 1264132
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.