1123386 - Upgrade clang on Linux and OSX static-analysis builders to tip-of-tree

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Comment 1

[(no longer active)]

Attachment: Update the clang build script for the most recent clang

Comment 2

[Mike Hommey [:glandium]]

Comment on attachment 8551377 [details] [diff] [review]
Update the clang build script for the most recent clang

Review of attachment 8551377 [details] [diff] [review]:
-----------------------------------------------------------------

Please don't land this without landing the corresponding tooltool manifest change.

::: build/unix/build-clang/build-clang.py
@@ +179,5 @@
>      if is_darwin():
>          extra_cflags = ""
> +        extra_cxxflags = "-stdlib=libc++"
> +    else:
> +        extra_cflags = ""

There's a reason why -static-libgcc -static-libstdc++ was used, and I don't think this should change. Without those, the resulting clang will likely fail to execute on the builders.

::: build/unix/build-clang/no-sse-on-linux-trunk.patch
@@ -6,5 @@
> -   // All x86 devices running Android have core2 as their common
> -   // denominator. This makes a better choice than pentium4.
> -   if (Triple.getEnvironment() == llvm::Triple::Android)
> --    return "core2";
> -+    return "686";

I wonder why we were changing this in the first place, as it only affects android...

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 8551377 [details] [diff] [review]
Update the clang build script for the most recent clang

My review would be a rubber stamp. glandium sounds like a candidate! :)

btw, if you want to try the binaries, grab the manifest and the binary from:
http://people.mozilla.org/~raliiev/clang.manifest
http://people.mozilla.org/~raliiev/89711c7040c7f6c02bd84d6b535d6a2dab78dd7efb7fa20020d2506baaff5d62160bb32302529bde778940b4a4a65a5934f99f582e62c134f854d6922edac74b

Comment 4

[(no longer active)]

(In reply to Mike Hommey [:glandium] from comment #2)
> Comment on attachment 8551377 [details] [diff] [review]
> Update the clang build script for the most recent clang
> 
> Review of attachment 8551377 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Please don't land this without landing the corresponding tooltool manifest
> change.

Note that I am not planning to update all of the clangs we use in various places, just the ones for static analysis.  So I'm not sure if there is a good reason to hold off.

> ::: build/unix/build-clang/build-clang.py
> @@ +179,5 @@
> >      if is_darwin():
> >          extra_cflags = ""
> > +        extra_cxxflags = "-stdlib=libc++"
> > +    else:
> > +        extra_cflags = ""
> 
> There's a reason why -static-libgcc -static-libstdc++ was used, and I don't
> think this should change. Without those, the resulting clang will likely
> fail to execute on the builders.

Hmm, I seem to remember the build output saying that these flags are unused...  But reading the clang code, they're most definitely not.  :/  Not sure what happened, but this probably explains why the binaries didn't run on the builders.  My bad.

> ::: build/unix/build-clang/no-sse-on-linux-trunk.patch
> @@ -6,5 @@
> > -   // All x86 devices running Android have core2 as their common
> > -   // denominator. This makes a better choice than pentium4.
> > -   if (Triple.getEnvironment() == llvm::Triple::Android)
> > --    return "core2";
> > -+    return "686";
> 
> I wonder why we were changing this in the first place, as it only affects
> android...

I have no idea!  But anyway a better version of this patch has been merged upstream, which is why I removed it.

Comment 5

[(no longer active)]

Even while preserving -static-libgcc -static-libstdc++, the binaries I produce on my machine (since I couldn't get the vagrant solution to work) do not work on the build machines:

<http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-83070c670ef1/try-linux64-st-an-debug/try-linux64-st-an-debug-bm76-try1-build4153.txt.gz>
07:07:01     INFO -  configure:3370: checking whether the C compiler (/builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang  ) works
07:07:01     INFO -  configure:3386: /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang -o conftest    conftest.c  1>&5
07:07:01     INFO -  /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang: /lib64/libz.so.1: no version information available (required by /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang)
07:07:01     INFO -  /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang)
07:07:01     INFO -  /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang)
07:07:01     INFO -  configure: failed program was:
07:07:01     INFO -  #line 3381 "configure"
07:07:01     INFO -  #include "confdefs.h"
07:07:01     INFO -  main(){return(0);}
07:07:01     INFO -  configure: error: installation or configuration problem: C compiler cannot create executables.

Any idea how I can fix this?

Comment 6

[(no longer active)]

Or maybe you really need to do these builds on the vagrant box?

Rail, can you please apply this patch on top of the previous one and see if you can give me new builds to try?  Thanks!

diff --git a/build/unix/build-clang/build-clang.py b/build/unix/build-clang/build-clang.py
index c97e40d..a753ba0 100755
--- a/build/unix/build-clang/build-clang.py
+++ b/build/unix/build-clang/build-clang.py
@@ -180,8 +180,8 @@ if __name__ == "__main__":
         extra_cflags = ""
         extra_cxxflags = "-stdlib=libc++"
     else:
-        extra_cflags = ""
-        extra_cxxflags = ""
+        extra_cflags = "-static-libgcc"
+        extra_cxxflags = "-static-libgcc -static-libstdc++"
         cc = gcc_dir + "/bin/gcc"
         cxx = gcc_dir + "/bin/g++"

Comment 7

[Rail Aliiev [:rail]]

Sure. I'll reset the needinfo flag whenever I have them.

Comment 8

[Trevor Saunders (:tbsaunde)]

(In reply to :Ehsan Akhgari [Away: 1/29-2/20] (not reading bugmail, needinfo? me!) from comment #6)
> Or maybe you really need to do these builds on the vagrant box?

you need to build against the same or earlier version of glibc than you want to be able to dynamically link to, and your pretty clearly not doing that.  Its probably easiest to build on a centos vm / container / chroot.
its pretty clear

Comment 9

[Rail Aliiev [:rail]]

http://people.mozilla.org/~raliiev/024dbdc973ea4c471efdc999464b936cbe61f57465364a0333529d4477cabfb4ab3baeb292b0b30cc45d359be538cf440ee321b0f8984c4b59e130c12f1dc1f7

Comment 10

[(no longer active)]

(In reply to Rail Aliiev [:rail] from comment #9)
> http://people.mozilla.org/~raliiev/
> 024dbdc973ea4c471efdc999464b936cbe61f57465364a0333529d4477cabfb4ab3baeb292b0b
> 30cc45d359be538cf440ee321b0f8984c4b59e130c12f1dc1f7

This didn't work either, now because the linker can't find libc++: <http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-a7b245fcdcf0/try-linux64-st-an-debug/try-linux64-st-an-debug-bm75-try1-build3954.txt.gz>

I think I need to find a way to run these vagrant images...

Comment 11

[(no longer active)]

OK I finally got the vagrant image to work, and now I get:

[vagrant@clang-centos62-64 build-clang]$ ./build-clang.py
Traceback (most recent call last):
  File "./build-clang.py", line 14, in <module>
    import argparse
ImportError: No module named argparse
[vagrant@clang-centos62-64 build-clang]$ pip install argparse
-bash: pip: command not found

Rail, can you please tell me how you did those builds?  I'm not familiar with CentOS at all.  :/

Comment 12

[(no longer active)]

nm, figured it out!

Comment 13

[(no longer active)]

Now, I get:

llvm[4]: Compiling CGAtomic.cpp for Release build
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/Sema/Release/AnalysisBasedWarnings.d.tmp': No such file or directory
make[4]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/Sema/Release/AnalysisBasedWarnings.o] Error 1
make[4]: Leaving directory `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/Sema'
make[3]: *** [Sema/.makeall] Error 2
make[3]: *** Waiting for unfinished jobs....
llvm[4]: Compiling CGBlocks.cpp for Release build
llvm[4]: Compiling ParseDeclCXX.cpp for Release build
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/AST/Release/ASTContext.d.tmp': No such file or directory
make[4]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/AST/Release/ASTContext.o] Error 1
make[4]: Leaving directory `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/AST'
make[3]: *** [AST/.makeall] Error 2
llvm[4]: Compiling ParseExpr.cpp for Release build
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/CodeGen/Release/CGAtomic.d.tmp': No such file or directory
make[4]: *** [/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/CodeGen/Release/CGAtomic.o] Error 1
make[4]: *** Waiting for unfinished jobs....
llvm[4]: Compiling ParseExprCXX.cpp for Release build
llvm[4]: Compiling ParseInit.cpp for Release build
make[4]: Leaving directory `/builds/slave/moz-toolchain/build/stage1/build/tools/clang/lib/CodeGen'
make[3]: *** [CodeGen/.makeall] Error 2

Rail, since you have done these builds on the same VM, can you please give me a detailed set of steps as to how to do the builds?  Including the exact VM state, the packages you need to install, the gcc version you used (I used the one in /tools/gcc-4.7.2-0moz1), etc.?  Thanks!

Comment 14

[Rail Aliiev [:rail]]

I think the only step that is missing at https://wiki.mozilla.org/ReleaseEngineering/How_To/Clang_update is bumping the VM memory from default (512?) to 2G. Probably it'd be better to distribute Vagrantfiles instead.

Can you shut down the VM and using the VirtualBox UI bump memory and maybe share more than 1 CPU?

Comment 15

[Rail Aliiev [:rail]]

(In reply to Rail Aliiev [:rail] from comment #14)
> I think the only step that is missing at
> https://wiki.mozilla.org/ReleaseEngineering/How_To/Clang_update is bumping
> the VM memory from default (512?) to 2G. Probably it'd be better to
> distribute Vagrantfiles instead.

I updated the docs: https://wiki.mozilla.org/index.php?title=ReleaseEngineering%2FHow_To%2FClang_update&diff=1049851&oldid=1048894

Nothing special, I just added Vagrantfiles that set 2G of RAM.

Comment 16

[(no longer active)]

Thanks, Rail!

I finally managed to build 660e62353088f75bc2950236de9dc11aec9312b54aa57d2337f817f5e276daae7073e868085302baab36eb32950fcd2d1b060a89a80bd7da76bbc48c5a7294e4, but that resulted in:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-15b988cc3b2f/try-linux64-st-an-debug/try-linux64-st-an-debug-bm75-try1-build4023.txt.gz

which I think is the same issue that I've been hitting on OSX which I can't explain yet.

Comment 17

[Mike Hommey [:glandium]]

Comment on attachment 8551377 [details] [diff] [review]
Update the clang build script for the most recent clang

Please attach an updated patch wrt comment 6 and possibly other changes you might have done.

As for comment 16, it would be good to file a llvm bug.

13:01:33     INFO -  clang: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
13:01:33     INFO -  clang: note: diagnostic msg:
13:01:33     INFO -  ********************
13:01:33     INFO -  PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
13:01:33     INFO -  Preprocessed source(s) and associated run script(s) are located at:
13:01:33     INFO -  clang: note: diagnostic msg: /tmp/TestNANTestingExprC-1fd4ef.c
13:01:33     INFO -  clang: note: diagnostic msg: /tmp/TestNANTestingExprC-1fd4ef.sh

It would be good to be able to get those files.

Comment 18

[(no longer active)]

(In reply to Mike Hommey [:glandium] from comment #17)
> Comment on attachment 8551377 [details] [diff] [review]
> Update the clang build script for the most recent clang
> 
> Please attach an updated patch wrt comment 6 and possibly other changes you
> might have done.

Will do.

> As for comment 16, it would be good to file a llvm bug.
> 
> 13:01:33     INFO -  clang: note: diagnostic msg: PLEASE submit a bug report
> to http://llvm.org/bugs/ and include the crash backtrace, preprocessed
> source, and associated run script.
> 13:01:33     INFO -  clang: note: diagnostic msg:
> 13:01:33     INFO -  ********************
> 13:01:33     INFO -  PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
> 13:01:33     INFO -  Preprocessed source(s) and associated run script(s) are
> located at:
> 13:01:33     INFO -  clang: note: diagnostic msg:
> /tmp/TestNANTestingExprC-1fd4ef.c
> 13:01:33     INFO -  clang: note: diagnostic msg:
> /tmp/TestNANTestingExprC-1fd4ef.sh
> 
> It would be good to be able to get those files.

See <http://comments.gmane.org/gmane.comp.compilers.clang.devel/40448>.  The issue is that we pick up some code from their obj files and link it into the clang plugin binary or some such which causes the crashes somehow.

Comment 19

[(no longer active)]

Attachment: Update the clang build script for the most recent clang; r=rail

Comment 20

[Mike Hommey [:glandium]]

(In reply to :Ehsan Akhgari [Away: 1/29-2/20] (not reading bugmail, needinfo? me!) from comment #18)
> See <http://comments.gmane.org/gmane.comp.compilers.clang.devel/40448>.  The
> issue is that we pick up some code from their obj files and link it into the
> clang plugin binary or some such which causes the crashes somehow.

That's not the same error as the one I'm quoting at all.

Comment 21

[(no longer active)]

(In reply to Mike Hommey [:glandium] from comment #20)
> (In reply to :Ehsan Akhgari [Away: 1/29-2/20] (not reading bugmail,
> needinfo? me!) from comment #18)
> > See <http://comments.gmane.org/gmane.comp.compilers.clang.devel/40448>.  The
> > issue is that we pick up some code from their obj files and link it into the
> > clang plugin binary or some such which causes the crashes somehow.
> 
> That's not the same error as the one I'm quoting at all.

It is, see the crash stack reported earlier in the log.  That is the exact same stack I get on OSX, which is why I started that thread.

Comment 22

[(no longer active)]

Bug 1180549 should fix the crash that I was talking about...

Comment 23

[(no longer active)]

So the good news is that with bug 1180549 being fixed, the new compiler works on OSX: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e6ab7c65ecff

But it doesn't work on Linux yet: https://treeherder.mozilla.org/#/jobs?repo=try&revision=cd5166f8f50c

See http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-cd5166f8f50c/try-linux64-st-an-debug/try-linux64-st-an-debug-bm76-try1-build525.txt.gz:
07:07:24     INFO -  /builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox/_virtualenv/bin/python /builds/slave/try-l64-st-an-d-00000000000000/build/src/config/expandlibs_exec.py --uselist --  /builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/bin/clang++ -Qunused-arguments  -I/builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/include  -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -fomit-frame-pointer -std=c++11 -fno-exceptions -fno-rtti -fPIC -ffunction-sections -fdata-sections -Wcast-qual -fno-rtti -fno-exceptions -fno-omit-frame-pointer  -fPIC -shared -Wl,-h,libclang-plugin.so -o libclang-plugin.so  clang-plugin.o   -lz -lpthread -lrt -ldl -lm -L/builds/slave/try-l64-st-an-d-00000000000000/build/src/clang/lib -lLLVMOption -lLLVMBitReader -lLLVMMCParser -lLLVMAsmParser -lLLVMAnalysis -lLLVMMC -lLLVMCore -lLLVMSupport -lclangASTMatchers  -Wl,-rpath-link,/builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox/dist/bin -Wl,-rpath-link,/usr/local/lib  ../../build/unix/stdc++compat/libstdc++compat.a     -ldl
07:07:24     INFO -  ../../../clang/bin/clang++: /lib64/libz.so.1: no version information available (required by ../../../clang/bin/clang++)
07:07:24     INFO -  TEST-UNEXPECTED-FAIL | check_stdcxx | We do not want these libstdc++ symbols to be used:
07:07:24     INFO -  0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.11 _ZSt15system_categoryv
07:07:24     INFO -  0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.11 _ZSt16generic_categoryv
07:07:24     INFO -  gmake[5]: *** [libclang-plugin.so] Error 1
07:07:24     INFO -  gmake[5]: *** Deleting file `libclang-plugin.so'
07:07:24     INFO -  gmake[5]: Leaving directory `/builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox/build/clang-plugin'
07:07:24     INFO -  gmake[4]: *** [build/clang-plugin/target] Error 2
07:07:24     INFO -  gmake[4]: Leaving directory `/builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox'
07:07:24     INFO -  gmake[3]: *** [export] Error 2
07:07:24     INFO -  gmake[3]: Leaving directory `/builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox'
07:07:24     INFO -  gmake[2]: *** [default] Error 2
07:07:24     INFO -  gmake[2]: Leaving directory `/builds/slave/try-l64-st-an-d-00000000000000/build/src/obj-firefox'
07:07:24     INFO -  gmake[1]: *** [realbuild] Error 2
07:07:24     INFO -  gmake[1]: Leaving directory `/builds/slave/try-l64-st-an-d-00000000000000/build/src'
07:07:24     INFO -  gmake: *** [build] Error 2

Also, during the stage2 build, I was getting quite a few warnings such as:

clang: warning: argument unused during compilation: '-static-libgcc'
clang: warning: argument unused during compilation: '-static-libstdc++'

It seems like clang doesn't support these flags any more.  What else should I be using instead?

Comment 24

[(no longer active)]

Attachment: Part 1: Update the clang build script for the most recent clang

This build script can create a compiler that passes try on OSX, but I'm running into libstdc++ versioning hell with this on Linux.  Rail is going to try running this script on an actual builder to see if it would work there...

Comment 25

[(no longer active)]

Mike, since the current clang that I have managed to build so far on Linux works for static analysis if we turn off the CHECK_STDCXX checks (https://treeherder.mozilla.org/#/jobs?repo=try&revision=6109f0cdddcb) and is blocking some work that we're planning to do on the clang plugin, would you be OK with turning those checks off for static analysis builds while we figure out how to fix things properly?

Comment 26

[Mike Hommey [:glandium]]

That would be okayish, but certainly not the way you did it.

I'm actually surprised disabling the test works. Does clang come with its own copy of a more recent libstdc++ or is the build environment not centos 6 for those builds?

Comment 27

[Mike Hommey [:glandium]]

(In reply to Mike Hommey [:glandium] from comment #26)
> Does clang come with its own copy of a more recent libstdc++

It seems not to.

> or is the build environment not centos 6 for those builds?

It seems it is.

So I'm really surprised this actually works. I'm inclined to think it doesn't, but the fact is hidden somehow. Please push another try that's supposed to turn red because of static analysis. *that* would be evidence that it work.

Comment 28

[Mike Hommey [:glandium]]

Oh, centos6 has libstdc++6 4.4 which does have the 3.4.11 symbols.

Comment 29

[Mike Hommey [:glandium]]

Add the following at the end of build/clang-plugin.Makefile.in:

 CHECK_STDCXX = $(call CHECK_SYMBOLS,$(1),GLIBCXX,libstdc++,v[1] > 3 || (v[1] == 3 && v[2] == 4 && v[3] > 11))

Comment 30

[(no longer active)]

(In reply to Mike Hommey [:glandium] from comment #29)
> Add the following at the end of build/clang-plugin.Makefile.in:
> 
>  CHECK_STDCXX = $(call CHECK_SYMBOLS,$(1),GLIBCXX,libstdc++,v[1] > 3 ||
> (v[1] == 3 && v[2] == 4 && v[3] > 11))

That works! <https://treeherder.mozilla.org/#/jobs?repo=try&revision=469658776183>

Here is the proof that the static analysis is actually running (I used new to create a MOZ_STACK_CLASS object on the heap): <https://treeherder.mozilla.org/#/jobs?repo=try&revision=1d846bface4c>.

Comment 31

[(no longer active)]

Attachment: Part 2: Temporarily relax the libstdc++ symbol version requirements for Linux static analysis builds

Comment 32

[(no longer active)]

Attachment: Part 3: Update the tooltool manifests for the OSX and Linux static analysis builds in order to upgrade clang

Comment 33

[Rail Aliiev [:rail]]

Comment on attachment 8632114 [details] [diff] [review]
Part 3: Update the tooltool manifests for the OSX and Linux static analysis builds in order to upgrade clang

stamp

Comment 34

[Rail Aliiev [:rail]]

Comment on attachment 8631365 [details] [diff] [review]
Part 1: Update the clang build script for the most recent clang

lgtm

Comment 35

[Mike Hommey [:glandium]]

Comment on attachment 8632112 [details] [diff] [review]
Part 2: Temporarily relax the libstdc++ symbol version requirements for Linux static analysis builds

Review of attachment 8632112 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/clang-plugin/Makefile.in
@@ +19,5 @@
>  # to be distributed.
>  MACOSX_DEPLOYMENT_TARGET :=
> +
> +# Temporarily relax the requirements for libstdc++ symbol versions on static
> +# analysis builds in order to use a recent clang by accepting libstdc++ from

s/builds/plugin/

@@ +22,5 @@
> +# Temporarily relax the requirements for libstdc++ symbol versions on static
> +# analysis builds in order to use a recent clang by accepting libstdc++ from
> +# gcc 4.4.0 (GLIBCXX_3.4.11).
> +ifneq (,$(MOZ_LIBSTDCXX_TARGET_VERSION)$(MOZ_LIBSTDCXX_HOST_VERSION))
> +ifneq ($(OS_ARCH),Darwin)

You can replace both conditions with
ifdef CHECK_STDCXX

Comment 36

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/40798fb2a58f
https://hg.mozilla.org/integration/mozilla-inbound/rev/5bd2edcc7779
https://hg.mozilla.org/integration/mozilla-inbound/rev/07ee075d8e6c

Comment 37

[(no longer active)]

Backed out the last part: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=07ee075d8e6c

Rail, do you know what these errors mean?

https://hg.mozilla.org/integration/mozilla-inbound/rev/206b12521652

Comment 38

[Mike Hommey [:glandium]]

Error is "ERROR - failed to load manifest file at '/builds/slave/m-in-m64-st-an-d-0000000000000/build/src/browser/config/tooltool-manifests/macosx64/clang.manifest': trying to read invalid manifest file"

See https://hg.mozilla.org/integration/mozilla-inbound/rev/07ee075d8e6c#l3.12 and guess what's invalid ;)

Comment 39

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8bbd02e838c9
https://hg.mozilla.org/integration/mozilla-inbound/rev/aa25eb834b5d

Comment 40

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/971d48d37cc5

Comment 41

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/8bbd02e838c9
https://hg.mozilla.org/mozilla-central/rev/aa25eb834b5d
https://hg.mozilla.org/mozilla-central/rev/971d48d37cc5