Closed Bug 1253299 Opened 8 years ago Closed 8 years ago

TC Linux 64 ASAN debug and opt builds Tier 1

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: selenamarie, Assigned: kmoir)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file, 3 obsolete files)

bug1272629new-2.patch
10.67 KB, patch
mtabara
: review+
mtabara
: feedback+
mtabara
: feedback+
Details | Diff | Splinter Review 
This is a planning bug for tracking work related to getting Linux 64 ASAN debug and opt builds as Tier-1.
https://reviewboard.mozilla.org/r/38719/#review35397

This is just a WIP for now.

Right now this fails to build on both opt and debug because clang can't seem to link things properly:

https://tools.taskcluster.net/task-inspector/#ecmw6igNTIC-J-zz36y67A/0
catlee's patch was out of date because of _clobber changes. This just adjusts for that. I get the same clang failure:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=76405d61a70e

19:34:18     INFO -  DEBUG: configure:3123: checking whether the C compiler (/home/worker/workspace/build/src/clang/bin/clang -fgnu89-inline -fsanitize=address -Dxmalloc=myxmalloc -fPIC -L/home/worker/workspace/build/src/gtk3/usr/local/lib -fsanitize=address) works
19:34:18     INFO -  DEBUG: configure:3139: /home/worker/workspace/build/src/clang/bin/clang -fgnu89-inline -o conftest -fsanitize=address -Dxmalloc=myxmalloc -fPIC  -L/home/worker/workspace/build/src/gtk3/usr/local/lib -fsanitize=address conftest.c  1>&5
19:34:18     INFO -  DEBUG: configure:3136:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
19:34:18     INFO -  DEBUG: main(){return(0);}
19:34:18     INFO -  DEBUG: ^~~~
19:34:18     INFO -  DEBUG: 1 warning generated.
19:34:18     INFO -  DEBUG: /usr/bin/ld: crtbegin.o: No such file: No such file or directory
19:34:18     INFO -  DEBUG: clang: error: linker command failed with exit code 1 (use -v to see invocation)
19:34:18     INFO -  DEBUG: configure: failed program was:
19:34:18     INFO -  DEBUG:
19:34:18     INFO -  DEBUG: #line 3134 "configure"
19:34:18     INFO -  DEBUG: #include "confdefs.h"
19:34:18     INFO -  DEBUG:
19:34:18     INFO -  DEBUG: main(){return(0);}
19:34:18     INFO -  DEBUG: configure: error: installation or configuration problem: C compiler cannot create executables.
19:34:18     INFO -  ERROR: old-configure failed
19:34:18     INFO -  *** Fix above errors and then restart with\
19:34:18     INFO -                 "/usr/bin/gmake -f client.mk build"
19:34:18     INFO -  gmake[2]: *** [configure] Error 1


/home/worker/workspace/build/src/clang/bin/clang appears to come from tooltool, same as on buildbot, and clang command line options appear to be the same as buildbot.
odd, maybe "/usr/bin/ld: crtbegin.o: No such file: No such file or directory", which would imply that we didn't get output from the compiler and ld is obviously failing.

maybe this is a permissions issue?  maybe a disk space issue?  I suspect downloading the image.tar and running it locally would yield a quick answer!  

Thanks for working on this bug.
Assignee: nobody → gbrown
See Also: → 1171841
crtbegin.o is in the image, at:

  /usr/lib/gcc/x86_64-redhat-linux/4.4.4/32/crtbegin.o
  /usr/lib/gcc/x86_64-redhat-linux/4.4.4/crtbegin.o

but clang doesn't seem to know that gcc is installed there:

23:24:51     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang --print-search-dirs
23:24:51     INFO -  programs: =/home/worker/workspace/build/src/clang/bin:/..//bin
23:24:51     INFO -  libraries: =/home/worker/workspace/build/src/clang/bin/../lib/clang/3.5:/lib/../lib64:/usr/lib/../lib64:/home/worker/workspace/build/src/clang/bin/../lib:/lib:/usr/lib
23:24:51     INFO - Return code: 0

23:24:51     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang --print-libgcc-file-name
23:24:51     INFO -  libgcc.a
23:24:51     INFO - Return code: 0

23:24:51     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang -v
23:24:51     INFO -  clang version 3.5 (trunk 200213)
23:24:51     INFO -  Target: x86_64-unknown-linux-gnu
23:24:51     INFO -  Thread model: posix
23:24:51     INFO -  Selected GCC installation:
23:24:51     INFO - Return code: 0


Happily, I notice that clang is already used in a tc job -- static analysis. One difference is it uses a different clang manifest:

'tooltool_manifest_src': 'browser/config/tooltool-manifests/linux64/clang.manifest.centos6'

instead of:

'tooltool_manifest_src': 'browser/config/tooltool-manifests/linux64/asan.manifest'
See Also: → 1203390
Simply changing to the clang.manifest.centos6 allows clang to find gcc, and most of the build completes.

16:45:22     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang --print-search-dirs
16:45:22     INFO -  programs: =/home/worker/workspace/build/src/clang/bin:/home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3/../../../../x86_64-unknown-linux-gnu/bin
16:45:22     INFO -  libraries: =/home/worker/workspace/build/src/clang/bin/../lib/clang/3.8.0:/home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3:/lib/../lib64:/usr/lib/../lib64:/home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3/../../..:/home/worker/workspace/build/src/clang/bin/../lib:/lib:/usr/lib
16:45:22     INFO - Return code: 0

16:45:22     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang --print-libgcc-file-name
16:45:22     INFO -  /home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3/libgcc.a
16:45:22     INFO - Return code: 0

16:45:22     INFO - Copy/paste: /home/worker/workspace/build/src/clang/bin/clang -v
16:45:22     INFO -  clang version 3.8.0 (trunk) (llvm/trunk 247539)
16:45:22     INFO -  Target: x86_64-unknown-linux-gnu
16:45:22     INFO -  Thread model: posix
16:45:22     INFO -  InstalledDir: /home/worker/workspace/build/src/clang/bin
16:45:22     INFO -  Found candidate GCC installation: /home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3
16:45:22     INFO -  Found candidate GCC installation: /usr/lib/gcc/i686-redhat-linux/4.4.4
16:45:22     INFO -  Found candidate GCC installation: /usr/lib/gcc/i686-redhat-linux/4.4.7
16:45:22     INFO -  Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.4.4
16:45:22     INFO -  Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.4.7
16:45:22     INFO -  Selected GCC installation: /home/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.7.3
16:45:22     INFO -  Candidate multilib: .;@m64
16:45:22     INFO -  Candidate multilib: 32;@m32
16:45:22     INFO -  Selected multilib: .;@m64
16:45:22     INFO - Return code: 0

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1c1d60b0cce2

17:12:26     INFO -  /home/worker/workspace/build/src/clang/bin/clang -fgnu89-inline -o /home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign -O2 -gdwarf-2 -fPIC -DLINUX2_1 -m64 -pipe -ffunction-sections -fdata-sections -DLINUX -Dlinux -DHAVE_STRERROR -Wall -DNSS_NO_GCC48 -DXP_UNIX -DSHLIB_SUFFIX=\"so\" -DSHLIB_PREFIX=\"lib\" -UDEBUG -DNDEBUG -D_REENTRANT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -I/home/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/home/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/home/worker/workspace/build/src/obj-firefox/dist/include/nss -I/home/worker/workspace/build/src/obj-firefox/dist/private/nss -Qunused-arguments -Qunused-arguments -fsanitize=address -Dxmalloc=myxmalloc -fPIC -std=gnu99 -fgnu89-inline -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -g -O2 -gline-tables-only -fno-omit-frame-pointer /home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign.o -m64  -L/home/worker/workspace/build/src/obj-firefox/dist/bin -lplc4 -lplds4 -lnspr4  -lpthread  -ldl -lc
17:12:26     INFO -  /usr/bin/ld: /home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign: undefined reference to symbol '__cxa_demangle@@CXXABI_1.3'
17:12:26     INFO -  /usr/bin/ld: note: '__cxa_demangle@@CXXABI_1.3' is defined in DSO /usr/lib64/libstdc++.so.6 so try adding it to the linker command line
17:12:26     INFO -  /usr/lib64/libstdc++.so.6: could not read symbols: Invalid operation
17:12:26     INFO -  clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
17:12:26     INFO -  gmake[6]: *** [/home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign] Error 1
I don't understand the cause of the link error in Comment 6:

/home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign: undefined reference to symbol '__cxa_demangle@@CXXABI_1.3'

The link error goes away if I suppress asan (remove -fsanitize=address and --enable-address-sanitizer):

https://treeherder.mozilla.org/#/jobs?repo=try&revision=358066c393bd

but that's not a helpful work-around!

:ehsan -- I'm trying to use clang from browser/config/tooltool-manifests/linux64/clang.manifest.centos6; is that reasonable? Do you have insight into the link error / how to avoid it?
Flags: needinfo?(ehsan)
Blocks: 1253301
(In reply to Geoff Brown [:gbrown] from comment #7)
> I don't understand the cause of the link error in Comment 6:
> 
> /home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/
> shlibsign: undefined reference to symbol '__cxa_demangle@@CXXABI_1.3'
> 
> The link error goes away if I suppress asan (remove -fsanitize=address and
> --enable-address-sanitizer):
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=358066c393bd
> 
> but that's not a helpful work-around!
> 
> :ehsan -- I'm trying to use clang from
> browser/config/tooltool-manifests/linux64/clang.manifest.centos6; is that
> reasonable? Do you have insight into the link error / how to avoid it?

Can you please retry on top of bug 1262735?  What version of clang are you using?

To answer your question: I think this error means that you don't have the right libc++abi version installed.
Flags: needinfo?(ehsan)
(In reply to :Ehsan Akhgari (busy, don't ask for review please) from comment #8)
> Can you please retry on top of bug 1262735?  What version of clang are you using?

Good idea. Thanks Ehsan.

But I still get the same failure on top of bug 1262735 / bug 1262781, using browser/config/tooltool-manifests/linux64/clang.manifest.centos6 to pull clang from tooltool with version tag "clang 3.8.0, libgcc 4.8.5".

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f936f328e9aa

21:18:56     INFO -  /usr/bin/ld: /home/worker/workspace/build/src/obj-firefox/security/nss/cmd/shlibsign/shlibsign: undefined reference to symbol '__cxa_demangle@@CXXABI_1.3'

        Attachment #8727972 -
        Attachment is obsolete: true

        Attachment #8738307 -
        Attachment is obsolete: true
Assignee: gbrown → nobody
Assignee: nobody → kmoir

    
    

    
  
Ran a try job this morning and it failed because the decision graph couldn't be generated, 
investigating
https://treeherder.mozilla.org/#/jobs?repo=try&revision=024ce62ee9848a73cd9c1b29fc9f6fdf80e3cbcc

    
    

    
  
so this submission is better
https://treeherder.mozilla.org/#/jobs?repo=try&revision=07e5007647c4

I omitted committing new files in the patch in the earlier run

    
    

    
  
Same error on that try run as last.  Looking at the manifests, the one we used for the try run is 
[
{
"version": "clang 3.8.0, libgcc 4.8.5",
"size": 118876936,
"digest": "f021d7b23cbbcc4086514b4bf20c86c9f94dbe5b4e0f1ef3aaf2bea337430f7c0d0965c55fe8be0e944a46c3b1555b9245c7af8b8b141eac4b47deea977b4852",
"algorithm": "sha512",
"filename": "clang.tar.xz",
"unpack": true
},
{
"size": 12072532,
"digest": "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",
"algorithm": "sha512",
"filename": "gtk3.tar.xz",
"setup": "setup.sh",
"unpack": true
}
]

old one for bb builds in tree is

[
{
"version": "clang 3.5/r200213",
"size": 71282740,
"digest": "ee9edb1ef3afd9ab29e39565145545ad57e8d8d2538be4d822d7dbd64038f4529b0b287cecf48bf83def52a26ac2c6faa331686c3ad5e8b4ba4c22686ee0808f",
"algorithm": "sha512",
"filename": "clang.tar.bz2",
"unpack": true
},
{
"size": 12072532,
"digest": "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",
"algorithm": "sha512",
"filename": "gtk3.tar.xz",
"setup": "setup.sh",
"unpack": true
}
]

I'm comaparing the two tooltool clang binaries and seeing if there are any glaring differences, if not, I'll ask for help from a build person

    
    

    
  
Note that you can also look up information about the tooltool uploads (comment, who uploaded, datestamp) in the relengapi UI
Blocks: LSan
See Also:  → 1267650

    
    

    
  
Looking at the patch attached to this bug, it seems to me that you are planning to remove the manifest for ASan and instead use the regular Clang manifest that we use for our release builds, is that correct?

If that is the case, please keep the separate manifest. We introduced it for a particular reason and that is that upgrading the ASan Clang version was nearly impossible when it was still tied to the regular Clang version. If someone commits to keeping Clang up-to-date in general, that's a different story though.

I also filed bug 1267650 to upgrade the ASan Clang version to trunk (what will be 3.9). I made a local build and it works fine with some fix I already landed. It would be nice if we could get that version for ASan on TaskCluster, it would greatly help the fuzzing team. If I can help solving problems, let me know.

    
    

    
  
I'm trying to build a new version of clang as described in bug 1267650 comment 4

https://treeherder.mozilla.org/#/jobs?repo=try&revision=972837dd076b

Once this is ready I will try another asan job with it on try (and keep the separate manifest as described in comment 16)

    
    

    
  
(In reply to Kim Moir [:kmoir] from comment #17)
> I'm trying to build a new version of clang as described in bug 1267650
> comment 4
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=972837dd076b
> 
> Once this is ready I will try another asan job with it on try (and keep the
> separate manifest as described in comment 16)

Since you didn't change anything in the tree, all you'll get there is the same clang as the one we already use (3.8).

    
    

    
  
So looking at a bug where we upgraded clang to 3.8.0 (bug 1262781) - we pulled from  (https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_380/).  However, 3.9 is not released yet.  So should just try trunk and hope for the best or is there a specific revision that we could use similar to the 3.8 bug?

https://reviewboard.mozilla.org/r/45685/diff/1#8
Flags: needinfo?(mh+mozilla)

    
    

    
  
If you really want a 3.9 development snapshot, pull from trunk with a specific revision and document the revision in the tooltool manifest (check how it was before bug 1262781).

That said, it would be better if we stopped using non-release versions of the toolchains. What's wrong with 3.8?
Flags: needinfo?(mh+mozilla)

    
    

    
  
(In reply to Mike Hommey [:glandium] from comment #20)
> If you really want a 3.9 development snapshot, pull from trunk with a
> specific revision and document the revision in the tooltool manifest (check
> how it was before bug 1262781).

But I still need to provide the necessary build tarball, right? I know how the tooltool manifests work but when I did the last Clang update on our infra, we still had to bug releng to upload a new tarball for us. Has that changed?

> 
> That said, it would be better if we stopped using non-release versions of
> the toolchains. What's wrong with 3.8?

ASan constantly gets improved and we want to have the newest ASan support available. Since 3.8, numerous fixes (mainly affecting Windows) have been made to compiler-rt. For Linux, 3.8 might actually suffice right now, but our experience is that we have to make fixes (to Clang) on such major upgrades and these fixes will end up on trunk only. We can try it with 3.8 first and then upgrade to trunk or 3.9 (if released), when we need it on Windows.

    
    

    
  
(In reply to Christian Holler (:decoder) from comment #21)
> (In reply to Mike Hommey [:glandium] from comment #20)
> > If you really want a 3.9 development snapshot, pull from trunk with a
> > specific revision and document the revision in the tooltool manifest (check
> > how it was before bug 1262781).
> 
> But I still need to provide the necessary build tarball, right? I know how
> the tooltool manifests work but when I did the last Clang update on our
> infra, we still had to bug releng to upload a new tarball for us. Has that
> changed?

You can ask for permission to upload to tooltool if necessary. And since bug 1262729, you can build linux toolchains by pushing to try.

> > That said, it would be better if we stopped using non-release versions of
> > the toolchains. What's wrong with 3.8?
> 
> ASan constantly gets improved and we want to have the newest ASan support
> available. Since 3.8, numerous fixes (mainly affecting Windows) have been
> made to compiler-rt. For Linux, 3.8 might actually suffice right now, but
> our experience is that we have to make fixes (to Clang) on such major
> upgrades and these fixes will end up on trunk only. We can try it with 3.8
> first and then upgrade to trunk or 3.9 (if released), when we need it on
> Windows.

Windows is using a snapshot of 3.9 already.

    
    

    
  
okay, now the issues with finding clang are fixed but the build itself still fails with compile errors
https://treeherder.mozilla.org/#/jobs?repo=try&revision=15debfd306fd1045c668cbcda821fd343873a789

(ignore the fact that the try run built clang that is just because of my patch queue)

taskcluster logs
https://public-artifacts.taskcluster.net/VITKbCVbT5O_ZmRPPO5Caw/0/public/logs/live_backing.log

This is clang revision 262971 (toward 3.9)

Not sure if/what patches are needed to make it work

    
    

    
  
:decoder do you have any insight on the source of the compile errors for this build? Not sure if additional patches are required for 3.9.  The existing ones for 3.8 in tree didn't apply anymore.
Flags: needinfo?(choller)

    
    

    
  
Why the f*ck does clang want to use a C++ symbol in a C program?

    
    

    
  
That's a good question. :)

I'm not sure if this is a clue or a red herring, but maybe worth repeating at this point: When I was trying to get this working and getting the same error (comment 7), I found the error went away if I suppressed asan (removed -fsanitize=address and --enable-address-sanitizer).

    
    

    
  
Well, AddressSanitizer itself (and its runtime) is written in C++, so that might explain where the C++ symbol came from, right?

We had issues before with our codebase due to C++ compile errors, but until now, --enable-stdcxx-compat resolved those.

I also vaguely remember that we sometimes had to add C++ functions to a certain compat list (?) in our tree?
Flags: needinfo?(choller)

    
    

    
  
Does anyone know where this compat list in the tree is for the C++ functions?

    
    

    
  
:decoder - Could you identify what needs to be updated in the compat file list mentioned in comment #30?  I don't have any expertise in this area.  I'm just trying to port this build to taskcluster.
Flags: needinfo?(choller)

    
    

    
  
Forwarding this to :glandium since he should know best (and maintains this file, as far as I know).
Flags: needinfo?(choller) → needinfo?(mh+mozilla)

    
    

    
  
Here is the catch: stdcxx-compat is only linked to C++ binaries. So even if adding something there was the right thing to do (which I doubt it would be anyways), it wouldn't change anything for those errors.

Now, what seems to be happening is:
- shlibsign is linked from clang, because it's a pure C program.
- As such, clang calls ld with the standard set of libraries for C programs (-lc. -lm, etc.) and *not* -lstdc++, which is for C++ programs
- because there is also -fsanitize=address, it also adds the asan library, which apparently wants symbols from -lstdc++
- ld sees that there's a symbol from libstdc++ is missing, and considers it
- it then later rejects linking libstdc++ because it was not on the command line

One way out of this is to manually add -lstdc++ to the LDFLAGS, but it seems ridiculous to me that clang is not doing it itself if -fsanitize=address requires it (clang bug?). Also, why is it not happening on the existing asan builds? Is the requirement for __cxa_demangle new?
Flags: needinfo?(mh+mozilla)

    
    

    
  
per irc:

19:54 <glandium> decoder: so, there is no problem with clang, it more or less does the right thing
19:55 <glandium> decoder: that is, whatever uses __cxa_demangle is using it with a weak reference
19:56 <glandium> decoder: what happens is that because of the combination of bug 743988 and bug 1230117, we now link nspr with libstdc++
19:57 <glandium> decoder: when ld links shlibsign, it sees that it's pulling libstdc++ from nspr, sees that __cxa_demangle is there too, so hey, why not use it
19:57 <glandium> decoder: and then, it says, oh, no, in fact, go away, I don't want symbol references to things I'm not depending on directly
19:59 <glandium> decoder: it's likely related to https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking
20:00 <glandium> decoder: and why it happens on automation and not on your build is likely that it's not the same on ubuntu
20:00 <glandium> or whatever distro you use
20:01 <decoder> glandium: im on ubuntu 16.04 (and 14.04 is the same)
20:03 <glandium> decoder: another possible explanation for the discrepancy is that, since it's a weak symbol, ld shouldn't complain anyways, and maybe that part was 
                 fixed in later versions
20:04 <glandium> decoder: so there are two things that we can do to work around, each of which should work independently, but both of which should actually happen for 
                 various reasons:
20:04 <glandium> decoder: - link nspr with $CC instead of $CXX (kind of a backout of bug 743988)
20:05 <glandium> decoder: - include, along clang, the same binutils we have along gcc
20:05 <decoder> the latter sounds a better to me in some ways
20:05 <decoder> who knows what other bugs are in those old binutils
20:05 <glandium> decoder: both should be done
20:05 <decoder> and better performance
20:07 <glandium> decoder: the easiest for you is probably the latter
20:07 <glandium> I guess changing build_clang.py to copy a few files from the gcc package, like it does for libgcc and headers, would be enough
20:08 <glandium> or, we could create a separate package for binutils
20:08 <glandium> actually, I think I prefer the latter

I will file bugs for both items.
Depends on: 1272626
Depends on: 1272629

    
    

    
  


  
new patches with fixes from bug 1272629. I used separate tooltool manifests for the tc and buildbot jobs since they use different versions of clang etc.  

The builds run green, now I will run with tests 
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e262205e76cecc66330b07a3ad2764bc2ab1befb

        Attachment #8741950 -
        Attachment is obsolete: true

    
    

    
  
The try run was green for buildbot tests except for tests that aren't enabled on asan yet anyways 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7b00f51c523f6aae82d08547775e792087f0f421

bug 1253301 tracks getting ASAN tests working on TC is in progress so I didn't add hooks to get them to run

    
    

    
  
Comment on attachment 8757285 [details] [diff] [review]
bug1272629new-2.patch

gbrown: I was going to add a second tool manifest to get the asan builds into production on tc and then deprecate the buildbot one once they are fully deployed on all trees - what are your thoughts?

        Attachment #8757285 -
        Flags: feedback?(gbrown)

    
    

    
  
gbrown is on pto for 2 weeks- possibly there is another person to feedback?

    
    

    
  
thanks joel, will ask Mihai for feedback since he has been working on tc migrations lately
Flags: needinfo?(mtabara)

    
  

        Attachment #8757285 -
        Flags: feedback?(gbrown) → feedback?(mtabara)

    
  
Flags: needinfo?(mtabara)

    
    

    
  
Comment on attachment 8757285 [details] [diff] [review]
bug1272629new-2.patch

Review of attachment 8757285 [details] [diff] [review]:
-----------------------------------------------------------------

Haven't played yet with manifests so not sure how much my feedback helps there. However, testing/taskcluster/... stuff looks good.

        Attachment #8757285 -
        Flags: feedback+

        Attachment #8757285 -
        Flags: feedback?(mtabara) → feedback+

    
  

        Attachment #8757285 -
        Flags: review?(mtabara)

        Attachment #8757285 -
        Flags: review?(mtabara) → review+

    
    

    
  
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/fd184690f95c
TC Linux 64 ASAN debug and opt builds Tier 1 r=mtabara

    
    

    
  
excellent!

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/fd184690f95c
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED

    
    

    
  
Kim, it looks like you accidentally landed an orig file together with your remaining patch:

https://hg.mozilla.org/mozilla-central/diff/fd184690f95c/testing/taskcluster/tasks/branches/base_job_flags.yml.orig
Flags: needinfo?(kmoir)

    
    

    
  
I'll land the fix for this as soon as trees reopen.
Flags: needinfo?(kmoir)

    
    

    
  
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4724bebdf93a
TC Linux 64 ASAN debug and opt builds Tier 1 r=mtabara DONTBUILD

    
  
Product: TaskCluster → Firefox Build System

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: