945879 - Produce crash report symbols for darwin-on-linux builds

Status: RESOLVED INCOMPLETE

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Hal Wine [:hwine] use NI!]

The "buildsymbols" step does not currently run on linux for at least two reasons:
 - the current process uses mac binaries that do not run under ma-loader
 - the breakpad tools are not set up for cross compilation (bug 543111)

For initial deploy, we'll transfer the data to a mac for these steps.

Comment 1

[(not currently active) Ted Mielczarek]

Specifically, we'll likely need to do the following:
1) Copy the entire object directory (or at least every .o file and every binary that gets linked) from the builder to a Mac machine, using the exact same directory structure.
2) Run the equivalent of "make buildsymbols" on the Mac, but pointing at a (pre-built?) dump_syms binary.
3) Run the equivalent of "make uploadsymbols" on the Mac to put the symbols on the symbol server.
4) Additionally upload the symbol package next to the app package on FTP.

Comment 2

[(not currently active) Ted Mielczarek]

This is to support the "build on linux, dump symbols on mac" case, which is what I thought this bug was about, but maybe I'm wrong?

Comment 3

[Hal Wine [:hwine] use NI!]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> This is to support the "build on linux, dump symbols on mac" case, which is
> what I thought this bug was about, but maybe I'm wrong?

You are correct -- thanks for the detailed steps in comment 1

Comment 4

[Nathan Froyd [:froydnj]]

Attachment: dump_syms modifications for *compiling* (doesn't link)

This is my WIP patch to make dump_syms compile under Linux.  There's a few symbols that would need to be fixed/added to make the whole thing link.

As you can see, it involves some dodgy modifications to things.  You also need a /usr/include from a Mac for some mach-o headers and the like.  I'm not planning on working on this further unless we decide that we really do want to do as much stuff on Linux as possible.

Comment 5

[Chris Cooper [:coop] (he/him)]

This doesn't appear to be going anywhere. :(

Comment 6

[(not currently active) Ted Mielczarek]

Did we decide that the work in comment 1 was too unwieldy? In the interim I also ran into a description of how dsymutil works:
http://stackoverflow.com/questions/10044697/where-how-does-apples-gcc-store-dwarf-inside-an-executable/12827463#12827463

Comment 7

[Nathan Froyd [:froydnj]]

I never got a sense of whether we wanted to copy objdirs all over to Linux, or whether we wanted to do all the processing on the Linux side...

Comment 8

[Hal Wine [:hwine] use NI!]

Summarizing as I understand it. Both prior approaches (fix linux side processing vs copy all obj to a mac) take significant implementation time. Factoring that cost in, it appears less expensive to continue to operate mac hardware builders.

If anything changes to make one approach or the other "easy peesy", we would revisit.

Comment 9

[(not currently active) Ted Mielczarek]

bug 543111 fixed this the right way, by making `buildsymbols` work fine on a Linux host.