607946 - Allow specifying a post-upload command for uploadsymbols

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[(not currently active) Ted Mielczarek]

In bug 607831, I'm investigating options for changing the way Socorro works with symbol files. We'll probably wind up converting our symbol files to a binary format in the short-term, and maybe moving them into HBase in the longer-term. Either way, it would be helpful to have a post-upload command that the build slaves could run from uploadsymbols, so that converting incoming symbols would be a simple process, and if we need to do something more complex (like store in HBase), we'll have that flexibility later.

Comment 1

[(not currently active) Ted Mielczarek]

Attachment: Allow specifying a post-upload command for uploadsymbols

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 486606 [details] [diff] [review]
Allow specifying a post-upload command for uploadsymbols

I hate that this is still a shell script, but switching things to use upload.py is probably more work than it's worth at the moment.

Comment 3

[(not currently active) Ted Mielczarek]

Comment on attachment 486606 [details] [diff] [review]
Allow specifying a post-upload command for uploadsymbols

This patch should be a no-op until we fix the bugs it's blocking. This is necessary for some Socorro work we want to do to make processing faster.

Comment 4

[(not currently active) Ted Mielczarek]

Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/ed88ab5f4dbf

Comment 5

[(not currently active) Ted Mielczarek]

Backed out, hg rebase made a mess of my changeset. Will re-land shortly.

Comment 6

[(not currently active) Ted Mielczarek]

Pushed to m-c again:
http://hg.mozilla.org/mozilla-central/rev/39a979e26931

Comment 7

[(not currently active) Ted Mielczarek]

I filed a bug upstream with hg on the rebase fail, FWIW:
http://mercurial.selenic.com/bts/issue2471

Comment 8

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Ted, when you request branch approvals for this, don't forget to ask for 1.9.0.next so that we can get Camino 2.0.x builds using the new system. ;)

I'll be happy to do the actual checking-in on 1.9.0 for you, though, since I still have a CVS tree (the patch doesn't apply cleanly, but it's a trivial merge, and presumably it would not hurt to update upload-symbols.sh entirely).

Comment 9

[(not currently active) Ted Mielczarek]

Do we actually still do approvals for 1.9.0.x? I thought that branch was effectively dead. (Who's managing the approvals?)

Comment 10

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #9)
> Do we actually still do approvals for 1.9.0.x?

Yes, with varying degrees of lag.

> (Who's managing the approvals?)

The same branch-drivers as 1.9.x.  Typically when approval1.9.0.next gets asked for at the same time as approval1.9.1.n/1.9.2.n, it's granted with the others; otherwise, I have to poke and poke.

Comment 11

[(not currently active) Ted Mielczarek]

Comment on attachment 486606 [details] [diff] [review]
Allow specifying a post-upload command for uploadsymbols

Requesting branch approvals. I'd like to land this on active branches in order to facilitate the Socorro work, so that we can do the symbol conversion server-side. This also gives us more flexibility in case we need to change something about symbol uploading in the future, we can simply change the script server-side.

Comment 12

[Daniel Veditz [:dveditz]]

Comment on attachment 486606 [details] [diff] [review]
Allow specifying a post-upload command for uploadsymbols

Approved for 1.9.2.13 and 1.9.1.16, a=dveditz for release-drivers

Does this really apply correctly to the 1.9.0 branch? Seems like there might have been changes since then. Approved for 1.9.0.20, a=dveditz for release-drivers	if so.

Comment 13

[(not currently active) Ted Mielczarek]

Maybe not, but I'd expect it to be a trivial merge if it doesn't. This code hasn't had a lot of churn.

Comment 14

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Yeah, it's trivial; I checked earlier and it's just context (m-c and 1.9.2 have two attempts to fix quoting/MOZ_PKG_PRETTYNAMES and two attempts to do collision hashing that 1.9.0 doesn't have, and 1.9.1 is also lacking all but the very first attempt to fix quoting/MOZ_PKG_PRETTYNAMES).

Comment 15

[Reed Loden [:reed]]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/b4bab4ea6184

Comment 16

[Reed Loden [:reed]]

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/179b608dc745

Comment 17

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Checking in Makefile.in;
/cvsroot/mozilla/Makefile.in,v  <--  Makefile.in
new revision: 1.405; previous revision: 1.404
done
Checking in toolkit/crashreporter/tools/upload_symbols.sh;
/cvsroot/mozilla/toolkit/crashreporter/tools/upload_symbols.sh,v  <--  upload_symbols.sh
new revision: 1.11; previous revision: 1.10
done

Comment 18

[(not currently active) Ted Mielczarek]

Thanks for the branch landings, I was travelling and busy with other things.