607951 - Add a post-uploadsymbols command

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[(not currently active) Ted Mielczarek]

The patch in bug 607946 allows specifying POST_SYMBOL_UPLOAD_CMD, a command that will be run on the symbol server after symbols are uploaded. I'd like to deploy a script and get our buildbot configs using it so that I have the flexibility to change our symbol format or storage in the near future (bug 607831).

Comment 1

[(not currently active) Ted Mielczarek]

Chris and Wolfgang, since you're not using the uploadsymbols target to upload your symbols, once I fix this bug you'll have to modify your symbol upload process to call the same script after uploading. That way your symbols will continue to work when we modify the script to convert the symbols to a new format.

Comment 2

[Wolfgang Rosenauer [:wolfiR]]

Looks like an easy change here. Could you please post in this bug the command we need to use once it's available/implemented?

Comment 3

[(not currently active) Ted Mielczarek]

Attachment: Add post-symbol-upload command to environment

This is not quite the right patch, since I don't know where the post-upload script will live yet, but I think it's mostly right. I probably need to copy this logic to all the other config.pys that live in this repo as well, I'm sure.

Comment 4

[(not currently active) Ted Mielczarek]

Attachment: No-op post-symbol-upload.py

This is a placeholder, the functional version will be added in bug 607961. This is just to get something in place so that making it functional requires changes only on the symbol server. (Actually making it do conversion requires some logistical changes.)

Comment 5

[(not currently active) Ted Mielczarek]

CCing some Camino folks, since I know their symbol upload process isn't 100% the same as what's in mozilla-central. It does look like Camino uses upload_symbols.sh, which is good, but I'm not sure that it will all work out of the box. You might need to port some bits of bug 607946, and you will need the environment var set properly in your build environment.

Comment 6

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #5)
> the box. You might need to port some bits of bug 607946

Bug 607946 looks to me like it's only doing 2 things:

1) 'export'ing the symbols index filename, which we'd have to add to our Makefile

2) making some changes to upload_symbols.sh to send along the new environment variable, which we should get for free--although for 2.0.x it'll need to be landed in CVS ;)

There's not more I'm missing, right?

> environment var set properly in your build environment.

OK.  That's '/path/to/post-symbol-upload.py' from the patch on this bug (which eventually will have some sort of on-server path, like SYMBOL_SERVER_PATH does)?

Assuming this is all that's needed, I'll also have to make corresponding changes to the Mac OS X symbols upload architecture, right?

Comment 7

[(not currently active) Ted Mielczarek]

Ok, sounds like you don't need any code changes then, just the environment change. The actual path to use will wind up in a patch on this bug soon.

Good point: re OS X symbols. I'm planning a mass conversion of existing symbols on the symbol server, but for new uploads you'll have to convert them. Let me know (or file a new bug) if you need any help with this. We're probably going to want a process in place for when we upload things like Flash symbols as well.

Comment 8

[(not currently active) Ted Mielczarek]

Comment on attachment 486625 [details] [diff] [review]
No-op post-symbol-upload.py

Pushed to build/tools:
http://hg.mozilla.org/build/tools/rev/483307ade8a1

Comment 9

[(not currently active) Ted Mielczarek]

Attachment: Add post-symbol-upload command to environment

This patch includes the correct path for the post-upload command. I grepped all the files in buildbot-configs for SYMBOL_SERVER_PATH and edited them as I thought appropriate. It's possible that I missed something or did something wrong, certainly. For example, I'm not 100% confident of the release_master.py changes.

Comment 10

[Chris AtLee [:catlee]]

Bits of this may need porting to buildbot-0.8.0 land.

Comment 11

[Philippe M. Chiasson (:gozer)]

Comment on attachment 487963 [details] [diff] [review]
Add post-symbol-upload command to environment

thunderbird/ bits looking good

Comment 12

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 487963 [details] [diff] [review]
Add post-symbol-upload command to environment

Calendar bits look fine, sorry for the delay!

Comment 13

[(not currently active) Ted Mielczarek]

Attachment: Add post-symbol-upload command to environment.

Rebased version of the patch.

Comment 14

[Laura Thomson :laura]

If this takes time to deploy, we can reprocess crashes on Socorro to get the symbols in.  Let me know.

Comment 15

[Chris AtLee [:catlee]]

Comment on attachment 508481 [details] [diff] [review]
Add post-symbol-upload command to environment.

http://hg.mozilla.org/build/buildbot-configs/rev/a13e44636b0d