Closed Bug 1162060 Opened 9 years ago Closed 9 years ago

Switch any remaining buildbot builds that upload symbols to use Socorro API

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox39 fixed, firefox40 fixed, firefox41 fixed, firefox-esr31 fixed, firefox-esr38 fixed, b2g-v2.0 fixed, b2g-v2.0M fixed, b2g-v2.1 fixed, b2g-v2.1S fixed, b2g-v2.2 fixed, b2g-master fixed)

RESOLVED FIXED
mozilla41
Tracking Status
firefox39 --- fixed
firefox40 --- fixed
firefox41 --- fixed
firefox-esr31 --- fixed
firefox-esr38 --- fixed
b2g-v2.0 --- fixed
b2g-v2.0M --- fixed
b2g-v2.1 --- fixed
b2g-v2.1S --- fixed
b2g-v2.2 --- fixed
b2g-master --- fixed

People

(Reporter: ted, Assigned: ted)

References

Details

Attachments

(1 file)

In bug 1085557 comment 22 nthomas wrote:
> The bit of the b2g build harness you want is at
> http://hg.mozilla.org/build/mozharness/file/default/scripts/b2g_build.
> py#l599, so whatever that uploadsymbols target is in the b2g build system.
> 
> We'll be calling that on any job that uses one these configs -
> http://mxr.mozilla.org/build/search?string=build-
> symbols&find=%2Fmozharness%2F&findi=&filter=^[^\0]*%24&hitlimit=&tree=build
> 
> Looks like a lot of device and emulator builds from a quick skim, but there
> isn't a good mapping to buildbot or Treeherder job names. I can hack/grep
> something up if that'd help.

We need to track down any jobs that are running in buildbot that are still uploading symbols but didn't get switched to use the Socorro API. bug 1085557 switched all the Firefox desktop builds, but Android builds and B2G builds wouldn't have been switched.
> The bit of the b2g build harness you want is at
> http://hg.mozilla.org/build/mozharness/file/default/scripts/b2g_build.
> py#l599, so whatever that uploadsymbols target is in the b2g build system.

uploadsymbols comes from:
https://github.com/mozilla-b2g/gonk-misc/blob/d3868ff4bb3a4b81382795e2784258c210fe6cb8/Android.mk#L325
...which just calls back into Gecko's "make uploadsymbols", naturally.

The default B2G mozconfig does source the in-tree mozconfig nowadays:
https://dxr.mozilla.org/mozilla-central/source/b2g/config/mozconfigs/common

...so we could stick stuff in there, although it's sourced for local builds too (as the file notes).

The Android common mozconfig is here:
https://dxr.mozilla.org/mozilla-central/source/mobile/android/config/mozconfigs/common

We'd just need to make sure the mock environments for these builders are configured to copy the auth token file like was done in bug 1119238 (attachment 8587035 [details] [diff] [review]). I have no idea how that works, so nthomas: if you can look at that that'd be super helpful.

The mozconfig changes should be just one line per product, adding the auth token bit.
Just for sanity, xulrunner builds got switched over because they all include the platform common in-tree mozconfigs where I put the auth token for Firefox builds. Example:
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/2015-05-27-03-02-04-mozilla-central/mozilla-central-linux-xulrunner-nightly-bm77-build1-build29.txt.gz

```
/builds/slave/m-cen-lx-xr-ntly-0000000000000/build/obj-firefox/_virtualenv/bin/python /builds/slave/m-cen-lx-xr-ntly-0000000000000/build/toolkit/crashreporter/tools/upload_symbols.py 'dist/xulrunner-41.0a1.en-US.linux-i686.crashreporter-symbols-full.zip'
Uploading symbol file "dist/xulrunner-41.0a1.en-US.linux-i686.crashreporter-symbols-full.zip" to "https://crash-stats.mozilla.com/symbols/upload"...
Uploaded successfully!
```
If I'm realistic I'm not going to have time to help with this in the near future. :-S
Assignee: nobody → ted
mshal has made me very happy because he switched the Android builds over to mozharness recently, and their mozharness configs have the auth token bits in them, so we should just need a one line mozconfig change to fix those.

RyanVM has also made me very happy by updating the pinned mozharness rev to current production on all branches, so backporting all this stuff to other branches should be easy.

Hooray!
bug 1162060 - add Socorro auth token to Android mozconfigs. r?mshal
Attachment #8612231 - Flags: review?(mshal)
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

bug 1162060 - add Socorro auth token to Android mozconfigs. r?mshal
Er, I forgot to update the changeset description, but I added it to the in-tree B2G mozconfig as well. I might still need to update the B2G device mozharness bits, but the patch as-written should be safe to land, it just won't use the token file unless it exists.
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

https://reviewboard.mozilla.org/r/9539/#review8327

::: b2g/config/mozconfigs/common:19
(Diff revision 2)
> +# try to set it if it doesn't exist.

Can you explain why the if-statement is necessary for B2G in the comment? It looks like the environment overriding / possibly non-existent /builds/crash-stats-api.token isn't used in any of the other configs.
Attachment #8612231 - Flags: review?(mshal)
I answered this on IRC, but I'll put it here as well (and in a comment in the mozconfig as suggested):
12:09 PM <ted> mshal: b2g, as usual, is a special snowflake
12:09 PM <ted> mshal: that mozconfig gets sourced for both automation and local builds
12:12 PM <ted> mshal: and it's going to wind up being used for builds on TaskCluster too, so allowing it to pass through if it's already set gives us flexibility there
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

bug 1162060 - add Socorro auth token to Android and B2G mozconfigs. r?mshal
Attachment #8612231 - Attachment description: MozReview Request: bug 1162060 - add Socorro auth token to Android mozconfigs. r?mshal → MozReview Request: bug 1162060 - add Socorro auth token to Android and B2G mozconfigs. r?mshal
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

bug 1162060 - add Socorro auth token to Android and B2G mozconfigs. r?mshal
Attachment #8612231 - Flags: review?(mshal)
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

https://reviewboard.mozilla.org/r/9539/#review8399

Ship It!
Attachment #8612231 - Flags: review?(mshal) → review+
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum
Attachment #8612231 - Attachment description: MozReview Request: bug 1162060 - add Socorro auth token to Android and B2G mozconfigs. r?mshal → MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum
Attachment #8612231 - Flags: review+ → review?(bhearsum)
Comment on attachment 8612231 [details]
MozReview Request: bug 1162060 - add socorro auth token to B2G device build mock environments. r?bhearsum

https://reviewboard.mozilla.org/r/9539/#review8543

Looks fine to me. Be careful if/when you backport - some branches may have additional configs.
Attachment #8612231 - Flags: review?(bhearsum) → review+
https://hg.mozilla.org/mozilla-central/rev/20d5183e9ecf
https://hg.mozilla.org/mozilla-central/rev/f0120e6d036a
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla41
Flags: needinfo?(ryanvm)
I spot-checked some of this morning's B2G device build nightlies and they uploaded to the Socorro API properly. I tried to check the Android nightlies, but it turns out they're not uploading symbols at all, so I filed bug 1171040.
Gonna let this bake a day or two before doing the remaining b2g/esr branches.
Flags: needinfo?(ryanvm)
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.