1422737 - [SeaMonkey] Change symbol upload URL from Socorro to Tecken

Status: RESOLVED FIXED

(SeaMonkey :: Release Engineering, defect)

Description

[Justin Wood (:Callek)]

+++ This bug was initially created as a clone of Bug #1422735 +++

peterbe has split symbol handling out of Socorro into a standalone service called Tecken: https://github.com/mozilla-services/tecken/

It's live serving symbols.mozilla.org right now, but we're still uploading symbols via Socorro's symbol upload API. We'd like to switch over uploads to Tecken. This should be as simple as just changing the upload URL to https://symbols.mozilla.org/upload/ here:
https://dxr.mozilla.org/mozilla-central/rev/574f4f58fe09dd590ea892406e237318c31705b4/toolkit/crashreporter/tools/upload_symbols.py#28

================

SeaMonkey will need a different approach, but before I hand off to the people there, :peterbe, is this an API that is suitable for SeaMonkey to switch to as well?

Comment 1

[(not currently active) Ted Mielczarek]

We're planning on switching over all symbol uploads to Tecken, including third-party uploaders, so this should be fine. The plan is to migrate existing tokens, so you should only have to change the upload URL.

Comment 2

[(not currently active) Ted Mielczarek]

You also don't have to be tied in any way to the timeline of the Firefox switchover, as we have no timeline to turn off the Socorro API currently. We'll wait until all uploaders have switched before discussing that.

Comment 3

[Peter Bengtsson [:peterbe]]

Symbols (codename Tecken) has, very similar to Socorro, the ability to upload to different S3 buckets depending on the email of the uploader. It's what we use for routing uploads of non-open symbols (e.g. Adobe Flash). This is that they can't be publicly downloaded.

We can config Symbols so that a dedicated email address (or email simple regex) redirects uploads into a different S3 bucket. 
I know it's maybe odd to have 1 email set up for this purpose even though it's a team of people. However, the API keys can be "shared". Meaning, instead of relying on one human doing a curl upload every time, you can (safely) put the API key into a script that the whole team shares. 

Where does SeaMonkey upload symbols at the moment?

Comment 4

[Peter Bengtsson [:peterbe]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> You also don't have to be tied in any way to the timeline of the Firefox
> switchover, as we have no timeline to turn off the Socorro API currently.
> We'll wait until all uploaders have switched before discussing that.

Depending on response from email communicating with all the non-Firefox-RelEng people, the symbol upload feature will probably be turned off in Socorro early next year. Once the main firehose of RelEng's Firefox builds has proven to work, I'm going to start dismantling the symbol upload functionality from Socorro. First by changing the UI so that it's clear and then eventually (roughly Q2 2018) we'll disable the symbol upload functionality in Socorro.

Comment 5

[Edmund Wong (:ewong)]

> Where does SeaMonkey upload symbols at the moment?

We upload to https://crash-stats.mozilla.com/symbols/upload

Comment 6

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #5)
> > Where does SeaMonkey upload symbols at the moment?
> 
> We upload to https://crash-stats.mozilla.com/symbols/upload

But, on further inspection, we use symbolpush.mozilla.org.  Is this still valid?

Comment 7

[Peter Bengtsson [:peterbe]]

(In reply to Edmund Wong (:ewong) from comment #6)
> (In reply to Edmund Wong (:ewong) from comment #5)
> > > Where does SeaMonkey upload symbols at the moment?
> > 
> > We upload to https://crash-stats.mozilla.com/symbols/upload
> 
> But, on further inspection, we use symbolpush.mozilla.org.  Is this still
> valid?

I don't know what symbolpush.mozilla.org is. It doesn't event connect. Not even on VPN. 

However, I can see in the Crash-stats symbol upload logs that [e-mail] is uploading actively. It's several uploads per day. 

This needs to change and your credentials are still valid. The new URL needs to be https://symbols.mozilla.org/upload/
Let me know how if anything I can do to help.

Comment 8

[(not currently active) Ted Mielczarek]

(In reply to Edmund Wong (:ewong) from comment #6)
> But, on further inspection, we use symbolpush.mozilla.org.  Is this still
> valid?

This is an artifact of the old SSH-based upload system. That host no longer exists. It's just a config variable that never got removed.

Comment 9

[Edmund Wong (:ewong)]

I've figured out what's going on.

Apparently we use the same url as Firefox and Thunderbird from
toolkit/crashreporter/tools/upload_symbols.py

Should I explicitly state the SOCORRO_SYMBOL_UPLOAD_URL in the env?

Comment 10

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #9)
> I've figured out what's going on.
> 
> Apparently we use the same url as Firefox and Thunderbird from
> toolkit/crashreporter/tools/upload_symbols.py
> 
> Should I explicitly state the SOCORRO_SYMBOL_UPLOAD_URL in the env?

and apparently https://hg.mozilla.org/mozilla-central/rev/57736bb879c1 has changed it such
that we need this secret_name, which points to a taskcluster url which gets the token.

And since we don't have access to the taskcluster secret_name..  we'll need to find an inventive
way of skipping it and let it use our socorrow key

Comment 11

[Edmund Wong (:ewong)]

Attachment: [mozconfigs] proposed patch

Note, while the patch specifies "SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE",
it isn't a Socorro token.  It's a Tecken token, which means
we do not need to override the upload url since we'll also
be using symbols.mozilla.org.

Comment 12

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8936329 [details] [diff] [review]
[mozconfigs] proposed patch

NIT missing LF on the .common files. 

> \ No newline at end of file

suite\config\mozconfigs\macosx64\debug does not include the .common. Please add if needed.

r+  with NITS fixed.

Comment 13

[Edmund Wong (:ewong)]

(In reply to Frank-Rainer Grahl (:frg) from comment #12)
> Comment on attachment 8936329 [details] [diff] [review]
> [mozconfigs] proposed patch
> 
> NIT missing LF on the .common files. 
> 
> > \ No newline at end of file
> 
> suite\config\mozconfigs\macosx64\debug does not include the .common. Please
> add if needed.

You mean macosx.common mozconfig, right?

Comment 14

[Edmund Wong (:ewong)]

Attachment: [mozconfigs] a simpler proposed patch [checked-in]

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/8ecd82f4b211
Port relevant parts of Bug 1424651 to SeaMonkey r=frg

Comment 16

[Edmund Wong (:ewong)]

Comment on attachment 8936345 [details] [diff] [review]
[mozconfigs] a simpler proposed patch [checked-in]

[Approval Request Comment]
Regression caused by (bug #): 1422737
User impact if declined: no symbols uploaded (or even nightly generated)
Testing completed (on m-c, etc.): 
Risk to taking this patch (and alternatives if risky):
String changes made by this patch: None

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/37c896fb3de9
Port relevant parts of Bug 1424651 to SeaMonkey r=frg

Comment 18

[Frank-Rainer Grahl (:frg)]

> # Common statements that are applicable to Windows x86 and x64.

You accidently added a blank as first char in the latest push.

Comment 19

[Ian Neal]

Comment on attachment 8936345 [details] [diff] [review]
[mozconfigs] a simpler proposed patch [checked-in]

a=me for those repos that have bug 1422735 landed on them.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/09fe256af08d
Use mk_add_options instead of just export and fix eof

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/283f951180b5
Use mk_add_options instead of just export and fix eof r=bustagefix

Comment 22

[Edmund Wong (:ewong)]

Attachment: [mozconfigs] need to specify MOZ_SCM_LEVEL in environment

apparently I missed the requirement of needing MOZ_SCM_LEVEL stated.
Pushing this for a bustage fix.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/536fa44fd7e4
Require the MOZ_SCM_LVL env set up in order to push to the real symbols url r=bustagefix

Comment 24

[Jorg K (CEST = GMT+2)]

Dec. 2017 was mozilla59, hence SM 2.46.