Closed Bug 1422737 Opened 2 years ago Closed 2 years ago

[SeaMonkey] Change symbol upload URL from Socorro to Tecken

Categories

(SeaMonkey :: Release Engineering, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
seamonkey2.46

People

(Reporter: Callek, Assigned: ewong)

References

Details

Attachments

(2 files, 1 obsolete file)

+++ This bug was initially created as a clone of Bug #1422735 +++

peterbe has split symbol handling out of Socorro into a standalone service called Tecken: https://github.com/mozilla-services/tecken/

It's live serving symbols.mozilla.org right now, but we're still uploading symbols via Socorro's symbol upload API. We'd like to switch over uploads to Tecken. This should be as simple as just changing the upload URL to https://symbols.mozilla.org/upload/ here:
https://dxr.mozilla.org/mozilla-central/rev/574f4f58fe09dd590ea892406e237318c31705b4/toolkit/crashreporter/tools/upload_symbols.py#28

================

SeaMonkey will need a different approach, but before I hand off to the people there, :peterbe, is this an API that is suitable for SeaMonkey to switch to as well?
Flags: needinfo?(peterbe)
Summary: Change symbol upload URL from Socorro to Tecken → [SeaMonkey] Change symbol upload URL from Socorro to Tecken
We're planning on switching over all symbol uploads to Tecken, including third-party uploaders, so this should be fine. The plan is to migrate existing tokens, so you should only have to change the upload URL.
You also don't have to be tied in any way to the timeline of the Firefox switchover, as we have no timeline to turn off the Socorro API currently. We'll wait until all uploaders have switched before discussing that.
Symbols (codename Tecken) has, very similar to Socorro, the ability to upload to different S3 buckets depending on the email of the uploader. It's what we use for routing uploads of non-open symbols (e.g. Adobe Flash). This is that they can't be publicly downloaded.

We can config Symbols so that a dedicated email address (or email simple regex) redirects uploads into a different S3 bucket. 
I know it's maybe odd to have 1 email set up for this purpose even though it's a team of people. However, the API keys can be "shared". Meaning, instead of relying on one human doing a curl upload every time, you can (safely) put the API key into a script that the whole team shares. 

Where does SeaMonkey upload symbols at the moment?
Flags: needinfo?(peterbe)
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> You also don't have to be tied in any way to the timeline of the Firefox
> switchover, as we have no timeline to turn off the Socorro API currently.
> We'll wait until all uploaders have switched before discussing that.

Depending on response from email communicating with all the non-Firefox-RelEng people, the symbol upload feature will probably be turned off in Socorro early next year. Once the main firehose of RelEng's Firefox builds has proven to work, I'm going to start dismantling the symbol upload functionality from Socorro. First by changing the UI so that it's clear and then eventually (roughly Q2 2018) we'll disable the symbol upload functionality in Socorro.
> Where does SeaMonkey upload symbols at the moment?

We upload to https://crash-stats.mozilla.com/symbols/upload
(In reply to Edmund Wong (:ewong) from comment #5)
> > Where does SeaMonkey upload symbols at the moment?
> 
> We upload to https://crash-stats.mozilla.com/symbols/upload

But, on further inspection, we use symbolpush.mozilla.org.  Is this still valid?
Flags: needinfo?(peterbe)
(In reply to Edmund Wong (:ewong) from comment #6)
> (In reply to Edmund Wong (:ewong) from comment #5)
> > > Where does SeaMonkey upload symbols at the moment?
> > 
> > We upload to https://crash-stats.mozilla.com/symbols/upload
> 
> But, on further inspection, we use symbolpush.mozilla.org.  Is this still
> valid?

I don't know what symbolpush.mozilla.org is. It doesn't event connect. Not even on VPN. 

However, I can see in the Crash-stats symbol upload logs that ewong@pw-wspx.org is uploading actively. It's several uploads per day. 

This needs to change and your credentials are still valid. The new URL needs to be https://symbols.mozilla.org/upload/
Let me know how if anything I can do to help.
Flags: needinfo?(peterbe)
(In reply to Edmund Wong (:ewong) from comment #6)
> But, on further inspection, we use symbolpush.mozilla.org.  Is this still
> valid?

This is an artifact of the old SSH-based upload system. That host no longer exists. It's just a config variable that never got removed.
I've figured out what's going on.

Apparently we use the same url as Firefox and Thunderbird from
toolkit/crashreporter/tools/upload_symbols.py

Should I explicitly state the SOCORRO_SYMBOL_UPLOAD_URL in the env?
(In reply to Edmund Wong (:ewong) from comment #9)
> I've figured out what's going on.
> 
> Apparently we use the same url as Firefox and Thunderbird from
> toolkit/crashreporter/tools/upload_symbols.py
> 
> Should I explicitly state the SOCORRO_SYMBOL_UPLOAD_URL in the env?

and apparently https://hg.mozilla.org/mozilla-central/rev/57736bb879c1 has changed it such
that we need this secret_name, which points to a taskcluster url which gets the token.

And since we don't have access to the taskcluster secret_name..  we'll need to find an inventive
way of skipping it and let it use our socorrow key
Attached patch [mozconfigs] proposed patch (obsolete) — Splinter Review
Note, while the patch specifies "SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE",
it isn't a Socorro token.  It's a Tecken token, which means
we do not need to override the upload url since we'll also
be using symbols.mozilla.org.
Assignee: nobody → ewong
Status: NEW → ASSIGNED
Attachment #8936329 - Flags: review?(frgrahl)
Comment on attachment 8936329 [details] [diff] [review]
[mozconfigs] proposed patch

NIT missing LF on the .common files. 

> \ No newline at end of file

suite\config\mozconfigs\macosx64\debug does not include the .common. Please add if needed.

r+  with NITS fixed.
Attachment #8936329 - Flags: review?(frgrahl) → review+
(In reply to Frank-Rainer Grahl (:frg) from comment #12)
> Comment on attachment 8936329 [details] [diff] [review]
> [mozconfigs] proposed patch
> 
> NIT missing LF on the .common files. 
> 
> > \ No newline at end of file
> 
> suite\config\mozconfigs\macosx64\debug does not include the .common. Please
> add if needed.

You mean macosx.common mozconfig, right?
Attachment #8936329 - Attachment is obsolete: true
Attachment #8936345 - Flags: review?(frgrahl)
Attachment #8936345 - Flags: review?(frgrahl) → review+
Pushed by ewong@pw-wspx.org:
https://hg.mozilla.org/comm-central/rev/8ecd82f4b211
Port relevant parts of Bug 1424651 to SeaMonkey r=frg
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Comment on attachment 8936345 [details] [diff] [review]
[mozconfigs] a simpler proposed patch [checked-in]

[Approval Request Comment]
Regression caused by (bug #): 1422737
User impact if declined: no symbols uploaded (or even nightly generated)
Testing completed (on m-c, etc.): 
Risk to taking this patch (and alternatives if risky):
String changes made by this patch: None
Attachment #8936345 - Attachment description: [mozconfigs] a simpler proposed patch → [mozconfigs] a simpler proposed patch [checked-in]
Attachment #8936345 - Flags: approval-comm-release?
Attachment #8936345 - Flags: approval-comm-esr52?
Attachment #8936345 - Flags: approval-comm-beta?
Pushed by ewong@pw-wspx.org:
https://hg.mozilla.org/comm-central/rev/37c896fb3de9
Port relevant parts of Bug 1424651 to SeaMonkey r=frg
> # Common statements that are applicable to Windows x86 and x64.

You accidently added a blank as first char in the latest push.
Comment on attachment 8936345 [details] [diff] [review]
[mozconfigs] a simpler proposed patch [checked-in]

a=me for those repos that have bug 1422735 landed on them.
Attachment #8936345 - Flags: approval-comm-release?
Attachment #8936345 - Flags: approval-comm-release+
Attachment #8936345 - Flags: approval-comm-esr52?
Attachment #8936345 - Flags: approval-comm-esr52+
Attachment #8936345 - Flags: approval-comm-beta?
Attachment #8936345 - Flags: approval-comm-beta+
Pushed by ewong@pw-wspx.org:
https://hg.mozilla.org/comm-central/rev/09fe256af08d
Use mk_add_options instead of just export and fix eof
Pushed by ewong@pw-wspx.org:
https://hg.mozilla.org/comm-central/rev/283f951180b5
Use mk_add_options instead of just export and fix eof r=bustagefix
apparently I missed the requirement of needing MOZ_SCM_LEVEL stated.
Pushing this for a bustage fix.
Pushed by ewong@pw-wspx.org:
https://hg.mozilla.org/comm-central/rev/536fa44fd7e4
Require the MOZ_SCM_LVL env set up in order to push to the real symbols url r=bustagefix
Dec. 2017 was mozilla59, hence SM 2.46.
Target Milestone: --- → seamonkey2.46
You need to log in before you can comment on or make changes to this bug.