1316929 - allow staging buckets to be whitelisted in Balrog

Status: RESOLVED INCOMPLETE

(Release Engineering Graveyard :: Applications: Balrog (backend), defect)

Description

[Mihai Tabara [:mtabara]⌚️GMT]

20:10:09 <mtabara> http://bucketlister-delivery.stage.mozaws.net/ is not whitelisted and not sure how to deal with this
20:10:10 <mtabara> since it's a staging bucklet
20:10:13 <~bhearsum> ah
20:10:28 <~bhearsum> that's interesting, we've not had to deal with that post-cloudops migration
20:10:52 <~bhearsum> do we want that domain whitelisted for stage permanently, or only temporarily?
20:11:03 <~bhearsum> ie: until we have a production bucket lister
20:11:43 <mtabara> our plan is: 1. make sure our balrogscript submits the correct release blob to http://bucketlister-delivery.stage.mozaws.net/ 2. switch to http://bucketlister-delivery.prod.mozaws.net/pub/mobile/nightly/ which if my understanding is correct backs-up http://archive.mozilla.org/pub/mobile/nightly/
20:12:25 <~bhearsum> ok, so using bucketlister stage is a temporary measure
20:12:46 <~bhearsum> long term, both balrog stage and prod would want to use bucketlister prod - right?
20:13:40 <mtabara> yes,  using bucketlister stage is a temporary measure
20:13:44 <~bhearsum> okay
20:14:11 <~bhearsum> so i'm going to ignore the general problem and see what we can do as a short term workaround...
20:15:04 <~bhearsum> those whitelists are defined in https://github.com/mozilla/balrog/blob/master/uwsgi/public.wsgi and admin.wsgi, and those files are shared between stage and prod
20:16:24 <mtabara> so should I add them https://github.com/mozilla/balrog/blob/master/uwsgi/admin.wsgi#L8 and do a PR?
20:16:48 <mtabara> well, not so much of "them" but "it" :)
20:16:57 <~bhearsum> maybe, but it's not acceptable to stage bucketlister stage to the whitelist for balrog prod for the real Fennec product - that's a security risk
20:17:45 <~bhearsum> one thing you might be able to do is whitelist bucketlister stage for some fake product names (eg: Fennec-bucketlisttest), and make sure you set that as the product when submitting to balrog stage
20:18:00 <~bhearsum> that'd require tweaking the submission scripts too probably
20:22:18 <mtabara> hm
20:22:51 <~bhearsum> not a great option, i know
20:23:31 <mtabara> so instead of "Fennec-mozilla-central-nightly-latest" id't be "Fennec-somethingXYZ-mozilla-central-nightly-latest"?
20:23:37 <mtabara> sorry
20:23:44 <mtabara> * "Fennec-date-nightly-latest"
20:23:47 <~bhearsum> no, you wouldn't need to adjust the Release name, just the product
20:24:25 <mtabara> ah, ok
20:24:28 <~bhearsum> in your pastebin you've got:
20:24:30 <~bhearsum> 2016-11-11T13:51:50     INFO - 2016-11-11 13:51:50,776 - DEBUG - Data sent: {'alias': 'null', 'product': u'Fennec', 'hashFunction': u'sha512', 'data_version': '1', 'data': '{"buildID": "20161111122445", "platformVersion": "52.0a1", "displayVersion": "52.0a1", "completes": [{"fileUrl": 
20:24:34 <~bhearsum> "http://bucketlister-delivery.stage.mozaws.net/pub/mobile/nightly/2016/11/2016-11-11-12-24-45-date-android-api-15/fennec-52.0a1.multi.android-arm.apk", "hashValue": "4e79b0d0deb76eb1340ad6f46bc6a0496c1f87e1f8d4f24816ab5897bd0c339a000fb9365993c5b156c5a0426d44d1eeb3058132e33663f6a6c7fb211a2a4edc", "from": "*", "filesize": 33212642}], "appVersion": "52.0a1"}', 'schema_version': 4}
20:24:39 <~bhearsum> and it's only "product" that needs to change - nothing in the data nor the URL

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Temporarily whitelist stage bucket to allow testing of nightlies Tier-2

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 8809918 [details] [review]
Temporarily whitelist stage bucket to allow testing of nightlies Tier-2

Review given in PR.

Comment 3

[[github robot]]

Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/7b46f60baad3023875efafdf340eeea0453dcac8
Bug 1316929 - temporarily whitelist stage bucket to allow testing of … (#176)

* Bug 1316929 - temporarily whitelist stage bucket to allow testing of nightlies Tier-2. r=bhearsum, r=Callek

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bug 1316929 - whitelist nightlies stage buckets for public uwsgi. r=bhearsum, r=Callek

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

In order to get our updates served by Balrog staging, we've switch to production credentials that are tightened to http://bucketlister-delivery.prod.mozaws.net/pub/{firefox,mobile}/nightly/ which is at its turn backfilling http://archive.mozilla.org/pub/{mobile,firefox}/nightly/ which is already whitelisted in both stage/prod Balrog instances. 

Therefore, I'm keeping this bug open as we'd still want the staging bucket http://bucketlister-delivery.stage.mozaws.net be whitelisted at least in the Balrog staging so that, for future use, we can have something to default to.

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/1d97cda509b1f19cb18114538e191e8fb18a8ba9
Backout stage bucket whitelisting from bug 1316929 (#176) because it is no longer necessary.

Comment 7

[[e-mail] (:bhearsum)]

We decided not to do this for now, but may come back to it later.

Comment 8

[Rail Aliiev [:rail]]

This change will be deployed in bug 1320966 sometime between 11am and 1pm pacific on 2016-11-30. IT will be deployed to stage today. Can you verify your fixes once stage is deployed please?

Comment 9

[Justin Wood (:Callek)]

(In reply to Rail Aliiev [:rail] ⌚️ET from comment #8)
> This change will be deployed in bug 1320966 sometime between 11am and 1pm
> pacific on 2016-11-30. IT will be deployed to stage today. Can you verify
> your fixes once stage is deployed please?

This was backed out.