Closed Bug 1580476 Opened 3 months ago Closed 3 months ago

Switch to new GCP bouncer workers

Categories

(Release Engineering :: Release Automation: Bouncer, task)

task
Not set

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: mtabara, Assigned: mtabara)

References

(Blocks 1 open bug)

Details

(Keywords: leave-open)

Attachments

(2 files)

+++ This bug was initially created as a clone of Bug #1579476 +++

Rail gave me a tour in the new world today. It looks like we can switch to the new bouncer workers hosted by cloudops \o/

Incoming patch. For now just on nightlies until we test the netflows. Failing to update the bouncer-locations jobs in nightlies is not a big deal so we can afford the testing. Once netflows work and confirmed, we can uplift to betas to start testing bouncer-submission and bouncer-aliases.

Blocks: 1580478
Blocks: 1533337
No longer blocks: 1580478
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/742ec1eaf446
Switch to new GCP bouncer workers. r=rail a=release
Keywords: leave-open

I turned on bouncer in nightlies for the bouncer-locations job. The worker exists in GCP, it claimed successfully, CoT passed but then it failed as it can't access bouncer.

2019-09-17 11:31:39,782 - bouncerscript.utils - INFO - Performing a GET request to https://bounceradmin.mozilla.com/api/product_show?product=firefox-nightly-latest with kwargs {'timeout': 60}
2019-09-17 11:31:39,900 - bouncerscript.utils - WARNING - Cannot access https://bounceradmin.mozilla.com/api/product_show?product=firefox-nightly-latest: Cannot connect to host bounceradmin.mozilla.com:443 ssl:None [Name or service not known]

Full log is here. I'll file a bug to talk this to CloudOps.

Depends on: 1581795

Note to self: once bouncer prod is whitelisted, we need to rerun a nightly job and that should go green. Then we uplift to beta and that's it!
Note to self again: we don't need autoscaling in bouncer. One pod should suffice as we only have bouncer-locations (one job per nightly graphs), bouncer-submission x 2 (one per promotion graph x 2 (Nazgul))andbouncer-aliases` in ship phase of the release graphs.

I'll test this in the morning with taskgraph-diff against beta and graft to catch the 70.0b8. But that's train riding optimization, from GCP perspective, we can close this.

Note for posterity: we don't have autoscaling for this particular workertype because we have very few jobs for this across all trees.

Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Regressions: 1583236
You need to log in before you can comment on or make changes to this bug.