1470226 - improve testing for bouncer aliases task as it's a leaf node

Status: VERIFIED FIXED

(Release Engineering :: Release Automation: Other, enhancement)

Description

[Mihai Tabara [:mtabara]⌚️GMT]

a) we can add a follow-up API check to ensure the proper aliases have been updated to the corresponding values - this ensures the service has been correctly updated

b) Sanity checks that the values are correspondent to the specific tree / product

c) (glorious future) something to double check that the values we play with in the bouncer aliases task are the correct ones.

====

Focusing on a) for now. b) has been done in bug 1445684 and c) is not necessarily a valid one as we trust the tree.

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

I'm trying to understand if there's a way to query the API to get the current values of the bouncer aliases. I've been trying various ways (via `product_show` and faking a GET method instead of the `create_update_alias` but sounds like the API doesn't have anything specific to return me that data. API is here[1].

Do you know anything offhand or is there no way other than maybe querying the public API via urls such as this[2]?

[1]: https://github.com/mozilla/tuxedo/blob/master/apps/api/views.py. do you know anything offhand or should I be parsing the public
[2]: https://download.mozilla.org/?product=firefox-latest-ssl

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Hm, playing a bit with requests and the public url.

Querying stuff like https://download.mozilla.org/?product={alias} (e.g. `firefox-latest-ssl`) and then maybe test if $version is in returned url? 

def validation(version, alias):
    r = requests.get('https://download.mozilla.org/?product={}'.format(alias))
    if version not in r.url:
         raise Exception("Bouncer not correctly updated or w\e ...")

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

nthomas|away> I think there might be something, have a vague memory of getting a plain text response from bouncer for something
11:29:06 <nthomas|away> might have been an alias, or just a regular request
11:30:27 <nthomas|away> the actual code serving redirects these days is https://github.com/mozilla-services/go-bouncer/
11:33:05 <nthomas|away> I was thinking of this last query arg print=yes
11:33:07 <nthomas|away> eg https://download.mozilla.org/?product=firefox-latest-ssl&os=osx&lang=en-US&print=yes
11:36:25 <nthomas|away> there's this AliasFor function to look up aliases, https://github.com/mozilla-services/go-bouncer/blob/9e6880874b7425e340c6a2080b673db056db45c3/bouncer/db.go#L37, but not exposed anywhere
11:37:29 <nthomas|away> so that only saves you parsing headers vs content :/

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

I've tested all aliases from here[1] for https://download.mozilla.org/?product={alias}&print=yes and Fennec / Thunderbird don't have those entries. We need to investigate more here before implementing this check.

[1]: https://github.com/mozilla-releng/bouncerscript/blob/master/bouncerscript/constants.py#L1

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

Note to self: if we go down this path we also need the version passed down in the in-tree.

Comment 6

[Rail Aliiev [:rail]]

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #1)
> I'm trying to understand if there's a way to query the API to get the
> current values of the bouncer aliases. I've been trying various ways (via
> `product_show` and faking a GET method instead of the `create_update_alias`
> but sounds like the API doesn't have anything specific to return me that
> data. API is here[1].
> 
> Do you know anything offhand or is there no way other than maybe querying
> the public API via urls such as this[2]?
> 
> [1]: https://github.com/mozilla/tuxedo/blob/master/apps/api/views.py. do you
> know anything offhand or should I be parsing the public
> [2]: https://download.mozilla.org/?product=firefox-latest-ssl

I don't see anything views.py nor in https://github.com/mozilla/tuxedo/blob/master/apps/api/urls.py. Probably there is no API for that... Shouldn't be hard to submit a PR if we are desperate ;)

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Maybe you can compare the results of (eg)
  https://download.mozilla.org/?product=firefox-latest-ssl&print=yes
  https://download.mozilla.org/?product=firefox-61.0&print=yes

Comment 8

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Nick Thomas [:nthomas] (UTC+12) from comment #7)
> Maybe you can compare the results of (eg)
>   https://download.mozilla.org/?product=firefox-latest-ssl&print=yes
>   https://download.mozilla.org/?product=firefox-61.0&print=yes

Brilliant idea Nick. I ran a small script to test which aliases are missing and I got this:

thunderbird-next-latest Not found!
thunderbird-next-latest-ssl Not found!
thunderbird-latest-ssl Not found!
fennec-beta-latest Not found!
fennec-latest Not found!
thunderbird-beta-latest => https://download-installer.cdn.mozilla.net/pub/thunderbird/releases/60.0b9/win32/en-US/Thunderbird%20Setup%2060.0b9.exe
thunderbird-beta-latest-ssl => https://download-installer.cdn.mozilla.net/pub/thunderbird/releases/60.0b9/win32/en-US/Thunderbird%20Setup%2060.0b9.exe
thunderbird-latest => https://download-installer.cdn.mozilla.net/pub/thunderbird/releases/52.9.0/win32/en-US/Thunderbird%20Setup%2052.9.0.exe
firefox-devedition-stub => https://download-installer.cdn.mozilla.net/pub/devedition/releases/62.0b6/win32/en-US/Firefox%20Installer.exe
firefox-devedition-latest => https://download-installer.cdn.mozilla.net/pub/devedition/releases/62.0b6/win32/en-US/Firefox%20Setup%2062.0b6.exe
firefox-devedition-latest-ssl => https://download-installer.cdn.mozilla.net/pub/devedition/releases/62.0b6/win32/en-US/Firefox%20Setup%2062.0b6.exe
firefox-beta-stub => https://download-installer.cdn.mozilla.net/pub/firefox/releases/62.0b6/win32/en-US/Firefox%20Installer.exe
firefox-beta-latest => https://download-installer.cdn.mozilla.net/pub/firefox/releases/62.0b6/win32/en-US/Firefox%20Setup%2062.0b6.exe
firefox-beta-latest-ssl => https://download-installer.cdn.mozilla.net/pub/firefox/releases/62.0b6/win32/en-US/Firefox%20Setup%2062.0b6.exe
firefox-stub => https://download-installer.cdn.mozilla.net/pub/firefox/releases/61.0.1/win32/en-US/Firefox%20Installer.exe
firefox-latest => https://download-installer.cdn.mozilla.net/pub/firefox/releases/61.0.1/win32/en-US/Firefox%20Setup%2061.0.1.exe
firefox-latest-ssl => https://download-installer.cdn.mozilla.net/pub/firefox/releases/61.0.1/win32/en-US/Firefox%20Setup%2061.0.1.exe
firefox-esr-latest => https://download-installer.cdn.mozilla.net/pub/firefox/releases/52.9.0esr/win32/en-US/Firefox%20Setup%2052.9.0esr.exe
firefox-esr-latest-ssl => https://download-installer.cdn.mozilla.net/pub/firefox/releases/52.9.0esr/win32/en-US/Firefox%20Setup%2052.9.0esr.exe
firefox-esr-next-latest => https://download-installer.cdn.mozilla.net/pub/firefox/releases/60.1.0esr/win32/en-US/Firefox%20Setup%2060.1.0esr.exe
firefox-esr-next-latest-ssl => https://download-installer.cdn.mozilla.net/pub/firefox/releases/60.1.0esr/win32/en-US/Firefox%20Setup%2060.1.0esr.exe
firefox-sha1 => https://download-installer.cdn.mozilla.net/pub/firefox/releases/52.9.0esr/win32-sha1/en-US/Firefox%20Setup%2052.9.0esr.exe
firefox-sha1-ssl => https://download-installer.cdn.mozilla.net/pub/firefox/releases/52.7.3esr/win32-sha1/en-US/Firefox%20Setup%2052.7.3esr.exe

Sounds like I can compare same url for alias vs version and it'll work smoothly!

Comment 9

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: [bouncerscript] Add bouncer-aliases post-update checks for data validation

Similar to what we had last week for bouncer submission PR, here's the bouncer-aliases counterpart.

Comment 10

[Mihai Tabara [:mtabara]⌚️GMT]

Comment on attachment 8990938 [details] [review]
[bouncerscript] Add bouncer-aliases post-update checks for data validation

Johan approved this in the PR with some comments which I addressed in follow-up commits. 

Merged this to master: https://github.com/mozilla-releng/bouncerscript/commit/205665d0a0ee8a496233c451c8ce6734fb29914a

Comment 11

[Mihai Tabara [:mtabara]⌚️GMT]

Closing this as this will get picked up by the next roll-out of bouncerscript and land in production.

Comment 12

[Mihai Tabara [:mtabara]⌚️GMT]

Rolled-out bump in bouncerscript production via https://github.com/mozilla-releng/build-puppet/pull/109/files

Comment 13

[Mihai Tabara [:mtabara]⌚️GMT]

https://taskcluster-artifacts.net/OgCFskaKSaGl5BwnJTfBBA/0/public/logs/live_backing.log
https://taskcluster-artifacts.net/Pu2JDGByTLKihDXwwOK0eA/0/public/logs/live_backing.log