1869592 - Automate chromedriver (stable) updates

Status: RESOLVED FIXED

(Testing :: Performance, enhancement, P1)

Description

[Kash Shampur [:kshampur] ⌚EST]

Although we can't yet use CfT builds to replace our chrome-release tests, this filed bug is to propose/look into a potential "middle ground" of automating just the chromedrivers downloads available from the JSON endpoints here https://googlechromelabs.github.io/chrome-for-testing/#stable

we could add a fetch that always grabs the N latest chromedrivers from the endpoint (or just singular, latest, so that we are forced to just maintain one e.g. bug 1860044)

Also if checksum is a concern, this was brought up https://github.com/GoogleChromeLabs/chrome-for-testing/issues/70#issuecomment-1822476957

so should be safe?

One thing to note is Autoupdates are not reliable. And doesn't happen at all for Macs (relevant, bug 1841106, comment 3)

But this bug could be a good step forward for this

Comment 1

[Greg Mierzwinski [:sparky]]

Regarding checksums, something that wasn't mentioned in the issue you linked is that they provide us with a way of verifying that the download wasn't corrupted from network errors (or other errors), but I don't think it's a big concern or blocker for us.

Comment 2

[Markus Stange [:mstange]]

In bug 1876822, Jeff suggests running \Program Files (x86)\Google\Update\GoogleUpdate.exe /ua /installsource scheduler to force checking for updates before running the test.

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

rcurran confirmed mac auto updates are enabled. So it seems linux + mac are good, but windows and android seem to be lagging by 1-2 versions according to redash and also just checking mozilla-central on treeherder

So the automation we could implement here is just to always grab the 2 latest chromedrivers (2 to be safe) using the CfT json endpoints

Comment 4

[Kash Shampur [:kshampur] ⌚EST]

rewording this task. it was determined that we can do it. And I think we should do it at some point. I've added the perftest triage white board as I'd like to potentially re groom this/ discuss again in perftest triage

Comment 5

[Kash Shampur [:kshampur] ⌚EST]

removing perftest:triage tag, as we decided we will do this sometime Q2/Q3

Comment 6

[Kash Shampur [:kshampur] ⌚EST]

thought: we can continue to manually update windows chromedriver as that is the one platform that is having issues with auto update, and instead automate linux/mac.

best way would be to expand this script https://searchfox.org/mozilla-central/rev/56dd89bcf4d3b85f66621e89eac6e2936ad382d9/taskcluster/scripts/misc/fetch-cft-chromedriver.py to accommodate a channel flag (defaults to canary). We'd use the stable flag for this and only do it for non-win platforms

Comment 7

[Kash Shampur [:kshampur] ⌚EST]

(In reply to Kash Shampur [:kshampur] ⌚EST from comment #6)

thought: we can continue to manually update windows chromedriver as that is the one platform that is having issues with auto update, and instead automate linux/mac.

in Bug 1876822, it seems auto updates have been set up on Windows now using chocolatey.
Apparently WMIC was not reading the correct version though. That may be fixed with Bug 1919649

Another strategy is to automate fetching stable + beta chromedriver, so that 2 versions are always available during the transition period of auto updates

Comment 8

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1869592 - Automate chromedriver (stable) updates. r?#perftest

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a6f3abb11685
Automate chromedriver (stable) updates. r=perftest-reviewers,aglavic,taskgraph-reviewers,bhearsum

Comment 10

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/a6f3abb11685