1900408 - Use CfT chromedriver (Canary or Dev channel) for CaR tests

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Kash Shampur [:kshampur] ⌚EST]

Currently we fetch the drivers from here

But it is not clear to me if these are necessarily stable builds or if they are similar to being experimental (like when we used to fetch chromium binaries from here )

Instead it might be more stable for us to use the download from the "last known good version with downloads" json endpoint, using the Canary channel. (I've got a local patch of this working actually for the Stable channel for Bug 1869592, so this could just be as simple as adding the ability to pass channel flags)

A really quick check - in these failures here (which, based on the error log, is an issue with chrome driver starting?) the revision of that fetched chromedriver from the snapshot link is 1309140
but the "last good" canary chromedriver from the json endpoint is 1309068. I would expect there to be some difference between Canary and raw chromium, but given that the major version in the sem ver are the same as CaR, the safest route is probably to use the builds reported as "good" in the end point

(also note to self, print the revision in the fetch log, even though we already save it as a file, it is convenient to just look a the logs)

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=d9bc0f2330300d649bbda1209d8dd248542f54fa

win/linux seem fine, waiting on intel mac

Comment 2

[Kash Shampur [:kshampur] ⌚EST]

(In reply to Kash Shampur [:kshampur] ⌚EST from comment #1)

https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=d9bc0f2330300d649bbda1209d8dd248542f54fa

win/linux seem fine, waiting on intel mac

mac seems to work with this now with the cft driver

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

the OSX failures haven't seem to resolved itself yet upstream, so I am going to work on this asap

Comment 5

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1900408 - Use CfT chromedriver for CaR tests. r?#perftest

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b700d9b49f9f
Use CfT chromedriver for CaR tests. r=perftest-reviewers,taskgraph-reviewers,sparky,bhearsum

Comment 7

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/b700d9b49f9f