1781237 - Permafailling android pixel 2 chrome tests

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P2)

Description

[Greg Mierzwinski [:sparky]]

All chrome tests on the pixel2 are currently permafailing, see here: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=chrome%2Cbrowsertime%2Cpixel&revision=00a40cdda673bdbe4f7831d1cf078909ad54182e

Relevant log portion:

[task 2022-07-25T21:36:00.017Z] 21:35:50     INFO -  raptor-browsertime Info: Run tests on Pixel 2 [HT7A51A04660] using Android version 8.0.0
[task 2022-07-25T21:36:00.017Z] 21:35:50     INFO -  raptor-browsertime Info: Running tests using Chrome - 1 iteration(s)
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info: chrome failed to start, trying 2 more time(s): unknown error: com.android.chrome is not installed on device HT7A51A04660
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info: chrome failed to start, trying 1 more time(s): unknown error: com.android.chrome is not installed on device HT7A51A04660
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info: chrome failed to start, trying 0 more time(s): unknown error: com.android.chrome is not installed on device HT7A51A04660
[task 2022-07-25T21:36:00.017Z] 21:35:51    ERROR -  raptor-browsertime Error: [chrome-m] Browsertime failed to run
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info:     at SeleniumRunner.start (/builds/task_165878474549360/fetches/browsertime/node_modules/browsertime/lib/core/seleniumRunner.js:87:13)
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info:     at processTicksAndRejections (node:internal/process/task_queues:96:5)
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info:     at async Iteration.run (/builds/task_165878474549360/fetches/browsertime/node_modules/browsertime/lib/core/engine/iteration.js:114:7)
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info:     at async Engine.runByScript (/builds/task_165878474549360/fetches/browsertime/node_modules/browsertime/lib/core/engine/index.js:266:20)
[task 2022-07-25T21:36:00.017Z] 21:35:51     INFO -  raptor-browsertime Info:     at async run (/builds/task_165878474549360/fetches/browsertime/node_modules/browsertime/bin/browsertime.js:74:22)
[task 2022-07-25T21:36:00.017Z] 21:35:51    ERROR -  raptor-browsertime Error: [chrome-m] Browsertime failed to run
[task 2022-07-25T21:36:00.017Z] 21:35:51 CRITICAL -  raptor-browsertime Critical: [chrome-m] No data to collect

Comment 1

[Andrej (:aglavic)]

Attachment: Bug 1781237 - Skip chrome tests on Pixel2. r=#perftest

What we are doing:

• Disabling chrome-m tests to run on Pixel2 devices

Why:

• Recent bitbar cleanup script update is causing permafails on chrome

Comment 2

[Andrej (:aglavic)]

Attachment: Bug 1781237 - Skip chrome tests on Pixel2. r=#perftest

What we are doing:

• Disabling chrome-m tests to run on Pixel2 devices

Why:

• Recent bitbar cleanup script update is causing permafails on chrome

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5bf851607e85
Skip chrome tests on Pixel2. r=perftest-reviewers,AlexandruIonescu

Comment 4

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/5bf851607e85

Comment 5

[Greg Mierzwinski [:sparky]]

The issue hasn't been fixed so I've added the leave-open flag until then.

Comment 6

[Andrej (:aglavic)]

Waiting on finishing the transition from the P2 to a51s. Which assuming all goes well should be done by late December/early January

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We're no longer running tests against Pixel 2, the A51 work will continue in bug 1780817.