1900471 - Are Chrome and Chromium-as-release actually different builds on Android?

Status: RESOLVED FIXED

(Testing :: Performance, task, P1)

Description

[Markus Stange [:mstange]]

This graph shows that the numbers from chrome-m and cstm-car-m on Pixel 6 are awfully in sync with each other:

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,5050973,1,13&series=mozilla-central,5053103,1,13&timerange=7776000

Are the two numbers actually collected from Chrome and Chromium-as-Release? Or are we maybe using Chromium-as-Release for both? Why would they move at the same time?

Comment 1

[Greg Mierzwinski [:sparky]]

:kshampur, can you look into this? Looks like it's chrome release being tested on the custom-car-m runs based on the versions in the perfherder data:

Custom-car-m: https://firefoxci.taskcluster-artifacts.net/a6CXx_HWRlCtxs6KbVdsEA/0/public/test_info/perfherder-data.json
Chrome-m: https://firefoxci.taskcluster-artifacts.net/ebFJKLuDRgyr7Uuf4zfGBQ/0/public/test_info/perfherder-data.json

Comment 2

[Kash Shampur [:kshampur] ⌚EST]

The correct --app and --binary are being used according to the logs
e.g.

[task 2024-06-04T04:45:18.376Z] 04:45:10     INFO -  raptor-main Info: received command line arguments: Namespace(test='speedometer3', app='cstm-car-m', binary='/builds/task_171747620672403/fetches/chromium/apks/ChromePublic.apk', ...

I'll keep looking

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

May have found the issue with the installation. Will verify with some Try runs

Comment 4

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1900471 - Fix Android CaR installation. r?#perftest

The custom Chromium-as-Release builds were not being correctly installed
before on android from the fetches. This patch fixes that by adding a
install-chromium-android method to install with adb via mozdevice.

Until now the logs were showing the correct binary and app so it was
easily missed. It was incorrectly implemented in the
install-chromium-distributions method but that only "installed" it on
the host Linux device, and not actually on the Android device

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fc4bf2933e4
Fix Android CaR installation. r=perftest-reviewers,sparky

Comment 6

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/8fc4bf2933e4