1894317 - Alert #34 - mozilla-central - 34.87% AndroidStartup:fenix cold_main_first_frame.median (android-hw-a51-11-0-aarch64-shippable)

Status: RESOLVED INVALID

(Fenix :: Performance, defect)

Description

[Chris Peterson [:cpeterson]]

Attachment: Perfherder_screenshot.png

Fenix's "cold_main_first_frame" perf test regressed about 30% (from 830 ms to 1100 ms) on 2024-03-26:

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,4677930,1,15&series=mozilla-central,4607255,1,15&series=mozilla-central,4607258,1,15&series=mozilla-central,4677937,1,15&timerange=7776000

Neither Focus's nor GeckoViewExample's cold_main_first_frame time regressed, so this seems to be something Fenix specific:

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,4607251,1,15&series=mozilla-central,4607261,1,15&series=mozilla-central,4677924,1,15&series=mozilla-central,4677934,1,15&timerange=7776000

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,4754836,1,15&series=mozilla-central,4754839,1,15&timerange=7776000

Curiously, the mozilla-central pushlog for the regression includes aglavic's change to start running "cold_main_first_frame" on Chrome Android (Bug 1836330). Could those Chrome test changes might have affected the Fenix tests?

The mozilla-central pushlog:

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=86f79ebd25420a0bc1feadea488ca0b1c59771c6&tochange=a93eeb987e4a1e533ac94ec0d1703c6f9b52b9ef

Comment 1

[Chris Peterson [:cpeterson]]

Perfherder regression alert:

https://treeherder.mozilla.org/perfherder/alerts?id=34

Comment 2

[Donal Meehan [:dmeehan]]

:kaya, pinging as triage owner, we are in the final week of beta for Fx126.
Could this be triaged for investigation?

Comment 3

[Chris Peterson [:cpeterson]]

Curiously, the mozilla-central pushlog for the regression includes aglavic's change to start running "cold_main_first_frame" on Chrome Android (Bug 1836330). Could those Chrome test changes might have affected the Fenix tests?

Andrej, did you have a chance to review whether your Chrome test changes might have caused a regression on Fenix's tests? It seems unlikely but the timing is suspicious. Note that the regression only affects Fenix: Focus and GeckoViewExample are unaffected, suggesting this is caused by some Fenix-specific code.

Comment 4

[Kayacan Kaya [:kaya]]

:dmeehan, I see Chris already asked for a review of a patch and waiting for the validation I'll quickly go over the pushlog that Chris linked in the description today and try to spot if there're any other suspicious changes. I don't set the prirority but in case I spot any suspicious patch, I'll comment it here.

Comment 5

[Andrej (:aglavic)]

I believe this is because of a code typo that skips the skip on-boarding, which causes significant delays. I've pushed a patch to double check this I will know soon what it says

Comment 6

[Andrej (:aglavic)]

I can confirm this is a regression due to infrastructure, sorry for the scare everyone

Comment 7

[Andrej (:aglavic)]

There is a smaller regression(~70ms) in comparison to after the march 26th infrastructure changes, I will bisect and see if I can figure out the date/dates(in case of multiple) the regression(s) started

Comment 8

[Chris Peterson [:cpeterson]]

(In reply to Andrej (:aglavic) from comment #6)

I can confirm this is a regression due to infrastructure, sorry for the scare everyone

That's a relief. In that case, I'll resolve the Perfherder alert as WONTFIX (even though you are still investigating the smaller ~70ms regression from the same day).

Comment 9

[Kayacan Kaya [:kaya]]

Hey Andrej, any updates for this ticket? Can we close this?

Comment 10

[Andrej (:aglavic)]

Yes this ticket can be closed, I mis-read a comment about closing as won't fix. I thought it was about the ticket, not the perfherder alert.
Sorry for that