1677128 - Migrate live desktop page load test running on autoland to browsertime

Status: RESOLVED FIXED

(Testing :: Raptor, task, P2)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

In bug 1661545 we started running a live page load test on autoland to detect regressions missed when testing recorded sites. With the ongoing migration to browsertime, we need to migrate this scheduled test to retain coverage.

Comment 1

[Alexandru Ionescu (needinfo me) [:alexandrui]]

sparky is this task blocked by anything?

Comment 2

[Greg Mierzwinski [:sparky]]

Nope, it's not blocked on anything.

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

For what I see, raptor-tp6-live is only running some cold pageload on autoland, the rest of it and the warm being scheduled for mozilla-central. It seems a bit odd to me.
Made a WIP patch. Please let me know if there's anything else to modify.

Comment 4

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1677128 Migrate live desktop page load test running on autoland to browsertime r?#perftest

Comment 5

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Sparky, can I have some help on this error? https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=dDVoINjAQOWGG3RiuX94Mw.0&tier=1%2C2%2C3&revision=83be5f87ab03f2deb995c3356205f408b7ad372c

Comment 6

[Alexandru Ionescu (needinfo me) [:alexandrui]]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #5)

Sparky, can I have some help on this error? https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=dDVoINjAQOWGG3RiuX94Mw.0&tier=1%2C2%2C3&revision=83be5f87ab03f2deb995c3356205f408b7ad372c

This is a blocking error of perfdocs. I can't advance with the migration because of it.

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #6)

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #5)

Sparky, can I have some help on this error? https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=dDVoINjAQOWGG3RiuX94Mw.0&tier=1%2C2%2C3&revision=83be5f87ab03f2deb995c3356205f408b7ad372c

This is a blocking error of perfdocs. I can't advance with the migration because of it.

Is this because cnn-amp is missing from testing/raptor/raptor/tests/tp6/live/browsertime-live.ini in your patch? This could be an opportunity to improve the perfdocs error catching.

I also note you have a description of "Raptor (webextension) tp6 on live-sites" but the test wouldn't be using webextension.

Comment 8

[Alexandru Ionescu (needinfo me) [:alexandrui]]

It's cnn-ampstories. The ini file needs the long form. (look at [ebay-kleinanzeigen], its short form is ebay-k)
Will fix the description inaccuracy.

Comment 9

[Greg Mierzwinski [:sparky]]

:alexandrui, can you debug this locally and see what's going on? It seems like we're not finding a test description for some test (likely cnn-ampstories).

Comment 10

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Ofcourse

Comment 11

[Alexandru Ionescu (needinfo me) [:alexandrui]]

might be an imperfection in the way perfdocs places each manifest under a suite, because i'm getting the error for the mobile suite when i add this to desktop. if i add ony cnn-amp, it doesn't give any error. if i add [cnn-ampstories, cnn-amp], it throws the error.

Comment 12

[Alexandru Ionescu (needinfo me) [:alexandrui]]

FrameworkGatherer._urls is organized by subtests in dictionary keys (which doesn't allow duplications), not by suites. amazon is a test under desktop and mobile suites, but it appears only for mobile. FrameworkGatherer._urls needs to be reorganized like _test_list.
Can you give this to Myeongjun to fix this? He knows better the entire implementation of _urls.

Comment 13

[Alexandru Ionescu (needinfo me) [:alexandrui]]

I opened a bug as dependency for this.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/eda4b16d6dd6
Migrate live desktop page load test running on autoland to browsertime r=perftest-reviewers,Bebe

Comment 15

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/eda4b16d6dd6