1547975 - Fennec64 cold load with live sites

Status: RESOLVED INVALID

(Testing :: Raptor, enhancement, P1)

Comment 1

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Try build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0709c08ceda5e8e0bdb2acdb9d1a4a85a2f099d

Comment 2

[Robert Wood [:rwood]]

(In reply to Alexandru Ionescu from comment #1)

Try build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0709c08ceda5e8e0bdb2acdb9d1a4a85a2f099d

Hey Alex!

You'll notice these jobs timed out - each job was trying to run the fennec-cold job as well as the fennec-v64-cold-live jobs that you added; so that exceeds the overall time limit of 30min.

That's because in each test INI in the try push had both test definitions for each page, i.e.:

[raptor-tp6m-facebook-fennec-cold]
apps = fennec
..

[raptor-tp6m-facebook-fennec-cold-live]
apps = fennec
..

And so all the Fennec tests were run.

Solution is to remove the original [raptor-tp6m-*-fennec-cold] test sections from each INI and push that to try. I'll apply your patch and do that and do another push.

Comment 3

[Robert Wood [:rwood]]

FYI, because of Bug 1546880, the HTTP test pages will timeout. There's only one:

[raptor-tp6m-espn-fennec-cold] (tp6m-cold-13)

We'll just have to go ahead and get numbers for the others (HTTPS).

Comment 4

[Robert Wood [:rwood]]

Hopefully this should do it:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=06c7ab039326c68a2406b7e5c416bb589f15aed1

Alex, if you could please look at ^ when you arrive, hopefully the jobs will be done and you can get the numbers from there for the comparison sheet. Thanks!

Comment 5

[Robert Wood [:rwood]]

Nope:

Running: python2.7 /builds/task_1556657194/workspace/mozharness/scripts/raptor_script.py --config-file /builds/task_1556657194/workspace/mozharness/configs/raptor/android_hw_config.py --cfg mozharness/configs/raptor/android_hw_config.py --test=raptor-tp6m-cold-2 --app=fennec --binary=org.mozilla.firefox --disable-e10s --download-symbols ondemand --test=raptor-tp6m-cold-2 --app=fennec --binary=org.mozilla.firefox --disable-e10s
Usage: raptor_script.py [options]

raptor_script.py: error: no such option: --disable-e10s

Comment 6

[Robert Wood [:rwood]]

Alex, I thought you landed a fix for ^ already?

Comment 7

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #6)

Alex, I thought you landed a fix for ^ already?

Ah, I found this:

https://phabricator.services.mozilla.com/D26901

Looks like I need to apply that also... will do that and make another try push attempt...

Comment 8

[Robert Wood [:rwood]]

Ok, let's hope this finally does it:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=dff5cb07be555b0c26c02db7923a896d277eefbc

Alex, if you could please look at ^ when you arrive, hopefully the jobs will be done and you can get the numbers from there for the comparison sheet. Thanks!

Comment 9

[Robert Wood [:rwood]]

Several raptor.yml taskcluster job limits exceeded (mostly on Moto G5). Another push with increased tc job limit:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad529236fd474ac35d28170c154c6f657b7bcb1d

Comment 10

[Robert Wood [:rwood]]

(Alex is on PTO so I've assigned this to me to finish up)

Comment 11

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #9)

Several raptor.yml taskcluster job limits exceeded (mostly on Moto G5). Another push with increased tc job limit:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad529236fd474ac35d28170c154c6f657b7bcb1d

P2 Ax64 looks great but MG5 jobs not running for some reason ^ so did another try push for them:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ff0ad42aeb5f919189fac834ee204ef3ea74d9c3

Comment 12

[Robert Wood [:rwood]]

The MG5 jobs did start running on the first try push (I didn't wait long enough) so cancelled the 2nd push. So this is try push for both P2 and MG5:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad529236fd474ac35d28170c154c6f657b7bcb1d

Comment 13

[Robert Wood [:rwood]]

Attachment: fennec64_cold_live.diff

Not for review as not landing this - just a record of the diff pushed to try in order to get Fennec v64 cold page-load live sites on try.

Comment 14

[Robert Wood [:rwood]]

Got most of the numbers, just another push to try for MG5 tp6m*-12 and tp6m*-14 with an even longer taskcluster job time, to try to get a couple more sites:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1f59723264d395b845272522fd1dbd83d795487c

Comment 15

[Robert Wood [:rwood]]

Finished gathering numbers for as many Fennec cold page-load with live sites as could get to pass. Closing this bug as resolved invalid as there is no patch to land, was just pushing to try for perf numbers.