1468535 - Port raptor-speedometer firefox to linux64

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement)

Comment 1

[Robert Wood [:rwood]]

Try run to see latest status of Raptor tests on all platforms (only OSX is green so far):

https://treeherder.mozilla.org/#/jobs?repo=try&revision=54dde805af6faeac619af64144b0310ce0f8ea75&filter-tier=1&filter-tier=2&filter-tier=3

Comment 2

[Robert Wood [:rwood]]

Also add it to mozilla-central tier 3.

Comment 3

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=76b860112fd0147856eb20d137c6c0554eb1d7f5

Comment 4

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=06356a6229308a7c12deace5fef0dd69e254c86c

Comment 5

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c53307dc4c79dda03cd5b1967ce00cb50845a903

Comment 6

[Robert Wood [:rwood]]

Ah, speedometer on linux failure was a legitimate intermittent failure, one cycle was sometimes taking longer than the page_timeout of 2 min. Increasing the page_timeout fixes this.

Comment 7

[Robert Wood [:rwood]]

Attachment: Bug 1468535 - Port raptor-speedometer firefox to linux64;

Review commit: https://reviewboard.mozilla.org/r/252382/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/252382/

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8987150 [details]
Bug 1468535 - Port raptor-speedometer firefox to linux64;

https://reviewboard.mozilla.org/r/252382/#review258856

::: testing/raptor/raptor/tests/raptor-speedometer.ini:11
(Diff revision 1)
>  
>  [DEFAULT]
>  type =  benchmark
>  test_url = http://localhost:<port>/Speedometer/index.html?raptor
> -page_cycles = 1
> -page_timeout = 120000
> +page_cycles = 5
> +page_timeout = 180000

do you think this is enough?  I guess we can revisit when we see more intermittent timeouts.  Also, it would be nicer to have a better error message to know this is a page_timeout issue- I know it said timeout, but we suspected other things.

Comment 9

[Robert Wood [:rwood]]

Comment on attachment 8987150 [details]
Bug 1468535 - Port raptor-speedometer firefox to linux64;

https://reviewboard.mozilla.org/r/252382/#review258856

> do you think this is enough?  I guess we can revisit when we see more intermittent timeouts.  Also, it would be nicer to have a better error message to know this is a page_timeout issue- I know it said timeout, but we suspected other things.

Did a bunch of try runs and it looks fine, this is just the timeout for a single cycle, not all 5 pagecycles. Right now raptor only has the one timeout - if measurements aren't received by the page_cycle timeout then it times out. That could be caused by lots of reasons i.e. webext is broken, etc. but there is actually only one overall timeout. I think the key is to capture other errors (Bug 1470450 - Caputure raptor webext js console errors).

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c2c89b7bb8b5
Port raptor-speedometer firefox to linux64; r=jmaher

Comment 11

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/c2c89b7bb8b5