1161730 - reduce the android test pages which are redundant

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

removing a lot of tp4m pages, 3 tsvgx pages (calling it tsvgm now) and reducing the cycles of tsvgm from 10 (which is defined in the talos.json) to 7.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: reduce android pages (1.0)

Comment 2

[Geoff Brown [:gbrown]]

Comment on attachment 8601702 [details] [diff] [review]
reduce android pages (1.0)

Review of attachment 8601702 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks Joel. How much time does this save? How did you determine redundancy?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I will verify how much time we save prior to pushing this.

As for how I determined redundancy, I used perfherder to look at the data over time, for example:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%257B%2522project%2522%253A%2522mozilla-central%2522%252C%2522signature%2522%253A%2522b78c4ec3f458b5d98643b770aca9ceff4e73cd4e%2522%252C%2522visible%2522%253Atrue%257D&series=%257B%2522project%2522%253A%2522mozilla-central%2522%252C%2522signature%2522%253A%25220bdc44ba4b39ccf94be2be8b9fd1e2a816772254%2522%252C%2522visible%2522%253Atrue%257D&zoom=%7B%7D

I did this and found that so many tests had virtually identical patterns and results.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

previously we had around 4.5 minutes to run the test, with the reduced pages we have around 3.5 minutes.

I need to figure out the svg stuff, that will be smaller as we just trimmed a couple of pages which are fast to load.

While this isn't much overall savings, it does make a difference when we scale out.  Sadly our tp4m jobs take ~60 minutes so the <5 minutes of testing doesn't account for much.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

gbrown, do you have additional thoughts?

Comment 6

[Geoff Brown [:gbrown]]

As long as we are not losing any testing value, a 1 minute time reduction is a good thing -- let's go ahead with this, then see what can be done about the other 55 minutes.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

https://hg.mozilla.org/build/talos/rev/800c3fa4a260