1577108 - Talos wiki page has confusing references to tp5 tests

Status: RESOLVED INCOMPLETE

(Testing :: Talos, defect, P3)

Description

[Mark Banner (:standard8)]

I had performance regression bug filed on tp5n nonmain_startup_fileio opt e10s stylo. So I tried looking for it on the Talos page at:

https://wiki.mozilla.org/TestEngineering/Performance/Talos/Tests

There were several problems:

• There's isn't a clear tp5n section. tp5n gets mentioned within a few sections, but the tp5 section doesn't mention itself as to the relations and meanings of tp5n and tp5o.
• nonmain_startup_fileio opt has a section, but only to mention a possible regression cause. It should have a clear description of all the items it measures.
• Information as to which platforms the test runs on would also be useful.
• Although the tp5 section lists Joel Maher as the contact, he redirected me to Dave Hunt as the ownership of the tests (and there could be other tests where this is no longer true).

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Mark Banner (:standard8) from comment #0)

I had performance regression bug filed on tp5n nonmain_startup_fileio opt e10s stylo. So I tried looking for it on the Talos page at:

https://wiki.mozilla.org/TestEngineering/Performance/Talos/Tests

There were several problems:

• There's isn't a clear tp5n section. tp5n gets mentioned within a few sections, but the tp5 section doesn't mention itself as to the relations and meanings of tp5n and tp5o.

I agree, this isn't clear. My understanding is that tp5n is a set of page loads that are used by other tests. In your case, these were "xperf" tests. I think we should move the xperf under a tp5n section, with details of each of subtest.

• nonmain_startup_fileio opt has a section, but only to mention a possible regression cause. It should have a clear description of all the items it measures.

As above, each subtest should be mentioned under a tp5n/xperf with a short description of what it's measuring.

• Information as to which platforms the test runs on would also be useful.

I believe xperf tests only run on Windows, which we should mention in the tp5n/xperf section.

• Although the tp5 section lists Joel Maher as the contact, he redirected me to Dave Hunt as the ownership of the tests (and there could be other tests where this is no longer true).

I've updated this to be myself, however I intend to change this to someone on my team once a few of us are more familiar with these tests. :)

Comment 2

[Greg Mierzwinski [:sparky]]

:standard8, can you provide a link to the regression bug that you mentioned in your description comment?

Comment 3

[Mark Banner (:standard8)]

(In reply to Greg Mierzwinski [:sparky] from comment #2)

:standard8, can you provide a link to the regression bug that you mentioned in your description comment?

Bug 1574445

Comment 4

[Greg Mierzwinski [:sparky]]

Resolving this as incomplete. A few of the issues were resolved, and we've filed a couple additional bugs for the remaining work: bug 1943978, and bug 1943975.