438725 - graph server test naming scheme incomplete (new front end)

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect, P2)

Description

[alice nodelman [:alice] [:anode]]

In an effort to make it easier to understand/find tests a naming scheme was put in place to replace the technical test name with an easier to understand short hand (thus tp_loadtime_avg became Tp3).  This is incomplete and doesn't include easy names for a number of tests (twinopen, tsspider, tsvg, tjss, tp_RSS_avg, etc).  We need a more complete mapping, have this half way done is making things more confusing.

Comment 1

[Michael Morgan [:morgamic]]

We should add this to a wiki page and document any naming scheme we arrive at.

Comment 2

[Michael Morgan [:morgamic]]

Moving to 0.4 since 0.3 is the 'get stable' milestone.

Comment 3

[Chris Cooper [:coop] (he/him)]

I've culled information on the existing test name mappings from graph.js and put them in the wiki:

https://wiki.mozilla.org/Perfomatic/Test_Name_Mappings

I'll start gathering the missing pieces today.

Comment 4

[Chris Cooper [:coop] (he/him)]

The above wiki page is now complete with all the tests I could find on graphs.m.o and graphs-stage.m.o. I've proposed short names for tests that don't currently have them, and marked them up in yellow.

Alice: can you sanity check the list before I generate a patch for this? Some of the tests I'm not all that familiar with, especially where two tests *might* refer to the same thing, e.g. tjss and tsspider.

Comment 5

[alice nodelman [:alice] [:anode]]

- Tjss and Tsunspider are different test suites.  Tjss is also know as Dromaeo, if that helps (https://wiki.mozilla.org/Dromaeo).

- all of the tp_js_* are tp2 tests.  This was something that I ended up doing in Talos as a stop gap to differentiate between tp2 and the pageloader before the pageloader was accepted as tp3

- Twinopen = Txul.  Again, my bad.  I tried to force a general re-name of Txul to Twinopen so that the test name would actually describe what was happening.  I think that I only managed to confuse people.

- tp_RSS should be in the set as Tp3 (Mem-RSS)

- tp_XRes should be in the set as Tp3 (XRes)

As a general rule, the naming scheme is tp_metric, where metric is the thing being monitored during the given run of tp3.  So, a given run will result in test results for tp_loadtime (all the page load results averaged), tp_RSS/PrivateBytes/WorkingSet (memory information sampled every second during the test and then averaged), tp_%CPU (cpu usage sampled every second during the test and then averaged).  I tried to extend that to tp_js/tp2.  It was supposed to indicate that RSS/%CPU/etc weren't extra tests, but are a part of a tp test.

Comment 6

[Chris Cooper [:coop] (he/him)]

Attachment: Add shortnames for all tests

Comment 7

[alice nodelman [:alice] [:anode]]

Comment on attachment 334019 [details] [diff] [review]
Add shortnames for all tests

tp_js_* is Tp2 - it just happens to be run under the talos framework and ended up with a different name, but it is a direct port of Tp2.

Comment 8

[Chris Cooper [:coop] (he/him)]

(In reply to comment #7)
> (From update of attachment 334019 [details] [diff] [review])
> tp_js_* is Tp2 - it just happens to be run under the talos framework and ended
> up with a different name, but it is a direct port of Tp2.

So are they all simply "Tp2," or "Tp2 (metric)" like for the Tp3 tests?

Comment 9

[alice nodelman [:alice] [:anode]]

"Tp2 (metric)" like for the Tp3 tests.

Comment 10

[Chris Cooper [:coop] (he/him)]

Attachment: [checked in] Add shortnames for all tests, take 2

Comment 11

[Chris Cooper [:coop] (he/him)]

Comment on attachment 334891 [details] [diff] [review]
[checked in] Add shortnames for all tests, take 2

changeset:   103:3579c94a5eb8

Comment 12

[Ryan Doherty (:rdoherty)]

Is this bug done? If so, let's close it.