In an effort to make it easier to understand/find tests a naming scheme was put in place to replace the technical test name with an easier to understand short hand (thus tp_loadtime_avg became Tp3). This is incomplete and doesn't include easy names for a number of tests (twinopen, tsspider, tsvg, tjss, tp_RSS_avg, etc). We need a more complete mapping, have this half way done is making things more confusing.
We should add this to a wiki page and document any naming scheme we arrive at.
Moving to 0.4 since 0.3 is the 'get stable' milestone.
I've culled information on the existing test name mappings from graph.js and put them in the wiki: https://wiki.mozilla.org/Perfomatic/Test_Name_Mappings I'll start gathering the missing pieces today.
The above wiki page is now complete with all the tests I could find on graphs.m.o and graphs-stage.m.o. I've proposed short names for tests that don't currently have them, and marked them up in yellow. Alice: can you sanity check the list before I generate a patch for this? Some of the tests I'm not all that familiar with, especially where two tests *might* refer to the same thing, e.g. tjss and tsspider.
- Tjss and Tsunspider are different test suites. Tjss is also know as Dromaeo, if that helps (https://wiki.mozilla.org/Dromaeo). - all of the tp_js_* are tp2 tests. This was something that I ended up doing in Talos as a stop gap to differentiate between tp2 and the pageloader before the pageloader was accepted as tp3 - Twinopen = Txul. Again, my bad. I tried to force a general re-name of Txul to Twinopen so that the test name would actually describe what was happening. I think that I only managed to confuse people. - tp_RSS should be in the set as Tp3 (Mem-RSS) - tp_XRes should be in the set as Tp3 (XRes) As a general rule, the naming scheme is tp_metric, where metric is the thing being monitored during the given run of tp3. So, a given run will result in test results for tp_loadtime (all the page load results averaged), tp_RSS/PrivateBytes/WorkingSet (memory information sampled every second during the test and then averaged), tp_%CPU (cpu usage sampled every second during the test and then averaged). I tried to extend that to tp_js/tp2. It was supposed to indicate that RSS/%CPU/etc weren't extra tests, but are a part of a tp test.
Created attachment 334019 [details] [diff] [review] Add shortnames for all tests
Comment on attachment 334019 [details] [diff] [review] Add shortnames for all tests tp_js_* is Tp2 - it just happens to be run under the talos framework and ended up with a different name, but it is a direct port of Tp2.
(In reply to comment #7) > (From update of attachment 334019 [details] [diff] [review]) > tp_js_* is Tp2 - it just happens to be run under the talos framework and ended > up with a different name, but it is a direct port of Tp2. So are they all simply "Tp2," or "Tp2 (metric)" like for the Tp3 tests?
"Tp2 (metric)" like for the Tp3 tests.
Created attachment 334891 [details] [diff] [review] [checked in] Add shortnames for all tests, take 2
Comment on attachment 334891 [details] [diff] [review] [checked in] Add shortnames for all tests, take 2 changeset: 103:3579c94a5eb8
Is this bug done? If so, let's close it.