1641933 - [perfdocs] Add raptor descriptions back to perfdocs in a non-verbose way

Status: RESOLVED FIXED

(Testing :: Performance, task, P2)

Description

[Greg Mierzwinski [:sparky]]

We should add the descriptions back but in a more non-verbose method. :myeongjun proposed one method here which would use abbreviations to denote various settings that these tests have - I think this might be the nicest way (we can provide a legend at the top of the doc): https://bugzilla.mozilla.org/show_bug.cgi?id=1640684#c3

Comment 1

[Myeongjun Go]

Hello Sparky :)
There are some things that need to be checked[0] before proceeding with the development.
It doesn't use 'tests' sub-description as the previous patch. (e.g - Cold and warm desktop page-load performance tests for amazon on firefox, chrome, and chromium using WebExtension.)

I think It need sub-description as in the following example.

desktop:
    description: "Tests for page-load performance. (W: WebExtension, B:Browsertime, F: Firefox, C: Chrome, CU: chromium)"
       tests:
         raptor-tp6-amazon: "W, F, C, CU"
         raptor-tp6-facebook: "W, C, CU"

So, I have to modify generator.py[1] and related part.

But, There is simple thing. I don't modify sub-description. I just add description to test title as in the following example.

desktop:
    description: "Tests for page-load performance. (W: WebExtension, B:Browsertime, F: Firefox, C: Chrome, CU: chromium)"
       tests:
         raptor-tp6-amazon (W, F, C, CU): "Cold and warm desktop page-load performance tests for amazon on firefox, chrome, and chromium using WebExtension."
         raptor-tp6-facebook (W, F, C, CU): "Cold and warm desktop page-load performance test for facebook, chrome, and chromium using WebExtension."

It'll look simple to the user in the same. (Because, It doesn't use sub-description anyway)
I wonder what you think :)
Thanks.

[0]
https://searchfox.org/mozilla-central/rev/598e50d2c3cd81cd616654f16af811adceb08f9f/testing/raptor/raptor/perfdocs/config.yml

[1]
https://searchfox.org/mozilla-central/rev/598e50d2c3cd81cd616654f16af811adceb08f9f/tools/lint/perfdocs/generator.py#98

Comment 2

[Greg Mierzwinski [:sparky]]

Hi Myeongjun! :)

I really like this method:

desktop:
    description: "Tests for page-load performance. (W: WebExtension, B:Browsertime, F: Firefox, C: Chrome, CU: chromium)"
       tests:
         raptor-tp6-amazon: "W, F, C, CU"
         raptor-tp6-facebook: "W, C, CU"

The descriptions that we currently have should probably be removed because they repeat the same thing that is in the desktop description.

We can replicate your second idea without much effort using the short descriptions above: https://searchfox.org/mozilla-central/source/tools/lint/perfdocs/framework_gatherers.py#174

return ["* " + title + " " + test_description]

The issue with changing the test name (i.e. to raptor-tp6-amazon (W, F, C, CU): "Cold ...) is that then you'll have to modify how we find the tests in our test manifests: https://searchfox.org/mozilla-central/source/tools/lint/perfdocs/verifier.py#86-200

What perfdocs does is it verifies that all of the tests you've defined have a description, and that the tests you've given descriptions for also exist as a test. If either these conditions fail then we get a reviewbot error in our phabricator reviews.

Comment 3

[Myeongjun Go]

Attachment: Bug 1641933 - [pefrdocs] add desciptions to perfdocs using single letter r=sparky

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43d1fd1d8e95
[pefrdocs] add desciptions to perfdocs using single letter r=sparky,perftest-reviewers

Comment 5

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/43d1fd1d8e95