Link to devtools test documentation
Categories
(Tree Management :: Perfherder, enhancement, P2)
Tracking
(Not tracked)
People
(Reporter: davehunt, Assigned: jdescottes)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
(Keywords: leave-open)
Attachments
(1 file)
Provide links from Perfherder to devtools test documentation.
|
Reporter | |
Comment 1•3 years ago
|
||
:jdescottes can you provide URLs for the following tests reporting to Perfherder?
- damp
- debugger-metrics
- inspector-metrics
- netmonitor-metrics
- server.pool
- total-after-gc
- webconsole-metrics
Ideally, these URLs would be identical, with an anchor based on the test name. For example AWSY uses URLs such as https://firefox-source-docs.mozilla.org/testing/perfdocs/awsy.html#base-content-explicit
|
Assignee | |
Comment 2•3 years ago
|
||
|
||
Updated•3 years ago
|
|
|
Reporter | |
Comment 3•3 years ago
|
||
We'll need to follow up with a patch for Perfherder.
|
|
Assignee | |
Comment 4•3 years ago
|
||
Once the patch has reached central, the URLs should be:
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#damp
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#debugger-metrics
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#inspector-metrics
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#netmonitor-metrics
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#webconsole-metrics
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#server.pool
- https://firefox-source-docs.mozilla.org/devtools/tests/performance-tests-overview.html#total-after-gc
Pushed by jdescottes@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/13bea1c387b9 [devtools] Add DevTools performance tests overview documentation page r=davehunt,nchevobbe
|
||
Comment 6•3 years ago
|
||
| bugherder | ||
|
||
Comment 7•3 years ago
|
||
After some looking around in Perfherder I think that there needs to be provided more URLs for the following tests:
- total-after-gc (This one was skipped initially by mistake?)
- reload-total-after-gc
- content-total-after-gc
- reload-content-total-after-gc
- toolbox-total-after-gc
- target-total-after-gc
- reload-inspector:content-process
- reload-inspector:parent-process
- reload-debugger:content-process
- reload-debugger:parent-process
- reload-no-devtools:content-process
- reload-no-devtools:parent-process
- reload-netmonitor:content-process
- reload-netmonitor:parent-process
- reload-webconsole:parent-process
- reload-webconsole:content-process
The ones mentioned here can be found on this page https://treeherder.mozilla.org/perfherder/tests?framework=12 in case of need.
|
|
Assignee | |
Comment 8•3 years ago
|
||
Some tests have been removed or renamed.
Using this link provides the list of all the tests that have run recently but it's misleading.
Old tests:
- total-after-gc
- reload-total-after-gc
- content-total-after-gc
- reload-content-total-after-gc
- toolbox-total-after-gc
- target-total-after-gc
Those should not be re-added.
For the others, yes we need to add them. I already added a first batch in https://bugzilla.mozilla.org/show_bug.cgi?id=1729671 but I'd like to wait until things settle a bit, to avoid updating this documentation page too often.
I will file a follow up.
|
|
Assignee | |
Comment 9•3 years ago
•
|
||
Quick question about documenting those tests. At the moment we are still going to modify the tests a lot, some of them might be converted to regular mochitests with asserts, some might be deleted, renamed etc... So there is going to be a good amount of changes.
Could you explain a bit how this documentation is used by sheriffs (taking into consideration that DevTools alerts are sheriffed by our team) ? Any way we could consider the non-documented tests as experimental?
|
|
||
Comment 10•3 years ago
•
|
||
(In reply to Julian Descottes [:jdescottes] from comment #9)
Quick question about documenting those tests. At the moment we are still going to modify the tests a lot, some of them might be converted to regular mochitests with asserts, some might be deleted, renamed etc... So there is going to be a good amount of changes.
Could you explain a bit how this documentation is used by sheriffs (taking into consideration that DevTools alerts are sheriffed by our team) ? Any way we could consider the non-documented tests as experimental?
I have the task to provide links to the DevTools documentation in Perfherder and this is mainly to help your team access the docs easily while sheriffing. There are two options I think, either I do not provide a link for what's left undocumented, either you can add only the title of the tests in Perfdocs and some description "To be defined" (here's an example https://firefox-source-docs.mozilla.org/testing/perfdocs/awsy.html#base-content-resident-unique-memory). Please let me know which approach suits this situation the best. Thanks.
|
|
||
Updated•3 years ago
|
|
|
Assignee | |
Comment 11•3 years ago
|
||
(In reply to Acasandrei Beatrice from comment #10)
(In reply to Julian Descottes [:jdescottes] from comment #9)
Quick question about documenting those tests. At the moment we are still going to modify the tests a lot, some of them might be converted to regular mochitests with asserts, some might be deleted, renamed etc... So there is going to be a good amount of changes.
Could you explain a bit how this documentation is used by sheriffs (taking into consideration that DevTools alerts are sheriffed by our team) ? Any way we could consider the non-documented tests as experimental?
I have the task to provide links to the DevTools documentation in Perfherder and this is mainly to help your team access the docs easily while sheriffing. There are two options I think, either I do not provide a link for what's left undocumented, either you can add only the title of the tests in Perfdocs and some description "To be defined" (here's an example https://firefox-source-docs.mozilla.org/testing/perfdocs/awsy.html#base-content-resident-unique-memory). Please let me know which approach suits this situation the best. Thanks.
Thanks for the info!
either I do not provide a link for what's left undocumented,
Since this is mainly for our team, let's skip it for now because everyone is working on performance and memory topics at the moment. I will just file a bug so that we do a final documentation when the "perf and stability" project is done.
Thanks!
|
|
Reporter | |
Comment 12•3 years ago
|
||
We have a perfdocs project, which generates the documentation based on the test configuration. This is currently only working for the Raptor and mozperftest frameworks, but is planned for Talos and AWSY. I believe devtools and Talos are similar, so perhaps once bug 1674220 is resolved we could implement this for devtools. It would mean that as you add/remove/edit tests, the documentation would be regenerated and always up to date. For now, it's necessary to have static documentation.
Description
•