1587080 - Build perf test doc verification task for reviewbot

Status: RESOLVED FIXED

(Testing :: Performance, task, P2)

Description

[Greg Mierzwinski [:sparky]]

This bug is to build a task which does the verification step that will be used by reviewbot. See this comment for information on how this should be done: https://bugzilla.mozilla.org/show_bug.cgi?id=1586017#c2

Comment 1

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1587080 - [WIP] Implement dynamic performance testing docs.

Do not land this patch - it's incomplete. Pushed only for discussion/viewing purposes.

Comment 2

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1587080 - Reviewbot task for perfdocs.

Depends on D48987

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1587080 - [WIP] Implement dynamic performance testing docs.

Do not land this patch - it's incomplete. Pushed only for discussion/viewing purposes.

Bug 1587080 - Build perf test doc verification task for reviewbot

Comment 5

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1587080 - Part 1: Add performance documentation verification code. r?ahal,#perftest

This patch adds the performance documentation (perfdocs) verification code under tools/lint/perfdocs. This tool currently validates perfdocs folders found within the testing folder to ensure all performance tests have documentation (it only does this for raptor at the moment). See tools/lint/docs/perfdocs.rst for more details.

Comment 6

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1587080 - Part 2: Integrate perfdocs into mozlint. r?ahal,#perftest

This patch integrates the performance documentation (perfdocs) verification into mozlint. It can be used with ./mach lint -l perfdocs.

Depends on D53647

Comment 7

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1587080 - Part 3: Add task for perfdocs verification through mozlint. r?bastien,ahal,#perftest

This is the final patch for perfdocs. It adds a taskcluster task called perfdocs-verify into mozlint that will run the performance documentation verification.

Depends on D53648

Comment 8

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1587080 - Part 4: Add raptor performance test documentation. r?#perftest

This patch adds in-tree performance documentation for raptor which is verified by perfdocs.

Depends on D53649

Comment 9

[Bastien Abadie [:bastien]]

You should use null values to specify full files issues as it's crashing the bot
I filed this issue to fix current crash though: https://github.com/mozilla/code-review/issues/265

Also you can check on our frontend the tasks being run on your revision: https://static-analysis.moz.tools/#/rev/53650

Comment 10

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Done, pushed already.
Thanks for the links, they are helpful!

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/22afcb56e8ea
Part 1: Add performance documentation verification code. r=ahal,perftest-reviewers,alexandru.irimovici,octavian_negru

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4a552742b1cd
Part 2: Integrate perfdocs into mozlint. r=ahal

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/34efe80bfd61
Part 3: Add task for perfdocs verification through mozlint. r=ahal

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/254fd0c3af4a
Part 4: Add raptor performance test documentation. r=perftest-reviewers,octavian_negru

Comment 15

[Noemi Erli[:noemi_erli]]

Backed out 4 changesets (Bug 1587080) for causing perfdocs Lint failure CLOSED TREE

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=280503992&resultStatus=testfailed%2Cbusted%2Cexception&revision=34efe80bfd6105472a1a13731a7087856f56fa02

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=280503992&repo=autoland&lineNumber=295

Backout: https://hg.mozilla.org/integration/autoland/rev/020fa463a54c9d10dc625cd68347b68dcb907baf

Comment 16

[Greg Mierzwinski [:sparky]]

Sorry about that, these should have been pushed all at the same time but were pushed one at a time causing part 3 to fail because part 4 adds the directories that it's looking for.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ec31e2d844f5
Part 1: Add performance documentation verification code. r=ahal,perftest-reviewers,alexandru.irimovici,octavian_negru
https://hg.mozilla.org/integration/autoland/rev/de4f60c208c5
Part 2: Integrate perfdocs into mozlint. r=ahal
https://hg.mozilla.org/integration/autoland/rev/ae682f58ca69
Part 3: Add task for perfdocs verification through mozlint. r=ahal
https://hg.mozilla.org/integration/autoland/rev/e0cf5552fd66
Part 4: Add raptor performance test documentation. r=perftest-reviewers,octavian_negru

Comment 18

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/ec31e2d844f5
https://hg.mozilla.org/mozilla-central/rev/de4f60c208c5
https://hg.mozilla.org/mozilla-central/rev/ae682f58ca69
https://hg.mozilla.org/mozilla-central/rev/e0cf5552fd66

Comment 19

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Sorry guys, there was no option to land them all. Thanks sparky for the help!