1470079 - Create an optimized coverage build with --enable-fuzzing

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Christoph Diehl [:posidron]]

This bug tracks the efforts in creating a coverage build with "--enable-fuzzing" enabled which the Fuzzing team needs for getting coverage data LibFuzzer and other built-in fuzzers like Faulty.

Comment 1

[Raymond Forbes[:rforbes]]

Attachment: Bug 1470079 - Add fuzzing coverage build to taskcluster

Review commit: https://reviewboard.mozilla.org/r/252124/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/252124/

Comment 2

[Raymond Forbes[:rforbes]]

Comment on attachment 8986893 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/252124/diff/1-2/

Comment 3

[Code Review Bot [:reviewbot]]

Comment on attachment 8986893 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252124/#review258614


Code analysis found 1 defect in this patch:
 - 1 defect found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: taskcluster/ci/build/linux.yml:1033
(Diff revision 2)
>      toolchains:
>          - linux64-clang
>          - linux64-gcc
>          - linux64-rust
>          - linux64-rust-size
>          - linux64-sccache

Error: No new line character at the end of file [yamllint: new-line-at-end-of-file]

Comment 4

[Raymond Forbes[:rforbes]]

Attachment: Bug 1470079 - Add fuzzing coverage build to taskcluster

Review commit: https://reviewboard.mozilla.org/r/252186/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/252186/

Comment 5

[Marco Castelluccio [:marco]]

Comment on attachment 8986948 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252186/#review258772

Looks good to me overall, we just need to figure out the tests part.

::: taskcluster/ci/build/linux.yml:617
(Diff revision 1)
> +        script: "mozharness/scripts/fx_desktop_build.py"
> +        secrets: true
> +        custom-build-variant-cfg: fuzzing-cov
> +        tooltool-downloads: public
> +        need-xvfb: true
> +    toolchains:

We might need to add a run-on-projects property to make this only run on mozilla-central and try.
I guess you don't want to run tests but only generate the build, right? If so, we should also disable tests in https://dxr.mozilla.org/mozilla-central/rev/681eb7dfa324dd50403c382888929ea8b8b11b00/taskcluster/taskgraph/transforms/tests.py#710.

Comment 6

[Andreea Pavel [:apavel]]

Cannot land this from mozzreview, it has 1 open issue. Please take a look.

Comment 7

[Raymond Forbes[:rforbes]]

Comment on attachment 8986948 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/252186/diff/1-2/

Comment 8

[Raymond Forbes[:rforbes]]

That should be resolved now. Thanks!

Comment 9

[Code Review Bot [:reviewbot]]

Comment on attachment 8986948 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252186/#review258996


Code analysis found 1 defect in this patch:
 - 1 defect found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: taskcluster/ci/build/linux.yml:1035
(Diff revision 2)
>          - linux64-clang
>          - linux64-gcc
>          - linux64-rust
>          - linux64-rust-size
>          - linux64-sccache
> +        

Error: Trailing spaces [yamllint: trailing-spaces]

Comment 10

[Raymond Forbes[:rforbes]]

Comment on attachment 8986948 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/252186/diff/2-3/

Comment 11

[Raymond Forbes[:rforbes]]

Comment on attachment 8986948 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252186/#review258772

> We might need to add a run-on-projects property to make this only run on mozilla-central and try.
> I guess you don't want to run tests but only generate the build, right? If so, we should also disable tests in https://dxr.mozilla.org/mozilla-central/rev/681eb7dfa324dd50403c382888929ea8b8b11b00/taskcluster/taskgraph/transforms/tests.py#710.

I will submit a new bug/change for that.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/24334b7cffc5
Add fuzzing coverage build to taskcluster r=marco

Comment 13

[Noemi Erli[:noemi_erli]]

Backed out changeset 24334b7cffc5 (bug 1470079) for Linting failure in builds/worker/checkouts/gecko/testing/mozharness/configs/builds/releng_sub_linux_configs/64_fuzzing_cov.py on a CLOSED TREE

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=24334b7cffc517fcdf60f76944c914478cc87b8a&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&selectedJob=184481176

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=184481176&repo=autoland&lineNumber=267

Backout: https://hg.mozilla.org/integration/autoland/rev/836d0afcac5548d250639df7414a331742aa4c10

Comment 14

[Raymond Forbes[:rforbes]]

Attachment: Bug 1470079 - Add fuzzing coverage build to taskcluster

Review commit: https://reviewboard.mozilla.org/r/252652/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/252652/

Comment 15

[Raymond Forbes[:rforbes]]

Comment on attachment 8987404 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/252652/diff/1-2/

Comment 16

[Marco Castelluccio [:marco]]

Comment on attachment 8987404 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252652/#review259188

::: taskcluster/taskgraph/transforms/tests.py:708
(Diff revision 2)
>  
>  @transforms.add
>  def enable_code_coverage(config, tests):
>      """Enable code coverage for the ccov and jsdcov build-platforms"""
>      for test in tests:
> -        if 'ccov' in test['build-platform']:
> +        if (

In the opt and fuzzing cases, we want to disable all tests, so we should override the test task definitions to have empty 'run-on-projects' (otherwise, some of them will run either because they are set to run always or because they inherit the 'run-on-projects' from the build).

You could do something like this:
> if 'ccov' in test['build-platform']:
>     # Ensure we don't run any test for fuzzing or opt ccov builds.
>     if 'opt' in test['build-platform'] or 'fuzzing' in test['build-platform']:
>         test['run-on-projects'] = []
>         continue
> 
>     ...

Comment 17

[Raymond Forbes[:rforbes]]

Comment on attachment 8987404 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/252652/diff/2-3/

Comment 18

[Marco Castelluccio [:marco]]

Comment on attachment 8987404 [details]
Bug 1470079 - Add fuzzing coverage build to taskcluster

https://reviewboard.mozilla.org/r/252652/#review259722

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f5075fe18837
Add fuzzing coverage build to taskcluster r=marco

Comment 20

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/f5075fe18837

Comment 21

[Jesse Schwartzentruber (:truber)]

Please name this -ccov like all the other builds. -cov isn't used anywhere else.

Comment 22

[Marco Castelluccio [:marco]]

(In reply to Jesse Schwartzentruber (:truber) from comment #21)
> Please name this -ccov like all the other builds. -cov isn't used anywhere
> else.

Can you file a new bug for this (blocking this bug)?