1819426 - Setup Code Coverage

Status: RESOLVED FIXED

(Thunderbird :: Build Config, task)

Description

[Melissa A [:melissa_a]]

We're looking into enable codecoverage for Thunderbird, here's the relevant setup information, if needed I can setup a meeting with macro to hammer out any remaining issues.

via marco on the codecoverage martix channel:

there is no documentation for the set up
for the build side of things, you can take a look at the mozconfigs
https://searchfox.org/mozilla-central/source/browser/config/mozconfigs/linux64/code-coverage-opt
and
https://searchfox.org/mozilla-central/source/browser/config/mozconfigs/linux64/code-coverage
you will need to set up a set of tasks with coverage enabled, and run a script at the end of the test tasks to parse the coverage
https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/codecoverage.py is the script
then there is a service at https://github.com/mozilla/code-coverage which handles getting these artifacts and storing them and serving them
which will need to be adjusted to support multiple repositories and not only mozilla-central
(it should not be too hard to do that)
I can guide you through it if necessary, I would suggest starting from defining the ccov build and the ccov test tasks

Comment 2

[Rob Lemley [:rjl]]

Attachment: Bug 1819426 - Use defaults more liberally for Linux CI builds. r=#thunderbird-build-system-reviewers

Comment 3

[Rob Lemley [:rjl]]

Attachment: Bug 1819426 - Add Linux64 code-coverage builds. r=#thunderbird-build-system-reviewers

Depends on D173222

Comment 4

[Rob Lemley [:rjl]]

Attachment: Bug 1819426 - Add tests for Linux64 code-coverage builds. r=#thunderbird-build-system-reviewers

Depends on D173223

Comment 5

[Rob Lemley [:rjl]]

Here's a try run with Linux64 ccov build and tests.
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=d422a517d0ec96457edfc769b9f361b7af5888d9

The build itself has a code-coverage-grcov.zip artifact, and the tests have code-coverage-grcov.zip and code-coverage-jsvm.zip.

There's a lot of errors like:

[task 2023-03-21T20:35:33.838Z] Error: No objdir path for /builds/worker/workspace/build/tests/xpcshell/tests/comm/mailnews/news/test/unit/test_xover.js.
[task 2023-03-21T20:35:33.838Z] Couldn't find source info for /builds/worker/workspace/build/tests/xpcshell/tests/comm/mailnews/news/test/unit/test_xover.js, removing record

Not sure if that's a problem or not.

Let's work with Linux64 for now and I will add macOS and Win64 as work progresses.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/79904918756a
Use defaults more liberally for Linux CI builds. r=dandarnell
https://hg.mozilla.org/comm-central/rev/3fc97de78116
Add Linux64 code-coverage builds. r=dandarnell
https://hg.mozilla.org/comm-central/rev/c39f9d9e5712
Add tests for Linux64 code-coverage builds. r=dandarnell

Comment 7

[Rob Lemley [:rjl]]

reopen for follow-up due to linux64-aarch64 build failures.

Comment 8

[Rob Lemley [:rjl]]

Attachment: Bug 1819426 - Follow-up: Fix Rust failure on linux64-aarch64 builds. r=#thunderbird-build-system-reviewers

Build was mistakenly changed to use "linux64-rust-dev".

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/215eadd6cb52
Follow-up: Fix Rust failure on linux64-aarch64 builds. r=dandarnell

Comment 10

[Intermittent Failures Robot]

1 failures in 3863 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 1

Platform and build breakdown:

• linux64-aarch64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1819426&startday=2023-03-27&endday=2023-04-02&tree=all