1612986 - Hook New Tab/ASRouter/about:welcome etc Karma unit test coverage into codecov.moz.tools & phabricator

Status: RESOLVED FIXED

(Firefox :: Messaging System, task, P2)

Description

[Ed Lee :Mardak]

+++ This bug was initially created as a clone of Bug #1567927 +++

After we get the activity-stream repo unit-tests working in taskcluster/treeherder, it would be nice to get them hooked into the tree-wide code-coverage infrastructure.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

https://github.com/mozilla/code-review/blob/master/docs/publication.md has the documentation related to how to do this. It's probably good to check in with bastien before moving forward to see the exact status of the migration to the new format and to make sure that outputting it will do the trick.

Comment 2

[Marco Castelluccio [:marco]]

(In reply to Dan Mosedale (:dmose, :dmosedale) from comment #1)

https://github.com/mozilla/code-review/blob/master/docs/publication.md has the documentation related to how to do this. It's probably good to check in with bastien before moving forward to see the exact status of the migration to the new format and to make sure that outputting it will do the trick.

Note that this is the documentation to publish linting/static analysis issues on Phabricator. Hooking up code coverage data is a different thing.
You would need to simply make your tasks generate a "code-coverage-grcov.zip" artifact (a zip file containing a text file in LCOV format) such as the ones you can find now on test tasks (e.g. https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cgtVg7oGQByhRbtpVo_A6A/runs/0/artifacts/public/test_info//code-coverage-grcov.zip).

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

We talked about this at the previous retrospective and agreed that we might want this sooner. Setting for reprioritization.

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1612986 - add newtab mocha code coverage info, r?andreio,marco

Comment 5

[Dan Mosedale (:dmosedale, :dmose)]

Attempting to make the above patch integrate correctly is failing in a way that doesn't tell us what the problem is. I've submitted a patch to the coverage bot over in https://github.com/mozilla/code-coverage/issues/1260 to improve the logging, and once that lands, we'll be able to submit another try job, which, if we're lucky, will give us some sense of the next steps are...

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

The problem is exemplified, by https://coverage.moz.tools/#revision=latest&path=browser%2Fcomponents%2Fnewtab&view=directory , where the content-src subdirectory (which contains lots of code) doesn't appear because it doesn't currently get any code coverage data generated. If we're lucky, after this lands to mozilla-central, that will change, and if not, we will have a situation where we can start digging into any necessary next steps.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f38653d6de8
add newtab mocha code coverage info, r=andreio,marco

Comment 8

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/9f38653d6de8

Comment 9

[Dan Mosedale (:dmosedale, :dmose)]

We don't seem to be there quite yet. This patch merged to mozilla-central about 5 hours ago, but https://coverage.moz.tools/#revision=latest&path=browser%2Fcomponents%2Fnewtab&view=directory still doesn't have a content-src subdirectory.

Reviewing lcov.info from inside the code-coverage-grcov.zip artifact from the newtab-ccov job on the treeherder merge commit shows entries with the content-src path, eg:

SF:content-src/aboutwelcome/components/HelpText.jsx

:marco, what would you suggest as far next debugging steps?

Comment 10

[Dan Mosedale (:dmosedale, :dmose)]

Interestingly, I don't see a source-test entry in the test-suite dropdown on https://coverage.moz.tools/.

Comment 11

[Marco Castelluccio [:marco]]

The source-test entry is now available, but there is nothing covered if you select it.

I think the reason for the error is that the SF entries should be full paths to files in the repo, and so SF:content-src/aboutwelcome/components/HelpText.jsx should actually be SF:browser/components/newtab/content-src/aboutwelcome/components/HelpText.jsx.

Comment 12

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1612986 - fix about:welcome/newtab/asrouter unit-test code-cov info, r?emcminn

Comment 13

[Dan Mosedale (:dmosedale, :dmose)]

This bug was mistitled; it does focus on code in the new tab directory, but that directory is shared by a number of things. I've put it back in the Messaging System component, because it's being driven as part of the messaging-system related stuff that is being driven by the OMC team.

And, yeah, this could use a clean up. But that's out of scope for this bug.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/abc3815492f8
fix about:welcome/newtab/asrouter unit-test code-cov info, r=emcminn

Comment 15

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/abc3815492f8