Closed Bug 512290 Opened 15 years ago Closed 6 years ago

NSS unit tests showing 0% coverage on mozilla-* trees

Categories

(Testing :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: johnath, Unassigned)

References

()

Details

Attachments

(1 file)

NSS runs their unit tests on their own tinderboxes, but because (aiui) we don't on mozilla-* trees, Murali's code coverage tools miss security/nss entirely.  We should figure out what we need to do to get those numbers reporting alongside our other tests.

In bug 504080 comment 21, Kai explains how to run the tests manually, but ideally this would integrate with our regular, automated runs.

Copying Nelson in as well as sheriff of the NSS trees, though I suspect this should just be a matter of waking the tests up on our trees, and making sure the code coverage instrumentation works.
I don't even see mozilla/security/nss anywhere in the report!
Do your coverage tests do absolutely no SSL at all?
I would imagine that, even if they just do a few SSL pages, 
some files of NSS/SSL would show up in the coverage reports.  ?!?!?
(In reply to comment #1)
> I don't even see mozilla/security/nss anywhere in the report!
> Do your coverage tests do absolutely no SSL at all?
> I would imagine that, even if they just do a few SSL pages, 
> some files of NSS/SSL would show up in the coverage reports.  ?!?!?
Yeah, I started researching this today after Jonathan and I talked.  It turns out that the way that we build NSS in the Mozilla tree prevents us from instrumenting that code for code coverage builds.  (See bug 512341 which I linked to above).

Also, we are not running the NSS test suite at all as part of the Mozilla Tunit test suite.  And so, if we were instrumenting that code we still wouldn't see great coverage there because we don't have enough tests using SSL and testing our NSS layers (though we do have that support in our test harness now).

So, I think going forward, we need to do to things:

1. We should look into fixing bug 512341 to make NSS build using our LDFLAGS so that we can instrument that code with code coverage.  If this is somehow bad for NSS as you allude to in comment 1 on that bug, then perhaps we can look into turning this on ONLY for code coverage builds.

2. We should get the NSS tests running on the standard Tunit test suite that we run for all Mozilla builds.  This looks pretty easy, we just need to do the work to include these tests in buildbot.
Depends on: 512341
For your browsing pleasure
(In reply to comment #3)
> Created an attachment (id=396328) [details]
> NSS test coverage report (from January)
> 
> For your browsing pleasure

How did you generate instrumented builds. Please let me know.
Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: