1378241 - Make it simpler to disable tests when running on linux64-jsdcov.

Status: RESOLVED INVALID

(Testing :: Code Coverage, enhancement)

Description

[Greg Mierzwinski [:sparky]]

This bug is for making it easier to disable tests on linux64-jsdcov and also cleaning up the 'runtests.py' (or other) files which have 'mozinfo.update({'coverage': True})'. There are a few things that need to be done here:

1. Add an environment variable that can tell us when the build is instrumented with the JS Debugger. Like the "MOZ_CODE_COVERAGE" flag that is used by linux64-ccov. Maybe "MOZ_JSDCOV_COVERAGE"? (There may be a better way of doing this also).

2. Use this flag to add an additional field to moz-info that will specify if js debugger code coverage is being run. The name should be 'jscoverage' so that we can disable tests within either linux64-ccov or linux64-jsdcov and not both. (See bug 1378239). We should add the field through this file: https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/mozinfo.py#77

3. Finally, clean up all 'runtests.py' files and redo the flag names used by all the skipped tests.

After these steps, the flag 'jscoverage' should be usable in any test suite to skip a test.

Comment 1

[Gabor Krizsanits (INACTIVE)]

So what's the current way to disable a test for jsdcov? Disable it for linux?

Comment 2

[Greg Mierzwinski [:sparky]]

Right now we add a flag to mozinfo right before the tests start (the flag name is 'coverage'): https://dxr.mozilla.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#1263-1264

It's far from ideal right now and only works in xpcshell, but it can be added to others if needed. What suite are you looking to disable a test in?

Comment 3

[Gabor Krizsanits (INACTIVE)]

(In reply to Greg Mierzwinski [:gmierz] from comment #2)
> Right now we add a flag to mozinfo right before the tests start (the flag
> name is 'coverage'):
> https://dxr.mozilla.org/mozilla-central/source/testing/xpcshell/
> runxpcshelltests.py#1263-1264
> 
> It's far from ideal right now and only works in xpcshell, but it can be
> added to others if needed. What suite are you looking to disable a test in?

It's a browser chrome mochitest. For now I just disabled the test for linux since I could not find any better way to land the patch. It's a platform independent test, so it should be OK for now, I'll just follow this bug for now.

Comment 4

[Greg Mierzwinski [:sparky]]

The linux64-jsdcov build has been disabled, and no longer runs in taskcluster, see bug 1496791.