1087567 - Add proper manifest for cppunit tests.

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Dan Minor [:dminor]]

Right now when we pass a directory to runcppunittests.py it attempts to run every file in the directory as a unit test. Since we package support files with the tests, things become awkward. As a start, we could look for an executable bit before attempting to run a file as a test.

Fixing this will make running the cppunit tests through mach from the test package nicer.

Comment 1

[(not currently active) Ted Mielczarek]

What do you think about just generating a test manifest from the build system? As a first pass we could just take everything in CPP_UNIT_TESTS and write out a cppunittest.ini in the test package, and then run only things listed there. A later version might be more like:
moz.build:
CPP_UNIT_TEST_MANIFESTS += ['cppunittest.ini']

cppunittest.ini:
[TestWhatever.cpp]
skip-if = os == 'win'

Either the test harness would have to know how to translate source file names to executable names, or we'd have to make the build system translate manifests at build time (neither of which seem particularly hard).

Comment 2

[Dan Minor [:dminor]]

We already have 'manifest' files for B2G and Android that are just lists of tests to skip on those platforms. I think you've convinced me to get rid of those and just add a proper manifest.

Comment 3

[Dan Minor [:dminor]]

Autoland request failed. Too many failures for b2g_emulator_vm try opt test marionette-webapi, Android 4.0 Panda try opt test jittest, Rev4 MacOSX Snow Leopard 10.6 try debug test jetpack, Android 2.3 Emulator try opt test mochitest-6, Rev5 MacOSX Mountain Lion 10.8 try debug test jetpack, Ubuntu VM 12.04 try debug test mochitest-browser-chrome-1, Rev5 MacOSX Mountain Lion 10.8 try opt test jetpack, Ubuntu VM 12.04 x64 try debug test jetpack, Windows 7 32-bit try debug test jetpack, Ubuntu VM 12.04 try debug test jetpack, Windows XP 32-bit try opt test jetpack, Ubuntu VM 12.04 x64 try opt test jetpack, Ubuntu VM 12.04 try opt test jetpack, Windows 7 32-bit try opt test jetpack, Rev4 MacOSX Snow Leopard 10.6 try opt test jetpack.

Comment 4

[Dan Minor [:dminor]]

Autoland request failed: could not transplant: Transplant failed

Comment 5

[Dan Minor [:dminor]]

Attachment: Add manifest for cppunit tests

Try run here:

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=157e4830e09f

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 8515952 [details] [diff] [review]
Add manifest for cppunit tests

Review of attachment 8515952 [details] [diff] [review]:
-----------------------------------------------------------------

This is a good start, it definitely makes life simpler to be able to use our existing manifest stuff. Can you file a followup to figure out how to better integrate this into the build system? I don't know what that will look like, but having to duplicate the info between CPP_UNIT_TESTS and this manifest isn't great.

Then again, maybe our longer-term strategy should just be to kill all these C++ unit tests and make them gtests.

::: testing/mach_commands.py
@@ +267,2 @@
>          else:
> +            tests = cppunittests.extract_unittests_from_args(params['test_files'], mozinfo.info)

Could we push this down into the harness to simplify the mach command?

Comment 7

[Dan Minor [:dminor]]

Pushed to: https://hg.mozilla.org/integration/mozilla-inbound/rev/80a4be1149a9

Comment 8

[Dan Minor [:dminor]]

Filed bug 1106080 for improved integration into the build system.

Comment 9

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/80a4be1149a9