cpppunittests can be skipped/fail silently
Categories
(Testing :: CPPUnitTest, defect, P5)
Tracking
(firefox79 fixed)
Tracking | Status | |
---|---|---|
firefox79 | --- | fixed |
People
(Reporter: aryx, Assigned: gbrown)
References
Details
(Whiteboard: dev-prod-2020)
Attachments
(1 file)
Bug 1573051 accidentally removed a skip-if = os != 'win'
rule for a Cpp test (moved it from TestNativeNt to TestUriValidation): https://hg.mozilla.org/mozilla-central/rev/4eca0f08c43b#l1.11
This got fixed in bug 1575655. In between, TestNativeNt should have run e.g. on Linux but it cannot be found in the log:
Formatted log: https://treeherder.mozilla.org/logviewer.html#?job_id=262411979&repo=mozilla-central
Plain text log: https://queue.taskcluster.net/v1/task/ZJTeZO8UR_KnaggrhWrugA/runs/0/artifacts/public/logs/live_backing.log
If the test compilation or execution fails, the task should report this as an error.
Comment 1•5 years ago
|
||
https://searchfox.org/mozilla-central/source/mozglue/tests/moz.build#26 determines if TestNativeNt should be built independently of testing/cppunittest.ini. Compilation failures would have appeared in the Build log not the test log.
https://searchfox.org/mozilla-central/source/testing/runcppunittests.py#266 silently drops tests specified in the manifest which do not exist. I suppose we could log these as warnings but I don't know how much noise there would be if we did so. As it stands, the manifest's exclusion of tests for platforms where the test is not built has no effect.
Assignee | ||
Updated•4 years ago
|
Assignee | ||
Comment 2•4 years ago
|
||
I suppose we could log these as warnings but I don't know how much noise there would be if we did so.
Let's find out!
https://treeherder.mozilla.org/#/jobs?repo=try&revision=86cf4569fdaf1d0ab40b998a78058286a9c4155c
All platforms have warnings! For linux/opt I see:
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/TestAppShellSteadyState - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/TestParseFTPList - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/TestPLDHash - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/TestSyncRunnable - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/TestVolatileBuffer - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/buffered_stun_socket_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/ice_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/test_nr_socket_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/multi_tcp_socket_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/nrappkit_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/rlogringbuffer_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/runnable_utils_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/sctp_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/signaling_unittests - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/signaling_unittests_standalone - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/simpletokenbucket_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/sockettransportservice_unittest - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/transport_unittests - skipped
[task 2020-06-01T16:23:33.771Z] 16:23:33 INFO - test file not found: /builds/worker/workspace/build/tests/cppunittest/turn_unittest - skipped
Assignee | ||
Comment 3•4 years ago
|
||
Most of these were switched to gtest; see 1239870, for instance. The remaining tests are:
TestAppShellSteadyState
signaling_unittests
signaling_unittests_standalone
Assignee | ||
Comment 4•4 years ago
|
||
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/94a6c07bd0d6 Warn when cppunit test file missing, and cleanup cppunit manifest; r=bc
Comment 6•4 years ago
|
||
bugherder |
Description
•