Closed Bug 1040056 Opened 7 years ago Closed 7 years ago
SUITE-START log line is incorrect
The gaiatests run with confusing messages like: 04:38:01 SUITE-START | Running 0 tests 04:38:01 TEST-START | test_camera_capture_video.py TestCamera.test_capture_a_video 04:38:47 TEST-END PASS | test_camera_capture_video.py TestCamera.test_capture_a_video | took 45845ms 04:38:47 SUITE-END | took 45s 04:38:48 SUITE-START | Running 0 tests 04:38:48 TEST-START | test_camera_multiple_shots.py TestCameraMultipleShots.test_capture_multiple_shots 04:39:33 TEST-END PASS | test_camera_multiple_shots.py TestCameraMultipleShots.test_capture_multiple_shots | took 45160ms 04:39:33 SUITE-END | took 45s from https://jenkins1.qa.scl3.mozilla.com/job/flame.mozilla-central.ui.adhoc/2/console Where "suites" claim they run 0 tests, but then the tests actually get run. This should be fixed in the output so we won't have confusing logs. Chris, can you take this one?
I looked into this briefly and the first thing that stood out was that the device test is using moztest 0.5 from pypi whereas the in-tree version is 0.6.
Could be a dupe of bug 1020625. We might just need a moztest version bump to fix.
Th problem is that we don't know the tests upfront, so we pass a zero-length list to suite_start. The formatter interprets that as 0 tests. The two possible fixes are to make the formatter deal with the empty list case more gracefully, and to figure out the list of tests before they are run (I don't know if this is possible in unittest).
I suspect davehunt is exactly right in comment 2, but I'll take a closer look at this today.
Assignee: nobody → cmanchester
Ok, one of my blunders here was not uploading a new moztest to pypi with bug 1020625, I've done that just now.
You should make marionette_client require moztest>=0.6 too.
When I run the gaia ui tests locally with moztest 0.6 and marionette 0.8, the output looks ok (no extra suite_start messages).
Attachment #8458290 - Flags: review?(mdas)
Comment on attachment 8458290 [details] [diff] [review] Increment version of moztest required by marionette to include structured logging modification. Review of attachment 8458290 [details] [diff] [review]: ----------------------------------------------------------------- lgtm, thanks!
Attachment #8458290 - Flags: review?(mdas) → review+
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla34
You need to log in before you can comment on or make changes to this bug.