Closed Bug 1201963 Opened 4 years ago Closed 9 months ago

The test suite should print something useful when you try to run something that is not a test

Categories

(Testing :: web-platform-tests, defect)

defect
Not set

Tracking

(firefox66 fixed)

RESOLVED FIXED
mozilla66
Tracking Status
firefox66 --- fixed

People

(Reporter: ehsan, Assigned: KWierso)

Details

Attachments

(1 file)

Currently this is what gets printed which is not quite helpful.

$ ./mach web-platform-tests testing/web-platform/mozilla/tests/service-workers/service-worker/resources/fetch-canvas-tainting-iframe.html
 0:00.48 LOG: MainThread INFO Using 1 client processes
 0:00.52 LOG: MainThread wptserve INFO Starting http server on 127.0.0.1:8001
 0:00.52 LOG: MainThread wptserve INFO Starting http server on 127.0.0.1:8000
 0:00.52 LOG: MainThread wptserve INFO Starting http server on 127.0.0.1:8443
 0:02.51 SUITE_START: MainThread 0
 0:02.51 LOG: MainThread INFO Running testharness tests
 0:02.52 LOG: MainThread INFO No testharness tests to run
 0:02.52 LOG: MainThread INFO Running reftest tests
 0:02.52 LOG: MainThread INFO No reftest tests to run
 0:02.52 LOG: MainThread INFO Got 0 unexpected results
 0:02.52 SUITE_END: MainThread
Summary
=======

Ran 0 tests
Expected results: 0
Unexpected results: 0

OK
 0:02.53 LOG: MainThread INFO Closing logging queue
 0:02.53 LOG: MainThread INFO queue closed
Three years later, this is what prints when you run something that's not a test:


$ ./mach wpt testing/web-platform/tests/not-a-test
 0:00.00 INFO Skipping manifest download because existing file is recent
 0:06.20 INFO Using 1 client processes
 0:09.45 INFO STDERR: WARNING:root:Unable to install font: Ahem.ttf
 0:09.50 INFO Starting http server on 127.0.0.1:8000
 0:09.50 INFO Starting http server on 127.0.0.1:8001
 0:09.53 INFO Starting https server on 127.0.0.1:8443
 0:10.11 SUITE_START: web-platform-test - running 0 tests
 0:10.11 INFO Running reftest tests
 0:10.11 INFO No reftest tests to run
 0:10.11 INFO Running wdspec tests
 0:10.11 INFO No wdspec tests to run
 0:10.11 INFO Running testharness tests
 0:10.11 INFO No testharness tests to run
 0:10.11 INFO Got 0 unexpected results
 0:10.11 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 0 checks ()
Expected results: 0
OK
 0:10.14 INFO Closing logging queue
 0:10.14 INFO queue closed
 0:10.14 ERROR No tests ran


So, it's better. Enough to call this fixed?
Flags: needinfo?(ehsan)
Well, compare with:

$ ./mach mochitest am/i/seeing/whats/not/there
The mochitest command could not find any mochitests under the following
test path(s):

am/i/seeing/whats/not/there

Please check spelling and make sure there are mochitests living there.

I guess we can close the bug because of statute of limitations, but I wouldn't really call this output three years after reporting this a lot more informative in telling the developer, hey my friend, you just made a typo.  :-)

If you think this bug will continue sitting here without receiving any attention, then fine let's close it I guess.
Flags: needinfo?(ehsan)
Assignee: nobody → wkocher
With this, the output is this:

wkocher-vffvh8:mozilla-central wkocher$ ./mach wpt testing/not/a/test some/other/path
 0:00.00 INFO Skipping manifest download because existing file is recent
 0:05.49 ERROR Unable to find any tests at the path(s):
 0:05.49 ERROR   testing/not/a/test
 0:05.49 ERROR   some/other/path
 0:05.49 ERROR Please check spelling and make sure there are tests in the specified path(s).
 0:05.49 INFO Closing logging queue
 0:05.49 INFO queue closed
 0:05.50 ERROR No tests ran




Word-smithing welcome :)
Attachment #9033208 - Flags: review?(james)
Beautiful, thank you!
Attachment #9033208 - Flags: review?(james)
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/13d9e2a48dbb
Give a better warning when no tests can be found in the specified path r=jgraham
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/14700 for changes under testing/web-platform/tests
Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.
https://hg.mozilla.org/mozilla-central/rev/13d9e2a48dbb
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
Upstream PR merged
You need to log in before you can comment on or make changes to this bug.