1087558 - Assert when a mochitest manifest entry is 'not a valid test'

Status: NEW

(Testing :: Mochitest, defect)

Description

[Kelsey Gilbert [:jgilbert]]

I had tests that were not running because the files started were like `test-foo.html` instead of `test_foo.html`, and the only message was a warning buried in the mochitest spew about them 'not being valid tests'.

It should assert.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

would you want this as a 'test-unexpected-fail' in the log file?  how would you like the notification to be shown?

Comment 2

[Kelsey Gilbert [:jgilbert]]

(In reply to Joel Maher (:jmaher) from comment #1)
> would you want this as a 'test-unexpected-fail' in the log file?  how would
> you like the notification to be shown?
No, this is not a test error, it's a manifest error. Just throw an assert from python.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

this failure comes from inside the browser (javascript) where the test definition is.  I would like to avoid duplicating the logic here, but I could duplicate it.  We have different rules for chrome, browser-chrome, plain.

Comment 4

[(not currently active) Ted Mielczarek]

Sadly at the moment I think we still have junk in our manifests (from when we auto-converted from Makefiles) that means we can't make this fatal yet.

Comment 5

[Kelsey Gilbert [:jgilbert]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #4)
> Sadly at the moment I think we still have junk in our manifests (from when
> we auto-converted from Makefiles) that means we can't make this fatal yet.

I'm willing do to the mechanical patches to fix this.

Comment 6

[(not currently active) Ted Mielczarek]

That's bug 983867. I am happy to review patches!

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

:vaibhav, there are a lot of dependencies here, this is additional manifest work you could easily help out on!

Comment 8

[Kelsey Gilbert [:jgilbert]]

I'd like to note that while it's easy to just remove these, since they're already not being run, a bunch of them look like valid tests. A module owner/peer should check that we can just remove them.