Need a way to avoid running multiple test suites

NEW
Unassigned

Status

Testing
Mochitest
3 years ago
5 months ago

People

(Reporter: billm, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

It's pretty common that I want to run some subset of the tests in browser/base/content/test/general. Usually I select them using the --start-at and --end-at parameters, like so:

../mach mochitest -f browser browser/base/content/test/general --start-at=browser_pageInfo.js --end-at=browser_restore_isAppTab.js

The problem is that after everything finishes, another process starts up and runs browser_parsable_css.js. As far as I can tell, that's because that test is also referenced in devtools/client/framework/test/browser.ini, which is part of the devtools suite.

I really just want a way to avoid this. I don't know what sort of solution people would prefer through. Here are some ideas:

1. Never run multiple test suites for a single command line. I basically never want this to happen since the experience is terrible. It seems like tests are done and then everything starts up again and you're not sure why. It's very difficult to figure out which suite a particular "Passes: X" line is for. If there are multiple suites possible, maybe we could just pick a default and tell people what option to use to run the others.

2. At least allow some option to avoid running multiple suites. It seems like you can use --subsuite to select devtools tests, but there's no way to ask for tests without a subsuite.

3. Make it so that the devtools version of browser_parsable_css.js is not considered when looking for tests in browser/base/content/test/general. It seems like it really ought to be considered part of devtools/client/framework/test.

Comment 1

3 years ago
`mach mochitest -f <flavor>`?
Even running with -f browser still runs devtools tests.
Although adding another flavor that runs browser tests and not devtools tests would be another acceptable solution here.
(Assignee)

Updated

5 months ago
Component: BrowserTest → Mochitest
Product: Testing → Testing
You need to log in before you can comment on or make changes to this bug.