(In reply to Edwin Gao (:egao) from comment #8) > I don't mind looking at this, though I'd like a couple of use cases so I have something to test against, that would be nice. > > The way I understand the problem, is given a mochitest manifest file, `./mach mochitest <manifest_file>` is ignoring all the `skip-if` clauses. This doesn't cover the scenario when a specific test file is passed into the argument. > > Does the above sound correct? No, the problem is that when you run: ``` ./mach mochitest path/to/directory/with/tests ``` locally, the skip-ifs are honoured (as they should be). When you push to try, and use try-test-paths or pass a path to `./mach try fuzzy`, they're ignored. An easy test is pushing an artifact trypush with the `browser/components/customizableui/` path and a single mochitest-browser-chrome job on macos. This currently fails because the skip-if at https://searchfox.org/mozilla-central/rev/4606c7974a68cab416c038acaedcae49eed93822/browser/components/customizableui/test/browser.ini#40-42 is ignored, running a test that can never pass on mac. Does that help?
Bug 1541424 Comment 9 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Edwin Gao (:egao) from comment #8) > I don't mind looking at this, though I'd like a couple of use cases so I have something to test against, that would be nice. > > The way I understand the problem, is given a mochitest manifest file, `./mach mochitest <manifest_file>` is ignoring all the `skip-if` clauses. This doesn't cover the scenario when a specific test file is passed into the argument. > > Does the above sound correct? No, when you run: ``` ./mach mochitest path/to/directory/with/tests ``` locally, the skip-ifs are honoured (as they should be). When you push to try, and use try-test-paths or pass a path to `./mach try fuzzy`, they're ignored, and that's the problem. An easy test is pushing an artifact trypush with the `browser/components/customizableui/` path and a single mochitest-browser-chrome job on macos. This currently fails because the skip-if at https://searchfox.org/mozilla-central/rev/4606c7974a68cab416c038acaedcae49eed93822/browser/components/customizableui/test/browser.ini#40-42 is ignored, running a test that can never pass on mac. Does that help?