Closed Bug 986163 Opened 6 years ago Closed 6 years ago

Webapp Runtime tests don't work anymore

Categories

(Testing :: Mochitest, defect, P1)

All
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED
mozilla31

People

(Reporter: marco, Assigned: billm)

References

Details

(Keywords: regression)

Attachments

(1 file)

This is a really recent regression (it should be max 2-3 days).

I can't run both webapprt-test-chrome and webapprt-test-content.

TEST-UNEXPECTED-FAIL | (browser-test.js) | No tests to run. Did you pass an invalid --test-path?
It's probably my fault. Do we run these tests on tinderbox? How do I run them locally?
(In reply to Bill McCloskey (:billm) from comment #1)
> It's probably my fault. Do we run these tests on tinderbox? How do I run
> them locally?

Yes, after a quick look I strongly suspect bug 938019.

These tests aren't yet run on tinderbox.
You can run them via mach:
./mach webapprt-test-chrome
./mach webapprt-test-content

In your patch I noticed:
- The tests' prefixes are wrong (webapprt-test-chrome tests' prefix is "browser_", webapprt-test-content tests' prefix is "webapprt_")
- I think you've only taken webapprt-test-chrome into account.
OK, it's pretty easy to fix the webapprt-test-chrome tests. I'm having a harder time figuring out the content tests though. Do we have manifest files for those tests?

If we don't, it should be easy to disable bug 938019 for webapprt tests. But it would be nice to fix it properly.
Flags: needinfo?(ted)
Flags: needinfo?(gps)
Unfortunately I don't actually know anything about webapprt tests.
Flags: needinfo?(ted)
Attached patch webapprt-fixSplinter Review
Not sure who should review this. Ted, please pass it on to someone else if you like.

As far as I can tell, the webapprtChrome tests have their own manifests. The webapprtContent tests piggyback on the mochitest-plain manifests and use a different filename prefix. This complicates bug 983867, because we can no longer assume that any test in a manifest that doesn't pass the isTest condition (which just looks at the prefix) is a mistake. I think eventually we want webapprtContent tests to use separate manifests. For now, though, I think this patch moves us in the right direction: it gets everything working as far as I can tell, and the webapprt tests are using manifests--just not in quite the way we want.

Marco, can you see if this works for you? I'm getting failures in some of the tests, but I'm guessing that's not the fault of this patch.
Assignee: nobody → wmccloskey
Status: NEW → ASSIGNED
Attachment #8394479 - Flags: review?(ted)
Flags: needinfo?(gps)
(In reply to Bill McCloskey (:billm) from comment #5)
> As far as I can tell, the webapprtChrome tests have their own manifests. The
> webapprtContent tests piggyback on the mochitest-plain manifests and use a
> different filename prefix. This complicates bug 983867, because we can no
> longer assume that any test in a manifest that doesn't pass the isTest
> condition (which just looks at the prefix) is a mistake. I think eventually
> we want webapprtContent tests to use separate manifests. For now, though, I
> think this patch moves us in the right direction: it gets everything working
> as far as I can tell, and the webapprt tests are using manifests--just not
> in quite the way we want.
> 
> Marco, can you see if this works for you? I'm getting failures in some of
> the tests, but I'm guessing that's not the fault of this patch.

Yes, it works!

The webapprt-test-content takes more time to start (because it logs a lot of warnings like "Warning: test_plugin_scroll_invalidation.html from manifest /home/marco/Documents/FD/src/obj-x86_64-unknown-linux-gnu/_tests/testing/mochitest/tests/widget/tests/mochitest.ini is not a valid test"), but it isn't a problem.

Notice that in the future we plan to run all the mochitest plain tests as webapprtContent tests (bug 899704).
Blocks: 938019
Attachment #8394479 - Flags: review?(ted) → review+
https://hg.mozilla.org/mozilla-central/rev/ca4eb9580260
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla31
You need to log in before you can comment on or make changes to this bug.