There's a big #ifdef in layout/base/tests/Makefile.in for tests that aren't supposed to run on Windows. It looks like a significant number of tests have been added to this ifdef even when they should have been run on all platforms.
I'm not confident enough in my guess at which tests aren't supposed to be there to just do it right now given how much time I've got, but this needs (1) to get fixed and (2) better comments in the makefile (probably at the top, bottom, and between each bug number for the cases people are sorting) to reduce the chance of this happening.
The tests I added to that list should absolutely run on all platforms. Sorry about that. :(
I think this should probably look like:
# BUG AAA should not be run on Windows because ...
TEST_PATH += Tests for Bug AAA only \
# BUG BBB should not be run on Windows because ...
TEST_PATH += Tests for Bug BBB only \
# End of tests that should not be run on Windows
All my tests in the list use WindowSnapshot, so are at least prima facie likely to have issues on Windows. I don't remember if I tested them outside the list or not.
(In reply to Simon Montagu from comment #4)
> All my tests in the list use WindowSnapshot, so are at least prima facie
> likely to have issues on Windows. I don't remember if I tested them outside
> the list or not.
Yeah, I came to the same conclusion yesterday, and the patch I wrote yesterday and pushed to try moves all of bzbarsky's and smaug's tests and none of yours:
Unfortunately the tests I wanted didn't actually run...
It doesn't seem like I can actually get try runs on Windows (bug 700168 / bug 697802), so I used inbound as try:
I diffed one pair of mochitest logs (just the TEST-START lines) for each of Linux, Mac, and Windows on each side of the mozilla-inbound landing to check that the results were as expected: no changes on Linux and Mac, and 4 new tests showing up on Windows.