602871 - XPCShell test harness should ignore system idle time while running tests

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Marco Bonardo [:mak]]

Currently some tinderboxes are reporting permanent idle (mac and windows for sure), this means that each xpcshell test will run idle and idle-daily tasks for each test.

This is making tests slower (think then in near future we will run databases vacuum at idle-daily) and more orange-prone.

We should find a way to reset system idle before each test, maybe synthesizing a keyboard event?

Comment 1

[Marco Bonardo [:mak]]

since this blocks my work, tentatively taking.

Comment 2

[(not currently active) Ted Mielczarek]

This is in fact sort of awful. Will this effect other test suites too?

Comment 3

[Marco Bonardo [:mak]]

(In reply to comment #2)
> This is in fact sort of awful. Will this effect other test suites too?
possibly, but mochitests and b-c tests are most likely already sythesizing events... Just I'm not sure if those synthesizing is enough to reset the counter, will have to check that.

Comment 4

[Marco Bonardo [:mak]]

Attachment: patch v1.0

If the approach is acceptable, this should work. It is replacing idle service with a mock by default, if a test needs the original service it can get it through let idleSvc = do_get_idle(); and the original service is restored.

Synthesizing eventis is most likely not going to work because I highly doubt they'll reach the OS, and we poll the OS for idle time.

Comment 5

[Marco Bonardo [:mak]]

And yes, we could need to do something similar for other harnesses, but any test harness that has a common profile for all tests is less affected (b-c and m-c?, not sure about the other ones) since idle-daily will be notified just once and most likely on a almost empty profile.

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 482385 [details] [diff] [review]
patch v1.0

Looks good with two comments:
1) Please update the MDC documentation to mention this.
2) You should probably add a simple test in testing/xpcshell/example/unit/ to check that do_get_idle works.

Comment 7

[Marco Bonardo [:mak]]

Attachment: patch v1.1

Added the requested test, Ted approved it on IRC

Comment 8

[Marco Bonardo [:mak]]

hm, there is a problem in restoring the original factory, looks like something is trying to QueryInterface idleService (the real one) to nsIFactory. To fix the bug I have to split the factory from the mock, unfortunately that makes the test useless.

Comment 9

[Marco Bonardo [:mak]]

Attachment: patch v1.2

This workarounds the issue by having a separate factory object, the test is still valid thanks to a forwarding in QI. The basic approach and effects have not changed and code changes are minimal, thus not reasking review.

Comment 10

[Marco Bonardo [:mak]]

http://hg.mozilla.org/mozilla-central/rev/6e4fbb231f1c

Comment 11

[Marco Bonardo [:mak]]

documented in https://developer.mozilla.org/en/Writing_xpcshell-based_unit_tests under do_get_idle()

Comment 12

[Rain [:rain1] (inactive)]

FWIW we had the same problem in comm-central and did the same thing (mock idle service) to fix it a while back. <http://hg.mozilla.org/comm-central/rev/960a02225e3d>

Comment 13

[Marco Bonardo [:mak]]

ah cool, I guess you could use the more general xpcshell-tests fix now?

Comment 14

[Rain [:rain1] (inactive)]

Yeah, we backed our fix out: <http://hg.mozilla.org/comm-central/rev/0c4e204d8575>