1632022 - test_nsIMsgFolderListenerIMAP.js fails when run by itself

Status: NEW

(Thunderbird :: Testing Infrastructure, defect)

Description

[Ben Campbell]

Running this unit test by itself:

$ ./mach xpcshell-test comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js

Fails with an error like this:

ERROR test_end for comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js logged while not in progress. Logged with data: {"status": "PASS", "test": "comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js", "message": "xpcshell return code: 0"}

I think this is because it is actually run twice - in parallel - and the two instances interact badly. In normal operation this isn't usually an issue, because the timing is such that the two instances don't start at exactly the same time. But when you run it by itself, they are.

A workaround is to run using the --sequential option, to prevent them from kicking off at the same time:

$ ./mach xpcshell-test --sequential comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js

The reason the test is run twice is that all the imap tests are run twice - there are two xpcshell.ini files. One for mbox, and one for maildir. This is good, but it'd be nice if we could guarantee simultaneous tests don't interact badly like in this case.

This could easily be the source of some intermittent failures in the IMAP tests (and perhaps elsewhere).

Comment 1

[gene smith]

Is this fixed? I now see this without --sequential:

$ ./mach xpcshell-test  comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js
:
 0:03.95 TEST_END: Test PASS. Subtests passed 85/85. Unexpected 0
 0:03.96 INFO Retrying tests that failed when run in parallel.
 0:03.97 TEST_START: comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js
:

I don't see the reported error:

ERROR test_end for comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js logged while not in progress. Logged with data: {"status": "PASS", "test": "comm/mailnews/imap/test/unit/test_nsIMsgFolderListenerIMAP.js", "message": "xpcshell return code: 0"}

I was just manually running tests and saw a similar error string and found this bug.

Comment 2

[Ben Campbell]

(In reply to gene smith from comment #1)

Is this fixed?

Not intentionally :-)
But yes, I just tried running random imap tests individually, and can no longer get it to fail quite so reliably, so it definitely seems to have gotten waaay better.

However, I did manage to get some failures from time to time. Some tests seem more stable than others. For example,
I didn't ever see comm/mailnews/imap/test/unit/test_imapUndo.js fail (when run individually), while comm/mailnews/imap/test/unit/test_offlineStoreLocking.js seemed to fail about 20% of the time. That seems higher than just background intermittent failure rate, so I suspect it's just got a lot better rather than gone away completely...