1669614 - todo_is assertion fails plain mochitest jobs when used inside SpecialPowers.spawn

Status: NEW

(Testing :: Mochitest, defect, P3)

Description

[Bianca Danforth [:bdanforth]]

Attachment: D90488_347977_test_basic_form_3pw_1.diff

STR:

1. Apply the attached diff to b3ea886b0ae7.
2. Run all password manager plain mochitests on any build of your choice in e10s on the Try server. E.g. ./mach try fuzzy toolkit/components/passwordmgr/test/mochitest/ and select the test-linux1804-64/opt-mochitest-plain-e10s-1 job.
3. Open the Treeherder link in Firefox and wait for the job to complete.

Actual results:
The Mochitest plain job fails.

Expected results:
The Mochitest plain job passes.
Expected fails (e.g. todo_is assertions), even if inside SpecialPowers.spawn, should not cause the job to fail.

More info:
While working on Bug 786276, I modified many tests to load test pages in a separate tab instead of in the mochitest page (which is loaded into an iframe).

When I moved the 10 todo_is assertions in test_basic_form_3pw_1.html inside a SpecialPowers.spawn task for this other window (See attached diff), these 10 expected fails got picked up by the Mochitest test harness and all Mochitest plain jobs on all builds and all platforms failed.

Attached also are the logs (roughly between SUITE-START and SUITE-END) for a representative plain Mochitest job that ran and failed on the Try server. Notice that despite all tests "passing", the Mochitest summary says there are "Failed: 10", which corresponds to the 10 todo_is assertions in test_basic_form_3pw_1.html:

It should be noted that I found these 10 expected failures by re-running the job with SimpleTest.requestCompleteLog(); at the beginning of the test.

Comment 1

[Bianca Danforth [:bdanforth]]

Attachment: plain_mochitest_job_logs.txt