1375654 - test_addonMayLoad.html fails when run alone, until you run entire directory

Status: RESOLVED FIXED

(Core :: Security: CAPS, defect)

Description

[Andrew McCreight [:mccr8]]

If I run "./mach build", then "./mach mochitest -f chrome caps/tests/mochitest/test_addonMayLoad.html", I get some test failures: 

2 INFO TEST-PASS | caps/tests/mochitest/test_addonMayLoad.html | cross-origin load should fail for addon A 
GECKO(13462) | [13462] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file /home/amccreight/mc/netwerk/base/nsChannelClassifier.cpp, line 304
3 INFO TEST-UNEXPECTED-FAIL | caps/tests/mochitest/test_addonMayLoad.html | whitelisted cross-origin load of test1 should succeed for addon A 
@chrome://mochitests/content/chrome/caps/tests/mochitest/test_addonMayLoad.html:71:5
promise callback*@chrome://mochitests/content/chrome/caps/tests/mochitest/test_addonMayLoad.html:66:3
GECKO(13462) | [13462] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file /home/amccreight/mc/netwerk/base/nsChannelClassifier.cpp, line 304
4 INFO TEST-PASS | caps/tests/mochitest/test_addonMayLoad.html | non-whitelisted cross-origin load of test1 should fail for addon B 
GECKO(13462) | [13462] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file /home/amccreight/mc/netwerk/base/nsChannelClassifier.cpp, line 304
5 INFO TEST-UNEXPECTED-FAIL | caps/tests/mochitest/test_addonMayLoad.html | whitelisted cross-origin load of test2 should succeed for addon B 
@chrome://mochitests/content/chrome/caps/tests/mochitest/test_addonMayLoad.html:77:5
promise callback*@chrome://mochitests/content/chrome/caps/tests/mochitest/test_addonMayLoad.html:66:3
GECKO(13462) | [13462] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file /home/amccreight/mc/netwerk/base/nsChannelClassifier.cpp, line 304
6 INFO TEST-PASS | caps/tests/mochitest/test_addonMayLoad.html | non-whitelisted cross-origin load of test2 should fail for addon A 

Note that the tests that pass are all saying that loading should fail, and the tests that fail are all saying that loading should succeed.

If I run this, everything passes:
  ./mach mochitest -f chrome caps/tests/mochitest/

...but wait, there's more!

Now, if I run "./mach mochitest -f chrome caps/tests/mochitest/test_addonMayLoad.html" again (the same command from before!), now it passes! And passes on subsequent runs. If you run "mach build" again, you get back to the initial state where running the test by itself fails.

Comment 1

[Kris Maglione [:kmag]]

That... is very odd. I can only reproduce it on debug builds.

And as far as I can tell, it doesn't actually have anything to do with the add-on policy code. If I change the first principal URL to "http://test1.example.org/", I get the same behavior. It fails when the test is run by itself, but passes after I run tests for the whole directory.

Ah. And now I see why. file_data.txt is only included in mochitest.ini, but not in chrome.ini, where that test is declared.

Comment 2

[Kris Maglione [:kmag]]

Attachment: Bug 1375654: Package file_data.txt for chrome mochitests.

Review commit: https://reviewboard.mozilla.org/r/151920/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/151920/

Comment 3

[Andrew McCreight [:mccr8]]

Comment on attachment 8880588 [details]
Bug 1375654: Package file_data.txt for chrome mochitests.

https://reviewboard.mozilla.org/r/151920/#review156922

Comment 4

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ca5b6c88e6696e66ed30fffe7ed96f44c1f51056
Bug 1375654: Package file_data.txt for chrome mochitests. r=mccr8

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ca5b6c88e669