927889 - Error when running a single b2g desktop mochitest with --test-path

Status: RESOLVED WONTFIX

(Testing :: Mochitest, defect)

Description

[Andrew Halberstadt [:ahal]]

When you point the --test-path parameter at a file the harness times out. The following can be seen in the logs:

JavaScript error: http://mochi.test:8888/tests/SimpleTest/SimpleTest.js, line 23: Permission denied to access property 'TestRunner'
JavaScript error: http://mochi.test:8888/tests/content/base/test/test_copyimage.html?autorun=1&closeWhenDone=1&logFile=%2Fhome%2Fahal%2Fhg%2Fmozilla-central%2Fobjdirs%2Fb2g_desktop%2F_tests%2Ftesting%2Fmochitest%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO, line 49: SimpleTest.waitForExplicitFinish is not a function

This works with desktop firefox. This also works if you specify a directory of tests instead of a file.

This doesn't necessarily block b2g desktop mochitests, but it makes triaging difficult. Reproducing a specific test failure is a lot harder, and therefore it will be harder to get people to look into them.

Comment 1

[Andrew Halberstadt [:ahal]]

Also works for mochitest-remote

Comment 2

[Andrew Halberstadt [:ahal]]

Source of error: http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/SimpleTest.js#23

When multiple mochitests run at once, they are run within an iframe, but the SimpleTest runner lives within the parent window. When a single mochitest is run, the test and the runner both live within the main window. Something about b2g desktop is different, where trying to call parent.TestRunner (as seen in the link above) doesn't work in the single test case.

Comment 3

[Andrew Halberstadt [:ahal]]

Hey Fabrice, can you think of a reason the line in comment 2 might cause the error in comment 0 on b2g desktop but not desktop firefox or b2g emulators? Or who should I contact?

This is making it difficult for developers to debug b2g desktop mochitest failures. Thanks!

Comment 4

[[:fabrice] Fabrice Desré]

I could see a difference between b2g desktop and emulators (that are OOP) but b2g desktop and desktop firefox should be similar.

One special thing though is that <iframe mozbrowser> element have special behavior to make them appear like top level windows even if they are wrapped in another page.

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

I've been looking into this because I didn't realize that IndexedDB tests are not expected to pass under b2g-desktop and have been disabled until I got in too deep.

The iframe doesn't know that it's a browser/app.  So all of top/parent/frameElement return the actual owning 

Here are the relevant gdb snippets for the docshell of the iframe that runs the single test:

mFrameType = nsDocShell::eFrameTypeRegular,
mOwnOrContainingAppId = 4294967295, // UNKNOWN_APP_ID


It appears that (in the b2g-desktop case I am testing) this is happening in nsFrameLoader because OwnerIsBrowserFrame returns false because nsGenericHTMLFrameElement::GetReallyIsBrowserOrApp returns false because "dom.mozBrowserFramesEnabled" is set to false for non-OOP builds.

The pref comes from (somehow; haven't found files name-checking it yet, but MDN says so):
http://mxr.mozilla.org/mozilla-central/source/testing/profiles/prefs_b2g_unittest.js

The angry check is:
http://mxr.mozilla.org/mozilla-central/source/content/html/content/src/nsGenericHTMLFrameElement.cpp#336


This also causes nsGenericHTMLFrameElement::GetReallyIsApp to also fail because it depends on GetAppManifestURL which says "At the moment, you can't be an app without being a browser" and fast-paths out:
http://mxr.mozilla.org/mozilla-central/source/content/html/content/src/nsGenericHTMLFrameElement.cpp#389


So I'm thinking we should set that pref to be true always for b2g mochitests.  I'll try that now, but am posting this comment now in case I don't make it.

Comment 6

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Andrew Sutherland (:asuth) from comment #5)
> The iframe doesn't know that it's a browser/app.  So all of
> top/parent/frameElement return the actual owning 

I meant to go on to say it returns the actual owning window.  And then it fails all the relevant security checks because one is an app origin and one is the mochi.test:8888 origin, so of course that's not going to work.

Comment 7

[Andrew Sutherland [:asuth] (he/him)]

Note that the prefs file also sets "dom.mozBrowserFramesWhitelist", but that has been irrelevant since bug 781355 was fixed in August 2012.

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

Okay, I made it work for me.  for b2g-desktop style mochitests, I changed the prefs to be:
// Needed for things to work at all!
user_pref("dom.mozBrowserFramesEnabled", true);
// this was false, but it should only be false for MOZ_GONK; linux desktop is not gonk
user_pref("dom.ipc.tabs.disabled", true);

And then I modified GAIA/test_apps/test-container/index.html to not set the "remote" attribute.  (I also closed the iframe tag, but the HTML parser was doing that for free anyways.)  Things did in fact-seem broken when we remoted, which I understand we still do not want to do on linux desktop quite yet.


Once I do this, I can see my test run on its own.  Also, it's an indexedDB test and it works, suggesting fixing this also may fix large swathes of bug 931116 too.

Comment 9

[Andrew Halberstadt [:ahal]]

Awesome thanks! Setting remote=False on the test-container app will impact emulator mochitests, but we should be able to do it at runtime depending on whether we expect to run oop or not. I'll see if I can reproduce this locally and get a patch up.

Comment 10

[Andrew Halberstadt [:ahal]]

Attachment: Patch 1.0 - fix prefs for b2g desktop mochitests

Try run: https://tbpl.mozilla.org/?tree=Try&rev=0f2470ddef6a

Comment 11

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 8333889 [details] [diff] [review]
Patch 1.0 - fix prefs for b2g desktop mochitests

Looks right to me and works for me locally.  I re-triggered the linux x64 try build twice to help determine what are new tests that should probably be black-listed versus what is just random.  It looks like there are various XBL failures, which is not totally surprising since XUL is not allowed in un-whitelisted content space and there may be checks that forbid XUL/XBL from ever existing in a non-chrome controlled context.

Comment 12

[Andrew Halberstadt [:ahal]]

So I removed the dom.mozBrowserFramesWhitelist pref as per comment 7, are you saying that's what is causing these failures? If so are you suggesting I leave the pref in, or disable the tests because they are irrelevant?

Comment 13

[Andrew Halberstadt [:ahal]]

Quick try run with the whitelist pref added back in: https://tbpl.mozilla.org/?tree=Try&rev=5d56f3120851

Comment 14

[Andrew Halberstadt [:ahal]]

So fabrice implied we do care about the xbl tests on b2g to some degree and it would be better if we could avoid disabling them. On the other hand comment 8 sounds like this patch fixes a flaw in how we are running b2g desktop mochitests which could potentially benefit others as well. Not sure if I should land this asap and disable xbl tests or spend some time trying to get both to work.

Andrew, any thoughts? How could I whitelist the content space the xbl tests are running in?

Comment 15

[Andrew Halberstadt [:ahal]]

Actually I was under the impression that mochitest server locations were already whitelisted:
http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/mozprofile/permissions.py#255

Comment 16

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Andrew Halberstadt [:ahal] from comment #15)
> Actually I was under the impression that mochitest server locations were
> already whitelisted:
> http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/
> mozprofile/permissions.py#255

I'll take a look since I really want those IndexedDB tests running.  Should be easier now that we can run single tests one at a time! ;)  ETA sometime tomorrow or friday; have some FxOS 1.3 features that want to land ASAP.

Comment 17

[Andrew Halberstadt [:ahal]]

Ok, let me know if you need help with mozprofile related things. In reftest we do: http://mxr.mozilla.org/mozilla-central/source/layout/tools/reftest/b2g_start_script.js#33 , so maybe a quick hacky fix is to add the same thing to http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/b2g_start_script.js

Though I'm pretty sure the code in comment 15 should be doing something equivalent.

Comment 18

[Yeuk Hon Wong [:yeukhon]]

I am having trouble with this as well. 

For example, mach mochitest-b2g-desktop ~/mozilla-central/content/base/test/csp/test_CSP.html

Can we confirm this is still the case for you guys? I tried to apply the patch but won't work for me.

Comment 19

[Andrew Halberstadt [:ahal]]

It's still a problem. I think where it left off was we had a patch that fixed it, but it caused all xbl tests to fail in the process. Are you saying the patch didn't work? Or didn't apply?

As a workaround you can modify testing/mochitest/b2g_desktop.json to only contain your test in the 'runtests' section.

Comment 20

[[ex-Mozilla] Agi Sferro | :agi]

I too am having this problem. Andrew, are you still working on this? I could try to look at it.

Comment 21

[Andrew Sutherland [:asuth] (he/him)]

I think this bug was mainly waiting on me to do some more investigation related to the XBL failures that showed up; I see from :ahal's display name that he's on PTO for a while, so I think it's fine to take over this bug.

Comment 22

[[ex-Mozilla] Agi Sferro | :agi]

OK. I'm taking it for know then. I'll see if I can figure something out, Andrew H. Feel free to re-grab it if you want to work on this. Thank you!

Comment 23

[Andrew Halberstadt [:ahal]]

Yes feel free to take it. Sorry, I should have unassigned myself, I haven't had time to look into this for quite some time now. Let me know if you need a hand with anything though.

Comment 25

[Garrett Robinson [:grobinson]]

FWIW, I had to do a clobber build to get this to take effect on m-c.

Comment 26

[[ex-Mozilla] Agi Sferro | :agi]

I'm sorry I'm not going anywhere with this :(

Comment 27

[Bobby Holley (:bholley)]

My workaround here is just to remove all the tests but the one I care about from mochitest.ini in a given directory.

Can't we just make the harness do that? That is to say, continue to run the test in an iframe, but just make the test list contain only one item?

Comment 28

[Andrew Halberstadt [:ahal]]

Yeah, we could. Originally I wanted to fix the underlying issue in case it meant we were doing something wrong in automation too, but this has been open long enough that I don't really care anymore :)

Also b2g desktop isn't the highest priority at the moment with Mulet almost being ready to replace it.

Comment 29

[Jonathan Griffin (:jgriffin)]

We're going to be using the same runner with Mulet, though, so we'll need to address this either way.

Comment 30

[(not currently active) Ted Mielczarek]

(In reply to Bobby Holley (:bholley) from comment #27)
> Can't we just make the harness do that? That is to say, continue to run the
> test in an iframe, but just make the test list contain only one item?

Gijs has a patch for that in bug 1014062.

Comment 32

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #29)
> We're going to be using the same runner with Mulet, though, so we'll need to
> address this either way.

I just built mulet and was able to run a single mochitest fine. This bug was something to do with b2g desktop's internal architecture, which I guess doesn't apply to mulet.

Resolving WONTFIX as b2g desktop mochitests aren't run anywhere anymore.