1465457 - Add ability to optionally load XUL reftests in a privileged context

Status: RESOLVED FIXED

(Testing :: Reftest, enhancement)

Description

[:Paolo Amadini]

Running the XUL reftests in a privileged context allows us to reduce the number of environments that we have to support for XBL bindings, especially once we convert them to Custom Elements. The infrastructure we are building for loading Custom Elements is only supported in privileged contexts at the moment.

Loading the tests from "chrome:" URLs instead of "file:" URLs will make them privileged. This will also help with bug 1460732 by removing one case where we require loading remote XUL from "file:" URLs.

Comment 1

[:Paolo Amadini]

Attachment: Bug 1465457 - Part 1 - Add a function to register chrome URL overrides in test harnesses.

Review commit: https://reviewboard.mozilla.org/r/247916/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/247916/

Comment 2

[:Paolo Amadini]

Attachment: Bug 1465457 - Part 2 - Load XUL reftests in a privileged context.

Review commit: https://reviewboard.mozilla.org/r/247918/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/247918/

Comment 3

[Brian Grinstead [:bgrins]]

Just to make sure I'm understanding the issue at https://bugzilla.mozilla.org/show_bug.cgi?id=1457719#c21:

Would loosening the restriction at https://searchfox.org/mozilla-central/rev/5a744713370ec47969595e369fd5125f123e6d24/toolkit/components/processsingleton/MainProcessSingleton.js#85-89 to also load the custom elements for XUL file URIs when dom.allow_XUL_XBL_for_file is true get us back into the same state we are currently in? Where the widget renders, but incorrectly, in unpriv contexts.

Comment 4

[:Paolo Amadini]

(In reply to Brian Grinstead [:bgrins] from comment #3)
> Would loosening the restriction at
> https://searchfox.org/mozilla-central/rev/
> 5a744713370ec47969595e369fd5125f123e6d24/toolkit/components/processsingleton/
> MainProcessSingleton.js#85-89 to also load the custom elements for XUL file
> URIs when dom.allow_XUL_XBL_for_file is true get us back into the same state
> we are currently in? Where the widget renders, but incorrectly, in unpriv
> contexts.

For Custom Elements, if we want to use anything like Services.scriptloader for lazy loading, we can only execute the "customElements.js" file in a privileged context, so this wouldn't effectively change anything. We may consider having a subset of Custom Elements that are also able to run unprivileged, which I guess could be the equivalent of loading them with a <script> tag, but I don't know if it is worth the effort if we don't use unprivileged XUL and plan to deprecate it.

Comment 5

[:Paolo Amadini]

With regard to the work-in-progress patches in this bug, this is one of several possible approaches to loading the files from "chrome:" URIs. This involves registering an override for each test file. The patch is incomplete because we would also need a way to specify additional dependencies for each test, for example an external CSS file.

Also, on Android the tests are loaded using HTTP. I don't know how well the "chrome" to "http" mapping would work, but if it works it's relatively simple to update the patch to support it.

The advantage of the approach above is that we don't need to change the build system or the test runner. Another approach involves adding a ".manifest" file to each of the "reftest" source directories, similarly to what in-tree extensions do. We would then register them at runtime and handle the mapping in the test suite. This works when tests are run locally, but I'm not sure about the Android case of loading over HTTP, which might make this whole approach unfeasible.

A third way involves making the way the tests are loaded more similar to other test suites like mochitest. The build system would take care of copying the files to a central location where they are then loaded from. This might involve changes to both the build system and the test runner, and again this may require listing the external dependent files.

The latter solution is definitely more laborious, and it's probably not something the XBL removal team could do, unless we find out it's simpler than it seems. Implementing it might reduce the number of special cases we have to handle in the build system for Desktop and Android. Greg, do you know to whom we can talk to help evaluate the cost of each of the solutions above, or suggest new approaches?

Comment 6

[:Paolo Amadini]

There is an assertion failure on debug builds when running the test in the content process, because IsOverlayAllowed is true for "chrome:" URLs and this requires accessing the chrome registry:

https://treeherder.mozilla.org/logviewer.html#?job_id=181085756&repo=try&lineNumber=17153

I guess this means we simply shouldn't run XUL reftests in the content process.

Anyways, if fixing this bug turns out to be required to unblock bug 1457719, I'll probably end up using the solution in the attached patches, and add a manifest directive to set an individual XUL test to run as privileged.

Comment 7

[Gregory Szorc [:gps]]

Those of us who hack on the test harnesses and the build system have long wanted to standardize on a common manifest format for declaring tests so the overall system is simpler.

I could probably dig up old bugs and email threads about this topic, but my recollection is essentially:

1) dbaron is the "owner" for the reftest harness and change proposals should go through him
2) dbaron and others have previously enumerated concerns with porting reftests to the ini-like manifest format
3) one such concern is around filtering; the reftest manifests rely on JS execution for filtering tests and the ini manifests aren't as expressive as JS expressions and porting the JS expressions to ini manifests programmatically would be difficult

I hope I summarized things accurately! TBH this isn't a problem I've thought about in a long time, so my recollection or understanding of the current world may be wrong!

I think you should reach out to the reftest harness maintainer (presumably dbaron) and ask for opinions.

Comment 8

[Brendan Dahl [:bdahl]]

The reftest overlay was removed awhile ago and IsOverlayAllowed was also removed. Maybe it's now possible to run the reftests in the content process?

Comment 9

[David Baron :dbaron:]

(In reply to Gregory Szorc [:gps] from comment #7)
> 3) one such concern is around filtering; the reftest manifests rely on JS
> execution for filtering tests and the ini manifests aren't as expressive as
> JS expressions and porting the JS expressions to ini manifests
> programmatically would be difficult

I think the concern around filtering wasn't so much expressiveness as rather the importance of conditions that are evaluated at runtime.  Many conditions around graphics acceleration, etc., are really determined on a per-machine basis rather than per-platform.  While there are many directories where we don't maintain these conditions accurately because developers don't run the tests locally, there are some reftest directories where developers do run the tests locally (because that's a much faster way to iterate) and the conditions are maintained for more than just the set of 5 or so configurations that we run in automation.

I think there's something else you wanted to ask me with this needinfo?... but it's not clear to me what it is, so could you ask it explicitly?

Comment 10

[:Paolo Amadini]

Yes, my original question in comment 5 was more related to the process of making the test files available on the machines, in other words the test running infrastructure.

The goal here is to run the XUL reftests in a privileged environment, because this is now the only supported case, and the only environment where we can use our lazy-loaded Custom Element scripts instead of XBL bindings. The steps to achieve this properly are unclear to me.

The proof-of-concept patch maps "chrome:" URLs to local test files, and this seems to work on Desktop, but for Android if I remember correctly this didn't work. On the other hand, we may not need to run XUL reftests on Android at all anymore. Is this correct?

There are also other approaches I mentioned in comment 5, and I'm trying to get a very high-level understanding of the costs and benefits of different solutions. In general, would it be better to try and patch the reftest infrastructure, or to migrate the XUL reftests to something more similar to a mochitest loader? In the latter case, in the future we may be able to extend this to all reftests and get rid of the old infrastructure. However, it seems to be more work, and I'm not sure our XBL removal team can do it. What's your take?

Comment 11

[David Baron :dbaron:]

(In reply to :Paolo Amadini from comment #10)
> Yes, my original question in comment 5 was more related to the process of
> making the test files available on the machines, in other words the test
> running infrastructure.
> 
> The goal here is to run the XUL reftests in a privileged environment,
> because this is now the only supported case, and the only environment where
> we can use our lazy-loaded Custom Element scripts instead of XBL bindings.
> The steps to achieve this properly are unclear to me.
> 
> The proof-of-concept patch maps "chrome:" URLs to local test files, and this
> seems to work on Desktop, but for Android if I remember correctly this
> didn't work. On the other hand, we may not need to run XUL reftests on
> Android at all anymore. Is this correct?

I'd be hesitant to stop running XUL reftests on Android without a reasonably formal and high-level decision that XUL is no longer supported on Android.  It's not clear to me whether that's a reasonable step at this point, and it might depend on what the relevant definition of "XUL" is.  For example, there are some relatively lower-level parts that are, I think, still used within HTML rendering in a number of places (e.g., for scrollbars, and I think still inside of some form controls).  However, these (other than scrollbars, at least... and scrollbars may be different on Android) may be separable from the higher level "set of XBL bindings" concept of XUL that maybe we don't support on Android, and maybe that's the part you need to be unsupported here.

> There are also other approaches I mentioned in comment 5, and I'm trying to
> get a very high-level understanding of the costs and benefits of different
> solutions. In general, would it be better to try and patch the reftest
> infrastructure, or to migrate the XUL reftests to something more similar to
> a mochitest loader? In the latter case, in the future we may be able to
> extend this to all reftests and get rid of the old infrastructure. However,
> it seems to be more work, and I'm not sure our XBL removal team can do it.
> What's your take?

So the reftest harness has a lot of features.  That said, we also have the ability to run reftests-in-mochitests, and we use that for a number of special cases (e.g., caret tests, CSS :visited tests) -- though the reftest-in-mochitest has much fewer capabilities.  It may be sufficient depending on how many of the capabilities we're using in the reftests that use XUL.

I think whether this latter approach is feasible depends on which interesting features of the reftest harness are used by XUL reftests.

Comment 12

[Brian Grinstead [:bgrins]]

Do you remember the source of Android failures over chrome://? That seems like it could be fixable, and sounds like there's not a general objection to loading over chrome at least for XUL as long as we can either accommodate Android or make a decision we aren't going to support it.

Comment 13

[:Paolo Amadini]

If I remember correctly, that was because on Android the tests are loaded using HTTP. I don't know if the files are actually local or remote, relative to the device that hosts the processes. In the latter case, they might be unavailable for "chrome:" mapping.

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

files are remote- the host machine calls over adb to talk to android and launch the browser- the host machine also sets up web servers to serve files.  It gets more complex on Android to server test files locally- we have storage and file transfer issues.  If we are talking about an addon, those are already copied to the device in the profile creation.

Comment 15

[:Paolo Amadini]

(In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #11)
> (In reply to :Paolo Amadini from comment #10)
> > On the other hand, we may not need to run XUL reftests on
> > Android at all anymore. Is this correct?
> 
> I'd be hesitant to stop running XUL reftests on Android without a reasonably
> formal and high-level decision that XUL is no longer supported on Android. 
> It's not clear to me whether that's a reasonable step at this point, and it
> might depend on what the relevant definition of "XUL" is.  For example,
> there are some relatively lower-level parts that are, I think, still used
> within HTML rendering in a number of places (e.g., for scrollbars, and I
> think still inside of some form controls).  However, these (other than
> scrollbars, at least... and scrollbars may be different on Android) may be
> separable from the higher level "set of XBL bindings" concept of XUL that
> maybe we don't support on Android, and maybe that's the part you need to be
> unsupported here.

Hm, so given comment 14 and the one above, there is maybe one thing we could do that doesn't involve changing how tests are loaded on Android. This would be to mark specific tests as privileged, and run them on Desktop only, from locally mapped "chrome:" URLs. We would do this for test related to chrome XBL bindings, that is a subset of XUL we already don't support on Android.

Comment 16

[Brian Grinstead [:bgrins]]

(In reply to :Paolo Amadini from comment #15)
> (In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #11)
> > (In reply to :Paolo Amadini from comment #10)
> > > On the other hand, we may not need to run XUL reftests on
> > > Android at all anymore. Is this correct?
> > 
> > I'd be hesitant to stop running XUL reftests on Android without a reasonably
> > formal and high-level decision that XUL is no longer supported on Android. 
> > It's not clear to me whether that's a reasonable step at this point, and it
> > might depend on what the relevant definition of "XUL" is.  For example,
> > there are some relatively lower-level parts that are, I think, still used
> > within HTML rendering in a number of places (e.g., for scrollbars, and I
> > think still inside of some form controls).  However, these (other than
> > scrollbars, at least... and scrollbars may be different on Android) may be
> > separable from the higher level "set of XBL bindings" concept of XUL that
> > maybe we don't support on Android, and maybe that's the part you need to be
> > unsupported here.
> 
> Hm, so given comment 14 and the one above, there is maybe one thing we could
> do that doesn't involve changing how tests are loaded on Android. This would
> be to mark specific tests as privileged, and run them on Desktop only, from
> locally mapped "chrome:" URLs. We would do this for test related to chrome
> XBL bindings, that is a subset of XUL we already don't support on Android.

Yeah, there's no reason to run reftests for stuff like <xul:checkbox> in Android. AFAIK the only XUL widget that will be loaded customElements.js that gets rendered in Android is the <browser> element (I'm not sure if there are reftests for that but we will see in Bug 1441935 if so).

There are cases as dbaron points out in Comment 11 for stuff like form controls, but those are all wired through either just NAC or UA Widgets which will work over http. I guess the trick will be making a list of tests that we don't care to run on Android, but given that we don't ever render this stuff in the real world it's probably worth doing anyway to save resources. Nick, does this sound reasonable?

Comment 17

[Nick Alexander :nalexander [he/him]]

(In reply to Brian Grinstead [:bgrins] from comment #16)
> (In reply to :Paolo Amadini from comment #15)
> > (In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #11)
> > > (In reply to :Paolo Amadini from comment #10)
> > > > On the other hand, we may not need to run XUL reftests on
> > > > Android at all anymore. Is this correct?
> > > 
> > > I'd be hesitant to stop running XUL reftests on Android without a reasonably
> > > formal and high-level decision that XUL is no longer supported on Android. 
> > > It's not clear to me whether that's a reasonable step at this point, and it
> > > might depend on what the relevant definition of "XUL" is.  For example,
> > > there are some relatively lower-level parts that are, I think, still used
> > > within HTML rendering in a number of places (e.g., for scrollbars, and I
> > > think still inside of some form controls).  However, these (other than
> > > scrollbars, at least... and scrollbars may be different on Android) may be
> > > separable from the higher level "set of XBL bindings" concept of XUL that
> > > maybe we don't support on Android, and maybe that's the part you need to be
> > > unsupported here.
> > 
> > Hm, so given comment 14 and the one above, there is maybe one thing we could
> > do that doesn't involve changing how tests are loaded on Android. This would
> > be to mark specific tests as privileged, and run them on Desktop only, from
> > locally mapped "chrome:" URLs. We would do this for test related to chrome
> > XBL bindings, that is a subset of XUL we already don't support on Android.
> 
> Yeah, there's no reason to run reftests for stuff like <xul:checkbox> in
> Android. AFAIK the only XUL widget that will be loaded customElements.js
> that gets rendered in Android is the <browser> element (I'm not sure if
> there are reftests for that but we will see in Bug 1441935 if so).
> 
> There are cases as dbaron points out in Comment 11 for stuff like form
> controls, but those are all wired through either just NAC or UA Widgets
> which will work over http. I guess the trick will be making a list of tests
> that we don't care to run on Android, but given that we don't ever render
> this stuff in the real world it's probably worth doing anyway to save
> resources. Nick, does this sound reasonable?

For not testing XUL bits on Android: yes, this seems reasonable.  I'll flag snorp to confirm, since he's one of the last folks standing who probably knows what XUL bits are relevant to GeckoView (and Fennec).

For distinguishing tests that we don't care to run on Android: this could be a good deal of effort.  I'll flag gbrown to talk about alternate approaches to the chrome:// issue.

On a related note, I once did something that sounds exactly like what you are trying to do for improving the edit-deploy-test cycle in Fennec: https://github.com/ncalexan/fennec-bootstrapper.  From the README:

"In addition, the add-on provides a custom bootstrapper:// protocol which allows to load remote resources as chrome resources. This means you can register a chrome override pointing to a resource that will be dynamically fetched. This is not usually possible since it opens a major security hole: remember that chrome resources are privileged."

That's exactly what you want, right: to load a resource over HTTP and pretend it's chrome://.  Maybe you could bake that into a test harness add-on and punt on the real solution?

Comment 18

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Nick Alexander :nalexander [he/him] from comment #17)
> (In reply to Brian Grinstead [:bgrins] from comment #16)
> > (In reply to :Paolo Amadini from comment #15)
> > > (In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #11)
> > > > (In reply to :Paolo Amadini from comment #10)
> > > > > On the other hand, we may not need to run XUL reftests on
> > > > > Android at all anymore. Is this correct?
> > > > 
> > > > I'd be hesitant to stop running XUL reftests on Android without a reasonably
> > > > formal and high-level decision that XUL is no longer supported on Android. 
> > > > It's not clear to me whether that's a reasonable step at this point, and it
> > > > might depend on what the relevant definition of "XUL" is.  For example,
> > > > there are some relatively lower-level parts that are, I think, still used
> > > > within HTML rendering in a number of places (e.g., for scrollbars, and I
> > > > think still inside of some form controls).  However, these (other than
> > > > scrollbars, at least... and scrollbars may be different on Android) may be
> > > > separable from the higher level "set of XBL bindings" concept of XUL that
> > > > maybe we don't support on Android, and maybe that's the part you need to be
> > > > unsupported here.
> > > 
> > > Hm, so given comment 14 and the one above, there is maybe one thing we could
> > > do that doesn't involve changing how tests are loaded on Android. This would
> > > be to mark specific tests as privileged, and run them on Desktop only, from
> > > locally mapped "chrome:" URLs. We would do this for test related to chrome
> > > XBL bindings, that is a subset of XUL we already don't support on Android.
> > 
> > Yeah, there's no reason to run reftests for stuff like <xul:checkbox> in
> > Android. AFAIK the only XUL widget that will be loaded customElements.js
> > that gets rendered in Android is the <browser> element (I'm not sure if
> > there are reftests for that but we will see in Bug 1441935 if so).
> > 
> > There are cases as dbaron points out in Comment 11 for stuff like form
> > controls, but those are all wired through either just NAC or UA Widgets
> > which will work over http. I guess the trick will be making a list of tests
> > that we don't care to run on Android, but given that we don't ever render
> > this stuff in the real world it's probably worth doing anyway to save
> > resources. Nick, does this sound reasonable?
> 
> For not testing XUL bits on Android: yes, this seems reasonable.  I'll flag
> snorp to confirm, since he's one of the last folks standing who probably
> knows what XUL bits are relevant to GeckoView (and Fennec).

Right, we really only use <browser>. Popups, prompts, etc, don't use XUL.

Comment 19

[Geoff Brown [:gbrown]]

(In reply to Nick Alexander :nalexander [he/him] from comment #17)
> For distinguishing tests that we don't care to run on Android: this could be
> a good deal of effort.  I'll flag gbrown to talk about alternate approaches
> to the chrome:// issue.

Thanks, but I can't think of anything to add to the discussion - sorry!

Comment 20

[Brian Grinstead [:bgrins]]

(In reply to Nick Alexander :nalexander [he/him] from comment #17)
> On a related note, I once did something that sounds exactly like what you
> are trying to do for improving the edit-deploy-test cycle in Fennec:
> https://github.com/ncalexan/fennec-bootstrapper.  From the README:
> 
> "In addition, the add-on provides a custom bootstrapper:// protocol which
> allows to load remote resources as chrome resources. This means you can
> register a chrome override pointing to a resource that will be dynamically
> fetched. This is not usually possible since it opens a major security hole:
> remember that chrome resources are privileged."
> 
> That's exactly what you want, right: to load a resource over HTTP and
> pretend it's chrome://.  Maybe you could bake that into a test harness
> add-on and punt on the real solution?

I think that's exactly what we want. Paolo, what do you think?

Comment 21

[:Paolo Amadini]

Yes, I also thought about a custom protocol handler, if we're not using the "chrome" scheme itself but just the protocol flags to determine what is available in relation to Custom Elements. This way, we could use this protocol for all the ".xul" reftests. I thought it would have been more work than adding the chrome annotation to the manifest, but if we have that prototyped already, it could work for our use case. Thanks, Nick!

One thing to be aware of though is that this may surface existing, potential intermittent failures due to the different pipeline of the load.

Comment 22

[Nick Alexander :nalexander [he/him]]

(In reply to :Paolo Amadini from comment #21)
> Yes, I also thought about a custom protocol handler, if we're not using the
> "chrome" scheme itself but just the protocol flags to determine what is
> available in relation to Custom Elements. This way, we could use this
> protocol for all the ".xul" reftests. I thought it would have been more work
> than adding the chrome annotation to the manifest, but if we have that
> prototyped already, it could work for our use case. Thanks, Nick!

"Prototyped" is a little strong: I built a bootstrapped addon that did this years ago, but it may have rotted or may have never been correct :)  See https://github.com/ncalexan/fennec-bootstrapper/blob/1824fc47db5783e55a95b15511d0775df7460bf6/extension/bootstrap.js#L28-L74.  The real magic of the addon is getting than in place really early in the startup process, IIRC.

Comment 23

[Brian Grinstead [:bgrins]]

I think probably the most correct thing to do here is finding reftests that are specifically testing XUL widgets that aren't relevant to android and flagging them as such, which would:

1) Skip them on Android
2) Turn on "chrome" mode

That way we could save resources and intermittents and also avoid automatically enabling chrome for _all_ reftests (even HTML stuff that really should be tested with content priv, I guess).

It may be a pain to do that, but we can find them as we go with binding conversions. The checkbox bug found the test in this folder: https://searchfox.org/mozilla-central/source/toolkit/themes/osx/reftests, and I suspect that entire folder should be flagged. Of course in this particular case they would already be skipped on Android but we will want to have (2) happen.

Comment 24

[Brian Grinstead [:bgrins]]

(In reply to :Paolo Amadini from comment #6)
> There is an assertion failure on debug builds when running the test in the
> content process, because ...

We will want these chrome reftests to run in the parent process (so that MainProcessSingleton picks them up and loads CEs). I think this can be done by setting g.browserIsRemote to false: https://searchfox.org/mozilla-central/rev/4e094f66ced333d69b24cd49273789e3a1173dfc/layout/tools/reftest/reftest.jsm#185, although I don't know if we know which URL is going to be loaded at that point.

Comment 25

[Brian Grinstead [:bgrins]]

If we want to avoid running all reftests in chrome (Comment 23) then I think we'll either need two different browser nodes (instead of the single g.browser) and choose which one to load the file in based on the URI, or deal with process flips on the g.browser node (to handle cases where we are loading some of the reftests in parent and some in content).

Comment 26

[Brian Grinstead [:bgrins]]

Attachment: Bug 1465457 - Part 2 - Load documents over chrome URIs in reftest-chrome

MozReview-Commit-ID: IgLcidpoDfq

Comment 27

[Brian Grinstead [:bgrins]]

Attachment: Bug 1465457 - Part 1 - Add a new reftest-chrome suite for reftests

MozReview-Commit-ID: 92yRtpn7k2g

Comment 28

[Brian Grinstead [:bgrins]]

Pushed up Paolo's MozReview patches to phab, and added the change to make them always load in parent process temporarily. I've confirmed that with these applied, the checkbox CE reftest from Bug 1455433 is passing.

Comment 29

[Tim Guan-tin Chien [:timdream] (please needinfo)]

I am worried that if we don't address this first, many of the tests will be in a false positive state as we migrate more bindings to custom elements.

In bug 1518932, I am forced to manually generate "anonymous nodes" in toolkit/themes/osx/reftests/nostretch.xul and its reference in order to make the test pass.

Comment 30

[Brian Grinstead [:bgrins]]

(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #29)

I am worried that if we don't address this first, many of the tests will be in a false positive state as we migrate more bindings to custom elements.

In bug 1518932, I am forced to manually generate "anonymous nodes" in toolkit/themes/osx/reftests/nostretch.xul and its reference in order to make the test pass.

Yeah, this is going to block a lot of the remaining bindings and we should take it ASAP. There's a bit of discussion going on in the phab review about how to make this happen. Should we make this block bug 1518932 so you don't need to manually generate the nodes?

Comment 31

[Tim Guan-tin Chien [:timdream] (please needinfo)]

(In reply to Brian Grinstead [:bgrins] from comment #30)

Yeah, this is going to block a lot of the remaining bindings and we should take it ASAP. There's a bit of discussion going on in the phab review about how to make this happen. Should we make this block bug 1518932 so you don't need to manually generate the nodes?

It's really depends on the timeline of this bug. I am hoping to land bug 1518932 this month so I am going for the //XXX generate "anonymous" nodes manually route for now.

Comment 32

[Brian Grinstead [:bgrins]]

Attachment: Bug 1465457 - Add support for chrome reftests

Comment 33

[Brian Grinstead [:bgrins]]

Attachment: Bug 1465457 - Load osx theme reftests over chrome

Comment 34

[Brian Grinstead [:bgrins]]

Attachment: Bug 1465457 - Remove assertion in nsChromeRegistryContent::IsLocaleRTL requiring "global" package

We are running certain xul reftests in the content process registered in chrome://reftest,
and it's OK for them to call this function (which is triggered at end of XUL doc loading).

Comment 35

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ec6c7dcf2f31
Add support for chrome reftests r=dbaron
https://hg.mozilla.org/integration/autoland/rev/ed020882a318
Load osx theme reftests over chrome r=dbaron
https://hg.mozilla.org/integration/autoland/rev/d7779ef1fa2f
Remove assertion in nsChromeRegistryContent::IsLocaleRTL requiring "global" package r=NeilDeakin

Comment 36

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/ec6c7dcf2f31
https://hg.mozilla.org/mozilla-central/rev/ed020882a318
https://hg.mozilla.org/mozilla-central/rev/d7779ef1fa2f

Comment 37

[Brian Grinstead [:bgrins]]

Updated documentation at https://developer.mozilla.org/en-US/docs/Mozilla/Creating_reftest-based_unit_tests#Reftests_and_elevated_privileges to reflect this as an option.