1027621 - Use local content where possible in the browser tests

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

If we're running on B2G desktop or have a device connected to the same network as the host via WiFi then we can serve local HTTP content to the tests. This will allow us to run the Browser tests that currently rely on external resources on TBPL.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20752

Requesting feedback for now, before applying to the remaining tests. Being connected to WiFi does not guarantee local content can be served to a device. We may instead want to somehow check if the device can communicate with the local server.

Comment 2

[Zac C (:zac)]

Comment on attachment 8442801 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20752

f+

I'm happy with the assumption that device and runner are on the same networks, I think it's a compromise we have to make to get better coverage (and it may happen again in another area of functionality). Perhaps you could pop something into the readme file, if anybody reads that!

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The patch uses the www content provided by marionette-client, however another approach we may want to use is to use http://hg.mozilla.org/qa/testcase-data/ by cloning that and specifying an alternate docroot using --server-root.

The advantage to this approach would be that the local/remote content remains consistent, and also gives us greater flexibility in adding new resources without having to release a new marionette client. The disadvantage would be that we have another repository to clone or at least need to fetch and extract http://hg.mozilla.org/qa/testcase-data/archive/tip.zip.

Any thoughts on a preferred approach? I think I'd prefer to use testcase-data as it's also hosted for when we can use external resources.

Comment 4

[Zac C (:zac)]

It sounds like a lot of extra hassle when it comes to setting up and running systems locally, I'd prefer not to have that barrier to people outside of Fx OS using the tests. Having to add another command line is a usability hassle in more than one way. (as you've seen I've already tried to reduce the amount of args we need!)

If server-root could be set from Gaiatest (ie when gaiatest sets up MarionetteTestCase) and then we deliver it straight from the gaiatest/resources I'd feel more comfortable as the user would just get the correct configuration and test coverage automatically.

Comment 5

[Jonathan Griffin (:jgriffin)]

I like Zac's approach too; we could strip these out of the pypi package, and serve them directly from the gaia repo, which we're already cloning.

This allows us to maintain different versions per branch if we need to, as well.

Comment 6

[Zac C (:zac)]

Similarly I'm happy with as it is now too :)

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8442801 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20752

I liked Zac's suggestion too. This version just brings in two new HTML resources, and sets server root if it's not being overridden on the command line.

Comment 8

[Zac C (:zac)]

Comment on attachment 8442801 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20752

r+

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Landed in:
https://github.com/mozilla-b2g/gaia/commit/645cd78cfe414a6e08ef77a8226e28539de4344b