987746 - Create a mach target for running gaia-ui tests

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P3)

Description

[Andrew Halberstadt [:ahal]]

I needed to run gaia-ui tests to test our a marionette refactor I'm working on, and figured it could benefit from a mach target. Seeing as the b2g-desktop mochitest/reftests also require a gaia profile, it didn't seem like a stretch to create a gaiatest command.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Create gaiatest mach command

This target should work both in m-c with b2g_desktop and in the B2G repo with a device. There is one problem with it, and that's that 'virtualenv_manager.call_setup()' uses the system python executable even after a call to 'virtualenv_manager.activate()'. :gps, is this a bug? Or is there another way I can install a module via setup.py with virtualenvmanager?

Zac, could you take a look at this and let me know if you would change anything? I especially want to know if you would add anything to the default 'testvars.json' or if there are any other command line options you think would be useful to pass on (currently this just does --profile and --testvars).

Comment 2

[Gregory Szorc [:gps]]

I think call_setup() using sys.executable() is a bug. If the virtualenv is activated, it should be using self.python_path.

Arguably, self.activate() should monkeypatch sys.path to the virtualenv's Python binary. But I think I ran into unexpected problems with this when I tried it before.

Hopefully bug 985141 can address these problems. If you want to work on that, power to you :)

Comment 3

[Andrew Halberstadt [:ahal]]

I think bug 985141 is a bit of a bigger scope than I have time for, maybe in the future :). I'll file a separate bug to get call_setup() using the proper executable.

Comment 4

[Zac C (:zac)]

Comment on attachment 8396430 [details] [diff] [review]
Create gaiatest mach command

Review of attachment 8396430 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not the right person to review this because I am not familiar with mach commands.

Comment 5

[Andrew Halberstadt [:ahal]]

I was mostly just looking for feedback around the cli options I was passing in and the dummy testvars.json I made.

Comment 6

[Zac C (:zac)]

Comment on attachment 8396430 [details] [diff] [review]
Create gaiatest mach command

f? me again

Comment 7

[Zac C (:zac)]

Comment on attachment 8396430 [details] [diff] [review]
Create gaiatest mach command

I didn't run it, I only reviewed the config. It looks ok to me except for the timezone settings in testvars; on desktopb2g it can pick that up from the host computer. We really only that on device where we wipe the profile and need to sync it back to the host computer.