Closed Bug 1245468 Opened 4 years ago Closed 4 years ago

Follow-ups to mach command to run firefox-ui-tests from the source directory

Categories

(Testing :: Firefox UI Tests, defect)

defect
Not set

Tracking

(firefox47 fixed)

RESOLVED FIXED
mozilla47
Tracking Status
firefox47 --- fixed

People

(Reporter: Pike, Assigned: sydpolk)

References

Details

Attachments

(1 file)

I've tried to run 

./mach firefox-ui-test --testvars foo.json path/to/test.py

and failed, deeply in stuff.

Trying to find out what happens, I recognized a few odd ends in https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/mach_commands.py.

It starts with an additional parsing of the arguments in line 32-34. I'm pretty sure we shouldn't do that, the arguments are passed allright in kwargs already. Also, at this point, passing in global mach options breaks in argsparse.

Looking further, I figured out what happens if we don't have an args object. That's passed into cli_functional.mn_cli, which is an alias for marionette.runtests.cli, https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/runtests.py#79.

That function always calls sys.exit(), which makes me think that we shouldn't call that. Which brought be to MarionetteHarness, which seems to be doing very little beyond setting pydebugger on MarionetteTestCase, if specified.

Short of that, I think the mach command should just directly call into FirefoxUITestRunner.runtests()?
Thanks for the bug report Axel. I won't have the time the next days but I will check that once I'm free. In general I simply took the code we have for running Marionette tests via mach.
This is actually a bug in the Firefox mach command. Moving over to the correct component.
Component: mach → Firefox UI Tests
Product: Core → Testing
QA Contact: hskupin
Please note that we need a fix similar to the one on bug 1250641.
The workaround is to change the ordering of the flags to put the test file first, for example:

    ./mach marionette-test FILE --gecko-log -
Duplicate of this bug: 1251036
Assignee: nobody → spolk
Comment on attachment 8723698 [details]
MozReview Request: Bug 1245468 - Fix parmaeter handling for firefox-ui tests. r?whimboo, r?gps

https://reviewboard.mozilla.org/r/36683/#review33371

This looks fine. Keep in mind that this is all firefox-ui related code, so you don't have to request review from gps here.
gps discovered a problem with my initial fix for 1250641, so I think he should see this and the fix for bug 1251037.
Comment on attachment 8723698 [details]
MozReview Request: Bug 1245468 - Fix parmaeter handling for firefox-ui tests. r?whimboo, r?gps

https://reviewboard.mozilla.org/r/36683/#review33575

LGTM
Attachment #8723698 - Flags: review?(gps) → review+
https://hg.mozilla.org/mozilla-central/rev/360821d7b53d
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
You need to log in before you can comment on or make changes to this bug.