1237179 - Add mach command to run firefox-ui-tests from the source directory

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, enhancement)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

With bug 1212609 we moved our firefox-ui-tests into mozilla-central. To make it easy for developers to run our tests we should add a mach command to directly execute our tests from the source directory.

We want this for mozilla-central and mozilla-aurora (which will become the next 45ESR).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Mach support for firefox-ui-tests

Simple addition by taking the marionette command as template. For now the firefox-ui-test command will work given that we cannot run our update cli due to no updates available. If more types will be added later we will extend the testtype option.

I hope it's ok that I did a small reordering of commands and search paths given that some entries had duplicates and I hate non-alphabetic sorting. :)

Comment 2

[Axel Hecht [:Pike]]

Just used it locally, and works like charm.

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8708424 [details] [diff] [review]
Mach support for firefox-ui-tests

Review of attachment 8708424 [details] [diff] [review]:
-----------------------------------------------------------------

Looks pretty straightforward!

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/853bcd30706a

Comment 5

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/853bcd30706a

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Patch applies cleanly on mozilla-aurora, so requesting check-in.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Henrik Skupin (:whimboo) from comment #6)
> Patch applies cleanly on mozilla-aurora, so requesting check-in.

Just to add, this patch only contains testing changes (a=testonly), nothing which affects Firefox as product.

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/a812bb3848d0

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Greg, I have seen that people are modifying https://dxr.mozilla.org/mozilla-central/source/build/virtualenv_packages.txt and get packages added. It's something what I haven't done on this bug. Do I have to get this fixed?

Comment 10

[Carsten Book [:Tomcat]]

backed this out for test failures in https://treeherder.mozilla.org/logviewer.html#?job_id=1790153&repo=mozilla-aurora

Comment 11

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

(In reply to Carsten Book [:Tomcat] from comment #10)
> backed this out for test failures in
> https://treeherder.mozilla.org/logviewer.html#?job_id=1790153&repo=mozilla-
> aurora

I'm preparing a fix now.

Comment 12

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Looks like we need to also uplift bug 1230279 for those test failures to go away.

Comment 13

[Gregory Szorc [:gps]]

(In reply to Henrik Skupin (:whimboo) from comment #9)
> Greg, I have seen that people are modifying
> https://dxr.mozilla.org/mozilla-central/source/build/virtualenv_packages.txt
> and get packages added. It's something what I haven't done on this bug. Do I
> have to get this fixed?

It depends. That file populates the default virtualenv in the objdir as needed by the build system. For Python packages used by mach commands, it is acceptable for the mach command to install packages at run time. Search for "virtualenv_manager" in mach_commands.py files. This won't fly in automation, however, as there may be firewalls in place. If you need to run a mach command in automation, it's best to just add stuff to virtualenv_packages.txt.

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, so we don't have to do it then. Regarding the backout of my patch due to bustage as caused by the not uplifted changes on bug 1230279, I will try to find a way without having to modify marionette itself. Even if we have a limit feature set it might be good to have.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Mach support for firefox-ui-tests (backport mozilla-aurora)

Backport patch for mozilla-aurora so that we can at least use mach to run the tests. The difference here is that we do not use our own cli entry function but directly invoke the MarionetteHarness. That way no change of Marionette is necessary which has been broken the media tests on its landing.

Comment 16

[Gregory Szorc [:gps]]

Comment on attachment 8710746 [details] [diff] [review]
Mach support for firefox-ui-tests (backport mozilla-aurora)

Review of attachment 8710746 [details] [diff] [review]:
-----------------------------------------------------------------

Looks just like what I reviewed previously!

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is a test-only change which we want to have on beta for esr45. Please land it.

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Pushed to beta since releng needed something pushed to test an unrelated fix:
https://hg.mozilla.org/releases/mozilla-beta/rev/ccc24c03342c