Last Comment Bug 1237179 - Add mach command to run firefox-ui-tests from the source directory
: Add mach command to run firefox-ui-tests from the source directory
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: mach (show other bugs)
: 45 Branch
: Unspecified Unspecified
-- normal (vote)
: mozilla46
Assigned To: Henrik Skupin (:whimboo) [away 02/18 - 02/27]
:
: Gregory Szorc [:gps] (away until 2017-03-20)
Mentors:
Depends on: 1212609 1230279
Blocks: 1237552 1239988 1245468
  Show dependency treegraph
 
Reported: 2016-01-05 23:42 PST by Henrik Skupin (:whimboo) [away 02/18 - 02/27]
Modified: 2016-02-03 04:02 PST (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
fixed
fixed


Attachments
Mach support for firefox-ui-tests (5.82 KB, patch)
2016-01-15 07:47 PST, Henrik Skupin (:whimboo) [away 02/18 - 02/27]
gps: review+
Details | Diff | Splinter Review
Mach support for firefox-ui-tests (backport mozilla-aurora) (5.11 KB, patch)
2016-01-21 15:48 PST, Henrik Skupin (:whimboo) [away 02/18 - 02/27]
gps: review+
Details | Diff | Splinter Review

Description User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-05 23:42:43 PST
With bug 1212609 we moved our firefox-ui-tests into mozilla-central. To make it easy for developers to run our tests we should add a mach command to directly execute our tests from the source directory.

We want this for mozilla-central and mozilla-aurora (which will become the next 45ESR).
Comment 1 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-15 07:47:54 PST
Created attachment 8708424 [details] [diff] [review]
Mach support for firefox-ui-tests

Simple addition by taking the marionette command as template. For now the firefox-ui-test command will work given that we cannot run our update cli due to no updates available. If more types will be added later we will extend the testtype option.

I hope it's ok that I did a small reordering of commands and search paths given that some entries had duplicates and I hate non-alphabetic sorting. :)
Comment 2 User image Axel Hecht [:Pike] 2016-01-15 08:01:26 PST
Just used it locally, and works like charm.
Comment 3 User image Gregory Szorc [:gps] (away until 2017-03-20) 2016-01-15 10:27:10 PST
Comment on attachment 8708424 [details] [diff] [review]
Mach support for firefox-ui-tests

Review of attachment 8708424 [details] [diff] [review]:
-----------------------------------------------------------------

Looks pretty straightforward!
Comment 5 User image Ryan VanderMeulen [:RyanVM] 2016-01-17 11:43:07 PST
https://hg.mozilla.org/mozilla-central/rev/853bcd30706a
Comment 6 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-20 14:00:46 PST
Patch applies cleanly on mozilla-aurora, so requesting check-in.
Comment 7 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-20 14:03:13 PST
(In reply to Henrik Skupin (:whimboo) from comment #6)
> Patch applies cleanly on mozilla-aurora, so requesting check-in.

Just to add, this patch only contains testing changes (a=testonly), nothing which affects Firefox as product.
Comment 9 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-21 04:17:02 PST
Greg, I have seen that people are modifying https://dxr.mozilla.org/mozilla-central/source/build/virtualenv_packages.txt and get packages added. It's something what I haven't done on this bug. Do I have to get this fixed?
Comment 10 User image Carsten Book [:Tomcat] 2016-01-21 05:21:30 PST
backed this out for test failures in https://treeherder.mozilla.org/logviewer.html#?job_id=1790153&repo=mozilla-aurora
Comment 11 User image Maja Frydrychowicz (:maja_zf) 2016-01-21 07:55:10 PST
(In reply to Carsten Book [:Tomcat] from comment #10)
> backed this out for test failures in
> https://treeherder.mozilla.org/logviewer.html#?job_id=1790153&repo=mozilla-
> aurora

I'm preparing a fix now.
Comment 12 User image Maja Frydrychowicz (:maja_zf) 2016-01-21 09:38:38 PST
Looks like we need to also uplift bug 1230279 for those test failures to go away.
Comment 13 User image Gregory Szorc [:gps] (away until 2017-03-20) 2016-01-21 11:22:01 PST
(In reply to Henrik Skupin (:whimboo) from comment #9)
> Greg, I have seen that people are modifying
> https://dxr.mozilla.org/mozilla-central/source/build/virtualenv_packages.txt
> and get packages added. It's something what I haven't done on this bug. Do I
> have to get this fixed?

It depends. That file populates the default virtualenv in the objdir as needed by the build system. For Python packages used by mach commands, it is acceptable for the mach command to install packages at run time. Search for "virtualenv_manager" in mach_commands.py files. This won't fly in automation, however, as there may be firewalls in place. If you need to run a mach command in automation, it's best to just add stuff to virtualenv_packages.txt.
Comment 14 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-21 13:46:24 PST
Ok, so we don't have to do it then. Regarding the backout of my patch due to bustage as caused by the not uplifted changes on bug 1230279, I will try to find a way without having to modify marionette itself. Even if we have a limit feature set it might be good to have.
Comment 15 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-21 15:48:13 PST
Created attachment 8710746 [details] [diff] [review]
Mach support for firefox-ui-tests (backport mozilla-aurora)

Backport patch for mozilla-aurora so that we can at least use mach to run the tests. The difference here is that we do not use our own cli entry function but directly invoke the MarionetteHarness. That way no change of Marionette is necessary which has been broken the media tests on its landing.
Comment 16 User image Gregory Szorc [:gps] (away until 2017-03-20) 2016-01-25 13:20:17 PST
Comment on attachment 8710746 [details] [diff] [review]
Mach support for firefox-ui-tests (backport mozilla-aurora)

Review of attachment 8710746 [details] [diff] [review]:
-----------------------------------------------------------------

Looks just like what I reviewed previously!
Comment 17 User image Henrik Skupin (:whimboo) [away 02/18 - 02/27] 2016-01-25 13:24:24 PST
This is a test-only change which we want to have on beta for esr45. Please land it.
Comment 18 User image Wes Kocher (:KWierso) 2016-01-25 14:56:03 PST
Pushed to beta since releng needed something pushed to test an unrelated fix:
https://hg.mozilla.org/releases/mozilla-beta/rev/ccc24c03342c

Note You need to log in before you can comment on or make changes to this bug.