Closed Bug 1237550 Opened 5 years ago Closed 5 years ago
[16Q1] Execute firefox-ui-tests in Task
Cluster on Linux and run those tests for each check-in on integration branches and try
58 bytes, text/x-review-board-request
This quarter we want to get our firefox-ui-tests executed in TaskCluster. For now they only support Linux, so we get started with this platform. Adding more platforms will be done once support gets added. Also the execution of tests has to happen for each check-in across our integration branches and try
I got my first firefox-ui-tests executed by TaskCluster for non-e10s and e10s mode. Both tasks passed without a failure: https://treeherder.mozilla.org/#/jobs?repo=try&revision=abef13e53bd7&filter-tier=1&filter-tier=2&filter-tier=3 I have to make tweaks to the group and job names so we do not end-up with such ultra-long descriptions in Treeherder. Also e10s tests are not correctly put into their own group yet. That's behavior of Treeherder if you only update the group symbol but not the group description. I will work with Cameron to get this fixed. I might have a patch later today.
Review commit: https://reviewboard.mozilla.org/r/36841/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/36841/
There is already a successful try run here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5b2a748fc3c&filter-tier=1&filter-tier=2&filter-tier=3 The one I started via mozreview is just for sanity.
Note to myself: Once the patch is in the tree I will also have to update http://trychooser.pub.build.mozilla.org/ to get our tests listed.
Comment on attachment 8724073 [details] MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal https://reviewboard.mozilla.org/r/36841/#review33519 Looks good, just a nit about the symbol being used. ::: testing/taskcluster/tasks/tests/fx_linux64_firefox_ui.yml:25 (Diff revision 1) > + groupSymbol: tc-Fxfn > + symbol: en-US I don't think this should be 'en-US'. It should be a unique symbol that identifies the job. What about something like: groupName: tc-FxUI symbol: fn
So we run tests for a lot of locales in case of nightly and release builds. That's why the job name has to be the locale of the build. If we would introduce a new group now, I feel that would cause confusion because there is a disconnect then. Also someone could think `fn` is a locale. Please have a look at this example: https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=7557d8e62bd9&filter-searchStr=Firefox%20UI&filter-tier=2&filter-tier=3 Keep in mind that in-tree we can only cover the functional but not the update tests. So I only added tasks for Fxfn to taskcluster.
Comment on attachment 8724073 [details] MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal https://reviewboard.mozilla.org/r/36841/#review33557 Ok, those are good arguments.
Attachment #8724073 - Flags: review+
Missed to mark it as keep-open. I still have to add the tests to the try choose webpage.
5 years ago
Depends on: 1252229
Actually there is a mistake in the landed patch. I'm using firefox-ui as name of the test jobs which would not scale. This should have been firefox-ui-functional instead. I noticed that while trying to update the trychooser web page and thinking about other fx ui test types. I will come up with a follow-up patch.
Status: REOPENED → ASSIGNED
5 years ago
Depends on: 1252242
All dependencies have been fixed. So this bug is done now.
Status: ASSIGNED → RESOLVED
Closed: 5 years ago → 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.