Note: There are a few cases of duplicates in user autocompletion which are being worked on.

[16Q1] Execute firefox-ui-tests in TaskCluster on Linux and run those tests for each check-in on integration branches and try

RESOLVED FIXED in Firefox 47

Status

Testing
Firefox UI Tests
RESOLVED FIXED
2 years ago
a year ago

People

(Reporter: whimboo, Assigned: whimboo)

Tracking

({meta})

unspecified
mozilla47
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox45 affected, firefox46 affected, firefox47 fixed, firefox-esr45 affected)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

This quarter we want to get our firefox-ui-tests executed in TaskCluster. For now they only support Linux, so we get started with this platform. Adding more platforms will be done once support gets added.

Also the execution of tests has to happen for each check-in across our integration branches and try
We might be blocked on bug 1246283 here. So adding it as dependency for now.
Depends on: 1246283
I got my first firefox-ui-tests executed by TaskCluster for non-e10s and e10s mode. Both tasks passed without a failure:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=abef13e53bd7&filter-tier=1&filter-tier=2&filter-tier=3

I have to make tweaks to the group and job names so we do not end-up with such ultra-long descriptions in Treeherder. Also e10s tests are not correctly put into their own group yet. That's behavior of Treeherder if you only update the group symbol but not the group description. I will work with Cameron to get this fixed.

I might have a patch later today.
Created attachment 8724073 [details]
MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal

Review commit: https://reviewboard.mozilla.org/r/36841/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/36841/
Attachment #8724073 - Flags: review?(ahalberstadt)
There is already a successful try run here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5b2a748fc3c&filter-tier=1&filter-tier=2&filter-tier=3

The one I started via mozreview is just for sanity.
Note to myself: Once the patch is in the tree I will also have to update http://trychooser.pub.build.mozilla.org/ to get our tests listed.
Comment on attachment 8724073 [details]
MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal

https://reviewboard.mozilla.org/r/36841/#review33519

Looks good, just a nit about the symbol being used.

::: testing/taskcluster/tasks/tests/fx_linux64_firefox_ui.yml:25
(Diff revision 1)
> +      groupSymbol: tc-Fxfn
> +      symbol: en-US

I don't think this should be 'en-US'. It should be a unique symbol that identifies the job.

What about something like:
groupName: tc-FxUI
symbol: fn
Attachment #8724073 - Flags: review?(ahalberstadt)
So we run tests for a lot of locales in case of nightly and release builds. That's why the job name has to be the locale of the build. If we would introduce a new group now, I feel that would cause confusion because there is a disconnect then. Also someone could think `fn` is a locale. Please have a look at this example:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=7557d8e62bd9&filter-searchStr=Firefox%20UI&filter-tier=2&filter-tier=3

Keep in mind that in-tree we can only cover the functional but not the update tests. So I only added tasks for Fxfn to taskcluster.
Flags: needinfo?(ahalberstadt)
Comment on attachment 8724073 [details]
MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal

https://reviewboard.mozilla.org/r/36841/#review33557

Ok, those are good arguments.
Attachment #8724073 - Flags: review+

Comment 9

a year ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/607da9c6ab37
https://hg.mozilla.org/mozilla-central/rev/607da9c6ab37
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
status-firefox47: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
Missed to mark it as keep-open. I still have to add the tests to the try choose webpage.
Status: RESOLVED → REOPENED
status-firefox45: --- → affected
status-firefox46: --- → affected
status-firefox-esr45: --- → affected
Flags: needinfo?(ahalberstadt)
Resolution: FIXED → ---
Depends on: 1252229
Actually there is a mistake in the landed patch. I'm using firefox-ui as name of the test jobs which would not scale. This should have been firefox-ui-functional instead. I noticed that while trying to update the trychooser web page and thinking about other fx ui test types. I will come up with a follow-up patch.
Status: REOPENED → ASSIGNED
Depends on: 1252242
All dependencies have been fixed. So this bug is done now.
Status: ASSIGNED → RESOLVED
Last Resolved: a year agoa year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.