Closed Bug 1237550 Opened 8 years ago Closed 8 years ago

[16Q1] Execute firefox-ui-tests in TaskCluster on Linux and run those tests for each check-in on integration branches and try


(Testing :: Firefox UI Tests, defect)

Not set


(firefox45 affected, firefox46 affected, firefox47 fixed, firefox-esr45 affected)

Tracking Status
firefox45 --- affected
firefox46 --- affected
firefox47 --- fixed
firefox-esr45 --- affected


(Reporter: whimboo, Assigned: whimboo)



(Keywords: meta)


(1 file)

This quarter we want to get our firefox-ui-tests executed in TaskCluster. For now they only support Linux, so we get started with this platform. Adding more platforms will be done once support gets added.

Also the execution of tests has to happen for each check-in across our integration branches and try
We might be blocked on bug 1246283 here. So adding it as dependency for now.
Depends on: 1246283
I got my first firefox-ui-tests executed by TaskCluster for non-e10s and e10s mode. Both tasks passed without a failure:

I have to make tweaks to the group and job names so we do not end-up with such ultra-long descriptions in Treeherder. Also e10s tests are not correctly put into their own group yet. That's behavior of Treeherder if you only update the group symbol but not the group description. I will work with Cameron to get this fixed.

I might have a patch later today.
There is already a successful try run here:

The one I started via mozreview is just for sanity.
Note to myself: Once the patch is in the tree I will also have to update to get our tests listed.
Comment on attachment 8724073 [details]
MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal

Looks good, just a nit about the symbol being used.

::: testing/taskcluster/tasks/tests/fx_linux64_firefox_ui.yml:25
(Diff revision 1)
> +      groupSymbol: tc-Fxfn
> +      symbol: en-US

I don't think this should be 'en-US'. It should be a unique symbol that identifies the job.

What about something like:
groupName: tc-FxUI
symbol: fn
Attachment #8724073 - Flags: review?(ahalberstadt)
So we run tests for a lot of locales in case of nightly and release builds. That's why the job name has to be the locale of the build. If we would introduce a new group now, I feel that would cause confusion because there is a disconnect then. Also someone could think `fn` is a locale. Please have a look at this example:

Keep in mind that in-tree we can only cover the functional but not the update tests. So I only added tasks for Fxfn to taskcluster.
Flags: needinfo?(ahalberstadt)
Comment on attachment 8724073 [details]
MozReview Request: Bug 1237550 - Add Taskcluster tasks for Firefox UI functional tests. r?ahal

Ok, those are good arguments.
Attachment #8724073 - Flags: review+
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
Missed to mark it as keep-open. I still have to add the tests to the try choose webpage.
Flags: needinfo?(ahalberstadt)
Resolution: FIXED → ---
Depends on: 1252229
Actually there is a mistake in the landed patch. I'm using firefox-ui as name of the test jobs which would not scale. This should have been firefox-ui-functional instead. I noticed that while trying to update the trychooser web page and thinking about other fx ui test types. I will come up with a follow-up patch.
Depends on: 1252242
All dependencies have been fixed. So this bug is done now.
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.