1275881 - Add client and scopes for firefox-ui-tests triggered from mozmill-ci

Status: RESOLVED FIXED

(Taskcluster :: Services, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I already had a conversation with Dustin and he initially did the following steps:

* The Client has been created as a temporary project with the name: project/qa/fx-ui-tests (https://tools.taskcluster.net/auth/clients/#project%252fqa%252ffx-ui-tests)

* The following scopes have been added:
** queue:route:tc-treeherder-stage.*
** queue:route:tc-treeherder.*
** docker-worker:cache:level-3-mozilla-central-test-workspace
** queue:create-task:aws-provisioner-v1/desktop-test
** queue:create-task:aws-provisioner-v1/qa-3-linux-fx-tests
** queue:define-task:aws-provisioner-v1/desktop-test
** queue:define-task:aws-provisioner-v1/qa-3-linux-fx-tests

qa-3-linux-fx-tests is the worker type for Linux as created on bug 1275681.

Today I tried to run a couple of tasks with desktop-test and then qa-3-linux-fx-tests, and noticed the following:

With the new worker type we always download the docker image. Is there something specific I would have to change to make sure caching is used? In that case its still the desktop-test image, and the following line is in my task definition:

"level-3-mozilla-central-test-workspace": "/home/worker/workspace"

The caching was working fine with desktop-test but with the new worker it always fails. Would I have to setup/define my own cache? Maybe the above is specific for desktop-test only.

I will do some more tests now for other branches and test flavors.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I tried to run our tests for an Aurora nightly on mozilla-aurora but it fails with the following missing scopes:

* docker-worker:cache:level-3-mozilla-aurora-test-workspace
* queue:route:tc-treeherder-stage.mozilla-aurora.bbf243cfc912c8d00e5e78157564f7e44168fe48

For the first its still related to the above question, if I need it or not. But I wonder about the treeherder stage scope. There is `queue:route:tc-treeherder-stage.` set, so why does it not cover everything?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

The images are cached on each instance, but you may have been using multiple images?  Also, where are you pulling desktop-test from?

I changed the cache scope to docker-worker:cache:level-3-* so that should help.

The message said you *need* those two scopes, right?  It didn't mention you already had one of them, perhaps?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> The images are cached on each instance, but you may have been using multiple
> images?  Also, where are you pulling desktop-test from?

The task definition I was using is:
https://queue.taskcluster.net/v1/task/ZN1S19gcRwyKCRYZ3upzSQ

What do you mean with pulling desktop-test from? The docker image? Maybe that is the problem here.

> I changed the cache scope to docker-worker:cache:level-3-* so that should
> help.
> 
> The message said you *need* those two scopes, right?  It didn't mention you
> already had one of them, perhaps?

I tried to run a task for an Aurora nightly build which has a different scope. That's why this additional one is necessary. All in all we would need the scopes for all branches down to the esr ones. Looks like with your change that should have been done.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Yes, how are you generating

    "image": {
      "path": "public/image.tar",
      "taskId": "YSckbw8PSK6wyTgDv6eeIA",
      "type": "task-image"
    },

as that taskId has a limited lifetime, and will change as the desktop-test docker image evolves..

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Dustin J. Mitchell [:dustin] from comment #4)
>     "image": {
>       "path": "public/image.tar",
>       "taskId": "YSckbw8PSK6wyTgDv6eeIA",
>       "type": "task-image"
>     },
> 
> as that taskId has a limited lifetime, and will change as the desktop-test
> docker image evolves..

That's an example as I used via the task creator. I actually never replaced that one! So I would have to fetch the last docker task via the index to figure out the appropriate taskid? When I do so I find the following:

https://tools.taskcluster.net/index/#docker.images.v1.mozilla-central.desktop-test/docker.images.v1.mozilla-central.desktop-test.latest

It tells me a taskid of `YSckbw8PSK6wyTgDv6eeIA` which is exactly what I have above.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Or do I always have to refer to inbound latest even when working on mozilla-aurora?

https://tools.taskcluster.net/index/#docker.images.v1.mozilla-inbound.desktop-test/docker.images.v1.mozilla-inbound.desktop-test.latest

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

You should probably refer to the latest from the branch you're testing.

Or maybe it's best to just build your own docker images for all of these, and put them on docker hub.  I don't think it's worth building the infra to find in-tree docker images for tasks that are only temporarily out-of-tree.

For low volume, it's OK to pull docker images from docker hub.  However, once this starts to scale up we'll need to move it in-tree and use the in-tree docker-image building system.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I talked with Dustin on IRC and we came to the conclusion that it will be better to build own docker images for firefox-ui-tests for the various Ubuntu versions we need. Those will be uploaded to docker hub, and the configs can be sticked into the tree. That means we can later easily reference those when the tests are getting triggered from in-tree graphs. I will file a new bug for that in a moment.

To check if all scopes are set I will try another task for mozilla-esr45 nightly builds. If that works well we can close this bug.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dustin, I think all works fine. So we can close this bug. Thank you.