Closed Bug 1379603 Opened 7 years ago Closed 7 years ago

Enable triggering of hardware and talos tests on TaskCluster Windows

Categories

(Infrastructure & Operations :: RelOps: General, task)

Product:
Infrastructure & Operations
Component:
RelOps: General
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: grenade, Assigned: grenade)

References

Details

Attachments

(2 files)

[treeherder] grenade:patch-2 > mozilla:master
47 bytes, text/x-github-pull-request
Details | Review
Bug 1379603 - enable windows hardware tests on try;
59 bytes, text/x-review-board-request
dustin
: review+
grenade
: review+
Details 
      No description provided.
Comment on attachment 8884812 [details]
Bug 1379603 - enable windows hardware tests on try;

https://reviewboard.mozilla.org/r/155694/#review160806

We talked a bit by video about this.  I think it would be best to fold these jobs into the existing "windows" test platform.  There's ample room to clean up the separate "VM" test platform later (so there are just four windwos test platforms: (win7, win10) X (opt, debug)), but that will be a lot easier once Buildbot's not in the picture anymore.

I think the `set_worker_type` transform is going to be critical here, as it is the place to figure out, based on the suite name or some other characteristic of the test description, which `worker-type` to use.  It's already got a small conditional set up for talos, and the remainder is using a dictionary mapping test platform to worker type.  But that isn't the only way to do it!  A little extra logic here, say mapping any of a set of suites to a particular workerType, should do the trick.

::: taskcluster/ci/test/test-platforms.yml:169
(Diff revision 2)
> +        - windows-hw-tests
> +windows7-32-hw/opt:
> +    build-platform: win32/opt
> +    test-sets:
> +        - windows-hw-tests
> +        - windows-talos



::: taskcluster/ci/test/test-sets.yml:136
(Diff revision 2)
>      - mochitest-gpu
>      - mochitest-webgl
>  
> -# these tests currently run on hardware, but may migrate above when validated
> -# see https://bugzilla.mozilla.org/show_bug.cgi?id=1280474#c36
> -#    - mochitest-chrome
> +windows-hw-tests:
> +    - mochitest-chrome
> +    - mochitest-clipboard
Attachment #8884812 - Flags: review?(dustin) → review-
dustin:

thanks for the suggestion to fold the test platforms into one. when i implemented that, it made perfect sense. now i'm struggling with how to determine that a talos test is meant for buildbot bridge or taskcluster workers. linux uses:

    `if config.config['args'].taskcluster_worker:`

up till now on windows, all talos builds were for bbb, but now that they aren't, i need to figure out a similar construct for windows. i don't really understand where the config.config['args'].taskcluster_worker is getting set or even if this is a good way to do it (for windows). any suggestions?
Flags: needinfo?(dustin)
think i found a solution in checking for "taskcluster" in the test config path
Flags: needinfo?(dustin)
(In reply to Rob Thijssen (:grenade - UTC+3) from comment #5)
>     `if config.config['args'].taskcluster_worker:`

This is looking for a try-syntax command-line option, I think `-w`.  It let me land support for Talos on taskcluster-worker on the moonshot hardware, and test it in try, without making *everyone's* try job run on that platform.  You could do a similar thing -- even using `.taskcluster_worker` despite the tasks running with generic-worker.  It's just a temporary hack anyway.  But it sounds like you've figured things out..
Comment on attachment 8884812 [details]
Bug 1379603 - enable windows hardware tests on try;

https://reviewboard.mozilla.org/r/155694/#review161242

::: taskcluster/ci/test/test-platforms.yml:144
(Diff revision 4)
> -        - windows-gpu-tests
> +        - windows-tests
> +        - headless
>  windows7-32/opt:
>      build-platform: win32/opt
>      test-sets:
> -        - windows-gpu-tests
> +        - windows-tests

It looks like this is condensing everything down to four platforms.  That's awesome, but from our chat yesterday it sounded like it would lead to extra work to do the same in Buildbot..

::: taskcluster/taskgraph/transforms/tests.py:47
(Diff revision 4)
>      'xlarge': 'aws-provisioner-v1/gecko-t-linux-xlarge',
>      'legacy': 'aws-provisioner-v1/gecko-t-linux-medium',
>      'default': 'aws-provisioner-v1/gecko-t-linux-large',
>  }
>  
> -# windows / os x worker types keyed by test-platform
> +# windows worker types keyed by test-platform and virtualization

awesome :)

::: taskcluster/taskgraph/transforms/tests.py:157
(Diff revision 4)
>      # The EC2 instance size to run these tests on.
>      Required('instance-size', default='default'): optionally_keyed_by(
>          'test-platform',
>          Any('default', 'large', 'xlarge', 'legacy')),
>  
> +    # type of virtualization or hardware required by test.

awesome :)

::: taskcluster/taskgraph/transforms/tests.py:719
(Diff revision 4)
>          if test_platform.startswith('macosx'):
>              # note that some portion of these will be allocated to BBB below
>              test['worker-type'] = MACOSX_WORKER_TYPES['macosx64']
>          elif test_platform.startswith('win'):
> -            if test.get('suite', '') == 'talos':
> +            if test.get('suite', '') == 'talos' and \
> +                    not any('taskcluster' in cfg for cfg in test['mozharness']['config']):

So, the effect here is that anything that includes taskcluster_windows_config.py will *always* run on hardware, whereas everything else will run in BBB.  Is that what you're going for?
Attachment #8884812 - Flags: review?(dustin) → review+
Comment on attachment 8884812 [details]
Bug 1379603 - enable windows hardware tests on try;

https://reviewboard.mozilla.org/r/155694/#review161242

> It looks like this is condensing everything down to four platforms.  That's awesome, but from our chat yesterday it sounded like it would lead to extra work to do the same in Buildbot..

yes, but after some thought, the only downside is that buildbot lists tests in an extra "-vm" row in treeherder, whereas tc doesn't. rather than invest time in patching bb to match, we can just focus on greening tc win and eliminating bb win talos sooner and the extra row will disappear on its own.

> So, the effect here is that anything that includes taskcluster_windows_config.py will *always* run on hardware, whereas everything else will run in BBB.  Is that what you're going for?

yes!
last push to review just corrected the alphabetical ordering of the test suites and corrected an omission of windows-talos from windows7-32/opt
Keywords: checkin-needed
Comment on attachment 8884812 [details]
Bug 1379603 - enable windows hardware tests on try;

https://reviewboard.mozilla.org/r/155694/#review161256
Attachment #8884812 - Flags: review+
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/8cffa1c2d102
enable windows hardware tests on try; r=dustin
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/8cffa1c2d102
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Blocks: 1381130
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: