Closed Bug 1522997 Opened 11 months ago Closed 10 months ago

[meta] Run taskcluster generic-worker on windows/aarch64

Categories

(Core :: General, enhancement)

enhancement
Not set

Tracking

()

RESOLVED FIXED

People

(Reporter: egao, Assigned: egao)

References

(Depends on 8 open bugs)

Details

(Keywords: meta)

Find out a way, and document a way to run Taskcluster generic-worker on aarch64.

No longer blocks: aarch64-windows-meta
Depends on: 1520867
No longer depends on: arm64-windows-tests

Sorry for the juggling of dependencies. Haven't done this before, simply trying to put it sufficiently low in the tree that it doesn't affect/cc everyone else.

I have got Taskcluster generic-worker running on aarch64.

Process

Prerequisites

  • win10/aarch64 hardware must be taken out of Windows S mode
  • User Account Control is disabled
  • Set up "Remote Desktop Users" on aarch64 machine: net localgroup "Remote Desktop Users" /add
    • my laptop did not have the group Remote Desktop Users added, hence this was a required step. YMMV.
  • Request Taskcluster scope assume:project:taskcluster:generic-worker-tester
    • have an account used to authenticate to Taskcluster
  • Download and unzip NSSM to a directory in C:\; should look something like C:\nssm-2.24\
  • disable Windows Firewall

Optional Requirements

  • Download and install Mozilla-Build 2.0.0, available from here
  • Download mozharness and extract to C:\mozilla-build\; the structure should be C:\mozilla-build\mozharness\...

Steps

  1. download the current 386 release of generic-worker-windows-386.exe from taskcluster generic-worker releases.
  2. download the latest 386 version of livelog.exe and taskcluster-proxy.exe.
  3. create new directory C:\generic-worker
  4. move the three executable files under the new directory.
  5. rename generic-worker-windows-386.exe to generic-worker.exe.
  6. generate two signing keys:
generic-worker new-openpgp-keypair --file <unique_file_name>
generic-worker new-ed25519-keypair --file <unique_file_name>
  1. create generic-worker.config and include the following:
{
    "accessToken":                "<access token tied to taskcluster>",
    "clientId":                   "<client ID tied to taskcluster>",
    "ed25519SigningKeyLocation":  "<file location you wrote ed25519 private key in step 6>",
    "livelogSecret":              "<any text>",
    "openpgpSigningKeyLocation":  "<file location you wrote gpg private key kn step 6>",
    "provisionerId":              "test-provisioner",
    "publicIP":                   "<ideally an IP address of one of your network interfaces>",
    "rootURL":                    "https://taskcluster.net",
    "workerGroup":                "test-worker-group",
    "workerId":                   "test-worker-id",
    "workerType":                 "<a unique string that only you will use for your test worker(s)>"
}
  1. launch cmd.exe with Administrator rights
  2. cd c:\generic-worker
  3. generic-worker.exe install service --config generic-worker.config --nssm c:\nssm-2.24\win32\nssm.exe
  4. reboot once installation completes
  5. launch cmd.exe with Administrator rights
  6. sc query "Generic Worker" - ensure the status is RUNNING

The above should get the laptop in a state where generic-worker is running and is ready to accept tasks. Once in this state, go to https://tools.taskcluster.net and create a task with provisionerId and workerType that match the content of generic-worker.config. An example is below:

provisionerId: test-provisioner
workerType: aarch64windowsedwin
retries: 5
created: '2019-01-31T17:47:35.526Z'
deadline: '2019-01-31T20:47:35.526Z'
expires: '2020-01-31T20:47:35.526Z'
scopes: []
taskGroupId: YUosbKEtSs2GZGLenPTPpw
dependencies:
  - DdbhnS_zRmiRXF5yh8UBiA
payload:
  maxRunTime: 120
  command:
    - dir
    - echo hi
metadata:
  owner: egao@mozilla.com
  source: >-
    https://hg.mozilla.org/mozilla-central/file/6b7bc40023111979a4755dfeca82629c9c022cd5/taskcluster/ci/test
  description: test run
  name: test run
Summary: Run taskcluster generic-worker on aarch64 → [meta] Run taskcluster generic-worker on aarch64

Taskcluster Generic-Worker was able to run on Windows 64 ARM reference laptop, accept tasks and execute them.

This task records the successful run.

Now moving to invoke a CI test on the reference laptop.

Depends on: 1523722
Depends on: 1524114
Depends on: 1524400
Summary: [meta] Run taskcluster generic-worker on aarch64 → [meta] Run taskcluster generic-worker on windows/aarch64
Depends on: 1524410
Blocks: 1525118
No longer blocks: 1525118
Depends on: 1525118

Marking this as complete.

This is a successful taskcluster generic-worker run that implements the entire process:

  • try push
  • build win10/aarch64
  • run tests using build artifact
  • report results to taskcluster
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED

(In reply to Edwin Gao (:egao) from comment #4)

Marking this as complete.

This is a
successful taskcluster generic-worker run that implements the entire process:

  • try push
  • build win10/aarch64
  • run tests using build artifact
  • report results to taskcluster

This is great! Just to confirm my understanding, there's still more to come in terms of additional test suites, right? Is it fair to expect that eventually we'll have a full taskcluster run where it's possible to see all of the failing tests from one place? (And be able to do the runs again over time and watch them green up?)

(In reply to David Major [:dmajor] from comment #5)

(In reply to Edwin Gao (:egao) from comment #4)

Marking this as complete.

This is a
successful taskcluster generic-worker run that implements the entire process:

  • try push
  • build win10/aarch64
  • run tests using build artifact
  • report results to taskcluster

This is great! Just to confirm my understanding, there's still more to come in terms of additional test suites, right? Is it fair to expect that eventually we'll have a full taskcluster run where it's possible to see all of the failing tests from one place? (And be able to do the runs again over time and watch them green up?)

Yes, jmaher, gbrown and I only managed to run xpcshell suite so far - we have only 1 machine executing tasks, so it is rather slow. This is the Treeherder view of Taskcluster tasks.

From Treeherder, I've checked the following functionality and it seems to work:

  • schedule new jobs via Treeherder
  • view results

(In reply to Edwin Gao (:egao) from comment #6)

[This](https://treeherder.mozilla.org/#/
jobs?repo=try&revision=7f2edbc029097dc7ca3c14f2ae488e4d7275d147) is the
Treeherder view of Taskcluster tasks.

\o/ Thanks!

Depends on: 1525434
Depends on: 1526015

There would be value in getting the most basic tests working, sufficient to prevent something like bug 1525588 making it to m-c.

Depends on: 1527469
Depends on: 1530737
Depends on: 1531876
Depends on: 1531878
Depends on: 1527177
Depends on: 1533114
Depends on: 1533880
Depends on: 1535467
Depends on: 1536283
Depends on: 1538785
Depends on: 1540213
Depends on: 1543521
Depends on: 1539693
Depends on: 1545810
Depends on: 1546532
Depends on: 1546728
Depends on: 1546732
Depends on: 1547820
Depends on: 1552051
Depends on: 1572185
You need to log in before you can comment on or make changes to this bug.