Closed Bug 1351366 Opened 7 years ago Closed 7 years ago

Test OCC configured Windows 7 hardware under TaskCluster

Categories

(Infrastructure & Operations :: RelOps: General, task)

Product:
Infrastructure & Operations
Component:
RelOps: General
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: markco, Assigned: markco)

References

Details

      No description provided.
Rob: What is the best way to start testing, and who should I cc on this bug?
Assignee: relops → mcornmesser
Flags: needinfo?(rthijssen)
pmoore runs tc jobs from a vm on his own pc, so he may be able to help with how to configure generic-worker to be able to claim tasks.

- the config in c:\generic-worker\generic-worker.config is normally provided by the aws provisioner but on hardware we need to instead query the new taskcluster-host-secrets to get the secrets this file will contain. For now, it's probably enough to just manually create the file and that's where Pete might be able to help with what should go in there or knowing wether we can just grab a config from an ec2 instance and expect that to work.
- once g-w is configured and running, it should be possible to create a task at https://tools.taskcluster.net/task-creator and just set the workerType there to gecko-t-win7-32-hw or gecko-t-win10-64-hw in order to have it claimed by your hw workers.
Flags: needinfo?(rthijssen)
Hey Mark,

This is the generic worker config I've just set up for OS X in scl3:
  https://hg.mozilla.org/build/puppet/file/fb264d238596/modules/generic_worker/templates/generic-worker.config.erb

As you see, there is already a global client id and access token you can use from hiera, until we have integrated with taskcluster-host-secrets service.

The combination of provisionerId and workerType uniquely identifies a work queue - on OS X I've used "scl3-puppet" for the provisionerId - I'd recommend using that too for windows workers, as it reflects that these workers are managed by puppet in scl3. If you do go with this provisionerId, make sure you enter it when creating tasks in the task creator (link from Rob above).

All of the generic worker configuration settings are documented in the README.md here:
https://github.com/taskcluster/generic-worker#set-up-your-env

Good luck, and give me a shout if you get stuck on anything! (or ask in #taskcluster)
Note, this is additional config we set in gecko-t-win10-64 (see https://tools.taskcluster.net/aws-provisioner/#gecko-t-win10-64/view), which might help inspire the config to set up on hardware (i.e. you might want to overlay this, or a modified form of it, on the puppet config linked from comment 3).

      {
        "cachesDir": "Z:\\caches",
        "downloadsDir": "Z:\\downloads",
        "livelogCertificate": "C:\\generic-worker\\livelog.crt",
        "livelogExecutable": "C:\\generic-worker\\livelog.exe",
        "livelogKey": "C:\\generic-worker\\livelog.key",
        "livelogSecret": "********************************************",
        "runAfterUserCreation": "C:\\generic-worker\\task-user-init.cmd",
        "signingKeyLocation": "not-configured",
        "subdomain": "taskcluster-worker.net",
        "tasksDir": "Z:\\",
        "usersDir": "Z:\\",
        "workerTypeMetadata": {
          "machine-setup": {
            **************************************************************
            ****** Put here whatever you consider appropriate, that ******
            ****** provides insight into how the machine was set up ******
            **************************************************************
            "created": "<date>",
            "manifest": "<link to the OCC manifest>",
            "maintainer: "Monkey Alan <monkey.alan@wernhamhogg.co.uk>"
            **************************************************************
            ********* This section is entirely freeform, so use it  ******
            ********* as you see fit. Be aware that this will also  ******
            ********* get logged in all tasks that run on this      ******
            ********* worker type, so don't make it too long! :)    ******
            **************************************************************
          }
        }
      }
I have the following so far:

Config file:
{
  "accessToken": "************",
  "cachesDir": "C:\\caches",
  "certificate": "",
  "checkForNewDeploymentEverySecs": 0,
  "cleanUpTaskDirs": true,
  "clientId": "***********",
  "deploymentId": "",
  "downloadsDir": "C:\\Users\\GenricWorker\\downloads",
  "idleTimeoutSecs": 0,
  "instanceId": "",
  "instanceType": "",
  "livelogCertificate": "C:\\generic-worker\\livelog.crt",
  "livelogExecutable": "C:\\generic-worker\\livelog.exe",
  "livelogGETPort": 60023,
  "livelogKey": "C:\\generic-worker\\livelog.key",
  "livelogPUTPort": 60022,
  "livelogSecret": "***************",
  "numberOfTasksToRun": 0,
  "privateIP": "",
  "provisionerId": "scl3-puppet",
  "publicIP": "10.26.40.41",
  "refreshURLsPrematurelySecs": 310,
  "region": "",
  "requiredDiskSpaceMegabytes": 10240,
  "runAfterUserCreation": "C:\\generic-worker\\task-user-init.cmd",
  "runTasksAsCurrentUser": false,
  "shutdownMachineOnInternalError": false,
  "shutdownMachineOnIdle": false,
  "signingKeyLocation": "not-configured",
  "subdomain": "taskcluster-worker.net",
  "tasksDir": "C:\\",
  "workerGroup": "scl3",
  "workerId": "t-w864-ix-011",
  "workerType": "gecko-t-win7-32-hw",
  "workerTypeMetadata": {
    "config": {
      "deploymentId": "",
      "runTaskAsCurrentUser": false
    },
    "generic-worker": {
      "go-arch": "386",
      "go-os": "windows",
      "go-version": "go1.7.5",
      "release": "https://github.com/taskcluster/generic-worker/releases/tag/v8.1.1",
      "version": "8.1.1"
    },
    "machine-setup": {
      "created": "20170404",
      "maintainer": "Markco \u003cmcornmesser@mozilla.com\u003e",
      "manifest": "https://github.com/markcor/OpenCloudConfig/blob/master/userdata/Manifest/gecko-t-win7-32-hw.json"
    }
  }

Then from an elevated command prompt I run: 
generic-worker.exe run --config C:\generic-worker\work.config > C:\log\genwork.log

Task wise I have been picking a random Win 7 task and updated it with: 

Provisioner	scl3-puppet
WorkerType	gecko-t-win7-32-hw

After running the command above, there is output to the command prompt, and then it appears that explorer.exe crashes. There is only wallpaper no mouse control and no task bar. What makes this a bit more fun is I can only redirect the a line of the command prompt output, so I am not sure where it is getting to. 


This brings me to a handful questions. First one any suggestion on where to go from here to try to pick up a task, or is there additional logging from the generic worker that I can look into?

The other questions are regarding the config file itself. Where is the process were we going to initial deploy/manage this file? Will it get all the info it needs from the provisioner and TC secrets? OR is this a file we will have to deploy through OCC initially and then have multiple parts building into it?
if you change runTasksAsCurrentUser to true & numberOfTasksToRun to 1, you'll get better results and mirror the settings we use on ec2 win7.
- runTasksAsCurrentUser causes the task to run as whatever user executes generic-worker.exe
- numberOfTasksToRun is actually the reboot interval which needs to be 1 on win 7 in order to work around a bunch of cleanup problems there.

the explorer crash is actually a desktop switch from the current user to a task user which doesn't work well for win 7 tests.

the config file will need to be completed by occ. you or i can write something into rundsc.ps1 that will query tc-host-secrets to get the secrets and then merge those into the template config that lives at: https://github.com/mozilla-releng/OpenCloudConfig/blob/master/userdata/Configuration/GenericWorker/generic-worker.config
pmoore: What should the clintID be in this case? And is it a secret? I ask because the osx one from the link above is in hiera. I wouldn't think it would be a secret, but I am asking just to be safe.
Flags: needinfo?(pmoore)
The clientId is not private, that's right.

Note the clientId is in hiera, but it isn't secret (it is plain text in the hiera file). It just makes it easier to keep clientId and accessToken in sync to have them stored together in hiera, but have the clientId plain text, and the access token encrypted.

From memory, I think this is also how it was set up with the releng worker types too (signing worker, script worker, buildbot bridge worker, ...).
Flags: needinfo?(pmoore)
What should the client ID be in this case? I have tried one I pulled from an existing win 7 instances and the osx one from the above link, and both seem to have caused an error.
Flags: needinfo?(pmoore)
What error did you get? You might just need to tweak some taskcluster roles.

I'm heading off on PTO now, but grenade/dustin should be able to help you. It also depends on whether releng want to have different clients for different worker pools in scl3 - probably would make sense to do that, to limit the workers to only being able to affect tasks that are for their own worker types etc.
Flags: needinfo?(pmoore)
Dustin: Rob: I am trying to get to a functioning for the generic worker for hardware Windows 7. The part I think I am getting hung up on is the clientID. Below are the various results from my attempts. Is there a clinetID that should work? DO we need create a new one with new scopes? I am trying to get to a point that I can test OCC hardware Windows 7 configuration.  

First using the one I took from an active AWS Windows 7 instance. project/taskcluster/aws-provisioner/production:

2017/04/07 19:13:57 Detected windows platform
2017/04/07 19:13:57 No previous task user desktop, so no need to close any open
desktops
2017/04/07 19:13:57 Trying to remove directory 'C:\task_1491592250' via os.Remov
eAll(path) call as GenericWorker user...
2017/04/07 19:13:57 Created dir: C:\task_1491592437\public\logs
panic: (Permanent) HTTP response code 401
HTTP/1.1 401 Unauthorized
Content-Length: 409
Access-Control-Allow-Headers: X-Requested-With,Content-Type,Authorization,Accept
,Origin
Access-Control-Allow-Methods: OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT
Access-Control-Allow-Origin: *
Access-Control-Request-Method: *
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Fri, 07 Apr 2017 19:13:56 GMT
Etag: W/"199-17677d86"
Server: Cowboy
Strict-Transport-Security: max-age=7776000
Via: 1.1 vegur
Www-Authenticate: hawk
X-Powered-By: Express

{
  "code": "AuthenticationFailed",
  "message": "Unauthorized: Bad mac\n----\nmethod:     pollTaskUrls\nerrorCode:
 AuthenticationFailed\nstatusCode: 401\ntime:       2017-04-07T19:13:56.514Z",
  "requestInfo": {
    "method": "pollTaskUrls",
    "params": {
      "provisionerId": "scl3-puppet",
      "workerType": "gecko-t-win7-32-hw"
    },
    "payload": {},
    "time": "2017-04-07T19:13:56.514Z"
  }
}

goroutine 5 [running]:
panic(0x8e2680, 0x1229a070)
        /home/travis/go/src/runtime/panic.go:500 +0x331
main.SignedURLsManager.func1()
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:35 +0x133
main.SignedURLsManager.func2(0x12272100, 0x12247554, 0x1224da20, 0x12233348, 0x1
2233358)
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:61 +0x92
created by main.SignedURLsManager
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:73 +0x1e0


Then I tried the one from the link in comment 3. I didn't expect it to work, but at least I got a different error. project/releng/generic-worker/os-x/production:    


2017/04/07 19:10:50 Detected windows platform
2017/04/07 19:10:50 No previous task user desktop, so no need to close any open
desktops
2017/04/07 19:10:50 Trying to remove directory 'C:\task_1491591963' via os.Remov
eAll(path) call as GenericWorker user...
2017/04/07 19:10:50 Created dir: C:\task_1491592250\public\logs
panic: (Permanent) HTTP response code 403
HTTP/1.1 403 Forbidden
Content-Length: 1284
Access-Control-Allow-Headers: X-Requested-With,Content-Type,Authorization,Accept
,Origin
Access-Control-Allow-Methods: OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT
Access-Control-Allow-Origin: *
Access-Control-Request-Method: *
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Fri, 07 Apr 2017 19:10:49 GMT
Etag: W/"qa59FycAoiiKASNoMXonnw=="
Server: Cowboy
Strict-Transport-Security: max-age=7776000
Via: 1.1 vegur
X-Powered-By: Express

{
  "code": "InsufficientScopes",
  "message": "You do not have sufficient scopes. This request requires you\nto h
ave one of the following sets of scopes:\n[\n  [\n    \"queue:poll-task-urls\",\
n    \"assume:worker-type:scl3-puppet/gecko-t-win7-32-hw\"\n  ],\n  [\n    \"que
ue:poll-task-urls:scl3-puppet/gecko-t-win7-32-hw\"\n  ]\n]\n\nYou only have the
scopes:\n[\n  \"assume:worker-id:*\",\n  \"assume:worker-type:scl3-puppet/os-x-*
\",\n  \"auth:sentry:generic-worker\",\n  \"auth:statsum:generic-worker\",\n  \"
queue:claim-task\",\n  \"queue:claim-work:scl3-puppet/*\",\n  \"queue:create-art
ifact:*\",\n  \"queue:pending-tasks:scl3-puppet/*\",\n  \"queue:poll-task-urls\"
,\n  \"queue:resolve-task\",\n  \"queue:worker-id:*\"\n]\n\nIn other words you a
re missing scopes from one of the options:\n * Option 0:\n    - \"assume:worker-
type:scl3-puppet/gecko-t-win7-32-hw\"\n * Option 1:\n    - \"queue:poll-task-url
s:scl3-puppet/gecko-t-win7-32-hw\"\n----\nmethod:     pollTaskUrls\nerrorCode:
InsufficientScopes\nstatusCode: 403\ntime:       2017-04-07T19:10:49.553Z",
  "requestInfo": {
    "method": "pollTaskUrls",
    "params": {
      "provisionerId": "scl3-puppet",
      "workerType": "gecko-t-win7-32-hw"
    },
    "payload": {},
    "time": "2017-04-07T19:10:49.553Z"
  }
}

goroutine 18 [running]:
panic(0x8e2680, 0x1229b670)
        /home/travis/go/src/runtime/panic.go:500 +0x331
main.SignedURLsManager.func1()
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:35 +0x133
main.SignedURLsManager.func2(0x1222ff00, 0x122494e4, 0x1224f7c0, 0x12235300, 0x1
2235310)
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:61 +0x92
created by main.SignedURLsManager
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:73 +0x1e0
Flags: needinfo?(rthijssen)
Flags: needinfo?(dustin)
I think you'll need to create a new clientId and accessToken pair.  The clientId and accessToken used in AWS is temporary (and specific to the worker and won't work without the corresponding certificate), and the clientId and accessToken for OS X does not have the scopes for this workerType.

You'll need a client with assume:worker-type:scl3-puppet/gecko-t-win7-32-hw

I've emailed you such a client, with clientId project/releng/worker/scl3-puppet/gecko-t-win7-32-hw
Flags: needinfo?(rthijssen)
Flags: needinfo?(dustin)
The clientID and accessToken worked. Thank you!
During IRC conversation Dustin added queue:worker-id:scl3/t-w864-ix-* to the scopes.
Dustin: Do you know which timestamp this is having an issue with? 

2017/04/11 23:00:53 Detected windows platform
2017/04/11 23:00:53 No previous task user desktop, so no need to close any open
desktops
2017/04/11 23:00:53 Trying to remove directory 'C:\task_1491878647' via os.Remov
eAll(path) call as GenericWorker user...
2017/04/11 23:00:53 Created dir: C:\task_1491951653\public\logs
panic: (Permanent) HTTP response code 401
HTTP/1.1 401 Unauthorized
Content-Length: 417
Access-Control-Allow-Headers: X-Requested-With,Content-Type,Authorization,Accept
,Origin
Access-Control-Allow-Methods: OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT
Access-Control-Allow-Origin: *
Access-Control-Request-Method: *
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Tue, 11 Apr 2017 16:00:53 GMT
Etag: W/"1a1-e6b284fa"
Server: Cowboy
Strict-Transport-Security: max-age=7776000
Via: 1.1 vegur
Www-Authenticate: hawk
X-Powered-By: Express

{
  "code": "AuthenticationFailed",
  "message": "Unauthorized: Stale timestamp\n----\nmethod:     pollTaskUrls\nerr
orCode:  AuthenticationFailed\nstatusCode: 401\ntime:       2017-04-11T16:00:53.
474Z",
  "requestInfo": {
    "method": "pollTaskUrls",
    "params": {
      "provisionerId": "scl3-puppet",
      "workerType": "gecko-t-win7-32-hw"
    },
    "payload": {},
    "time": "2017-04-11T16:00:53.474Z"
  }
}

goroutine 5 [running]:
panic(0x8e2680, 0x12105920)
        /home/travis/go/src/runtime/panic.go:500 +0x331
main.SignedURLsManager.func1()
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:35 +0x133
main.SignedURLsManager.func2(0x12132140, 0x121054a4, 0x1210da00, 0x120f3348, 0x1
20f3358)
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:61 +0x92
created by main.SignedURLsManager
        /home/travis/gopath/src/github.com/taskcluster/generic-worker/signedurlm
anager.go:73 +0x1e0
Flags: needinfo?(dustin)
It's the timestamp in the authorization header.  Usually that means the clock on the client is off.  The server is correct (1600 UTC), but hte client seems to think it's 2300, which is not the same as 1600UTC in any of our common timezones.
Flags: needinfo?(dustin)
I have been able to get the machine pick up and pass random tests.  Currently I am working and hashing the rest of the generic worker support, so that the machine runs a tests reboots and picks up another task. 

Rob: jmaher: Instead of picking random tests are there specific tests I should start the testing process? I am trying figure out a direction to move in and find the point in which we can say the Win 7 hardware configuration is good and put it into production.
Flags: needinfo?(rthijssen)
Flags: needinfo?(jmaher)
for windows hardware, we will be running:
mochitest-clipboard
talos *

we still run:
mochitest-chrome-{1|2|3}, but hopefully not for long.


I have fairly high confidence that if talos run mochitest-clipboard/chrome will run- talos jobs are the performance jobs.  do you know how to do those?  Is this setup on try server, or do you manually copy commands and run those?
Flags: needinfo?(jmaher)
Currently I am going into treeherder, selecting inspect task, edit the task updating the provisioner and workertype fields, and then retrigger the task. Which probably won't work for the above mentioned tests.
correct that we don't really run those- but you could get them running on try, let them fail- then run them on hardware :)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8cd864ce89551a016b6b1748b498106d87cc5d7c&filter-searchStr=clipboard%20tc&filter-tier=1&filter-tier=2&filter-tier=3

lets start with that- if it goes well update the bug and I will hack on talos tests on try via win7-vm tc
and for talos:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8376d4193a9ad27df128eeeb4c2187365239832c

they are all orange and will be orange as there is a different log parser between buildbot and taskcluster- but if they generate a PERFHERDER_DATA object in the log file, then all is well- I am particular about the 'x' job and the 'g1' job- all would be good to validate, but those are probably the most concerning.
Blocks: 1366828
I am doing a fresh try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4f340dd78273387a31ca8cc880a951147f29bd31

waiting on build/tests to run, but this should give us fresh data to do retriggers or more experiments.
Depends on: 1371780
Test thus far:

test-windows7-32/opt-talos-g1-e10s
https://tools.taskcluster.net/groups/cQgRxZg_TdKguXSPp1IpiQ/tasks/cQgRxZg_TdKguXSPp1IpiQ/details
43 minutes

test-windows7-32/opt-talos-g2-e10s
https://tools.taskcluster.net/groups/Prv745orQp-tx4ptTyfEHg/tasks/Prv745orQp-tx4ptTyfEHg/details
1 hour

test-windows7-32/opt-talos-g4-e10s
https://tools.taskcluster.net/groups/JWZO-ShiRJWcF-IvG4Djuw/tasks/JWZO-ShiRJWcF-IvG4Djuw/details
12 minutes

Complete:
test-windows7-32/opt-talos-perf-reftest-e10s
https://tools.taskcluster.net/groups/OYWNvuvuRc2vJ-zEYdJqGQ/tasks/OYWNvuvuRc2vJ-zEYdJqGQ/details
3 minutes

test-windows7-32/opt-talos-other-e10s
https://tools.taskcluster.net/groups/bwPuaFIMRMujVTjlDDSF4Q/tasks/bwPuaFIMRMujVTjlDDSF4Q/details
12 minutes

test-windows7-32/opt-talos-perf-reftest-singletons-e10s
https://tools.taskcluster.net/groups/fBwOJjKZSQeDwohNF2oNqA/tasks/fBwOJjKZSQeDwohNF2oNqA/details
3 minutes

test-windows7-32/opt-talos-svgr-e10s
https://tools.taskcluster.net/groups/d2KdEfenQsyp_mWRBuf1HA/tasks/d2KdEfenQsyp_mWRBuf1HA/details
23 minutes

Failed:
test-windows7-32/opt-talos-g5-e10s
https://tools.taskcluster.net/groups/Dh_rvKb-R-CRwEg2kddgUA/tasks/Dh_rvKb-R-CRwEg2kddgUA/details


After test-windows7-32/opt-talos-svgr-e10s the VM became unresponsive. It was caused by what seemed to be a need reboot for the xen tools install.
as a note, those runtimes look fairly decent!  I would need a larger data set to compare the reported values, but it sounds like that will need to be saved for another day.
Move the installation of xen tools to start up script triggered after MDT is done but before OCC is applied and it has seemed to remedy the issue with the VM becoming unresponsive. I will eventually move that process into OCC. 

Following tests passed: 

test-windows7-32/opt-talos-chrome-e10s
https://tools.taskcluster.net/groups/YJ6gmRP1SnG3-ODxCYB9kA/tasks/YJ6gmRP1SnG3-ODxCYB9kA/details (12 minutes)
https://tools.taskcluster.net/groups/D330BreaTE6WNB7w9Nlakw/tasks/D330BreaTE6WNB7w9Nlakw/details (12 minutes)

test-windows7-32/opt-talos-dromaeojs-e10s 
https://tools.taskcluster.net/groups/G3Ao0ON3SYSgc3TEfkCGOg/tasks/G3Ao0ON3SYSgc3TEfkCGOg/details (22 minutes)
https://tools.taskcluster.net/groups/dImk7V2MSSmD4t_8T_2Imw/tasks/dImk7V2MSSmD4t_8T_2Imw/details (22 minutes)

test-windows7-32/opt-talos-g1-e10s
https://tools.taskcluster.net/groups/BEtN9W8wRxidDm3qfvLIsg/tasks/BEtN9W8wRxidDm3qfvLIsg/details (41 minutes)
https://tools.taskcluster.net/groups/TYLCahTxS8WZvZuMR9mFwg/tasks/TYLCahTxS8WZvZuMR9mFwg/details (41 minutes)

test-windows7-32/opt-talos-g2-e10s
https://tools.taskcluster.net/groups/Tc40MdEOT3ueNi0d0zw-gA/tasks/Tc40MdEOT3ueNi0d0zw-gA/details (40 minutes)

test-windows7-32/opt-talos-g4-e10s
https://tools.taskcluster.net/groups/bB4dzcUbRPK-w0l1G_WNSg/tasks/bB4dzcUbRPK-w0l1G_WNSg/details (12 minutes)

test-windows7-32/opt-talos-g5-e10s
https://tools.taskcluster.net/groups/NXorJo_OSzqhZoBOD_v6Dw/tasks/NXorJo_OSzqhZoBOD_v6Dw/details (32 minutes)

test-windows7-32/opt-talos-other-e10s
https://tools.taskcluster.net/groups/SDZ-6iLYRo-EsiNzBFAQNg/tasks/SDZ-6iLYRo-EsiNzBFAQNg/details (12 minutes)

test-windows7-32/opt-talos-perf-reftest-e10s
https://tools.taskcluster.net/groups/XNxcZULiR82kKFv5Vzs6UQ/tasks/XNxcZULiR82kKFv5Vzs6UQ/details (3 minutes)

test-windows7-32/opt-talos-perf-reftest-singletons-e10s
https://tools.taskcluster.net/groups/U-4J7npbTL6OAh9XVFwxyQ/tasks/U-4J7npbTL6OAh9XVFwxyQ/details (3 minutes)

test-windows7-32/opt-talos-svgr-e10s
https://tools.taskcluster.net/groups/Zd6qfcGSTle1g0WLjoV4Uw/tasks/Zd6qfcGSTle1g0WLjoV4Uw/details (21 minutes)
The failing tests were looking for c:/mozilla-build/python27/python27.dll which is actually at c:/mozilla-build/python/python27.dll .

test-windows7-32/opt-talos-tp5o-e10s
https://tools.taskcluster.net/groups/W3K2iBRXRgaDFteFUiIJJQ/tasks/W3K2iBRXRgaDFteFUiIJJQ/details (32 minutes)
https://tools.taskcluster.net/groups/FJYo4hdGTzWQnt4pKf7-QA/tasks/FJYo4hdGTzWQnt4pKf7-QA/details (32 minutes)
WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502777159\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!

test-windows7-32/opt-talos-tp6-e10s
https://tools.taskcluster.net/groups/AuGLao1eQe-4PBkzFPKqCg/tasks/AuGLao1eQe-4PBkzFPKqCg/details (19 minutes)
16:37:38  WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502798653\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!

test-windows7-32/opt-talos-tp6-stylo-e10s
https://tools.taskcluster.net/groups/Udmw0_VkTMqe38jD3rEK_w/tasks/Udmw0_VkTMqe38jD3rEK_w/details  (1 minute)
16:41:32  WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502815107\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!

test-windows7-32/opt-talos-tp6-stylo-threads-e10s
https://tools.taskcluster.net/groups/PZ_RdAzJQAKq4q27RR1_mg/tasks/PZ_RdAzJQAKq4q27RR1_mg/details (1 minute)
16:44:53  WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502815337\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!

test-windows7-32/opt-talos-xperf-e10s
https://tools.taskcluster.net/groups/brjtw071QuWmPAC3kdS9JQ/tasks/brjtw071QuWmPAC3kdS9JQ/details (2 minutes)
16:48:48  WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502815540\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!

test-windows7-32/opt-talos-xperf-e10s
https://tools.taskcluster.net/groups/QXXARXgpQle-xsgQemp1TQ/tasks/QXXARXgpQle-xsgQemp1TQ/details (2 minutes)
16:52:52  WARNING - Can't copy c:/mozilla-build/python27/python27.dll to C:\Users\task_1502815776\build\venv\Scripts\python27.dll: [Errno 2] No such file or directory: 'c:/mozilla-build/python27/python27.dll'!
odd about this common error.  tp5o shouldn't have issues like that unless this is related to how it collects counters from the OS.  I could maybe believe the tp6* tests as they load up python3.0 in addition to python2.7 for a specific tool.

Laslty xperf might have issues with accessing different OS binaries for running the xperf toolchain (assuming it is installed)

Given that this is a path only issue, it might be related to:
http://searchfox.org/mozilla-central/source/testing/mozharness/configs/talos/windows_config.py#5
TaskCluster pieces are functioning and QA has a test pool.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.