Bugzilla

we haven't yet found a resolution to get the yogas taking tasks. the machines are mostly up and running (with the exception of 002, 004, 005, 008, 013, 019, 028, 030, 031, 032, 035, which i have asked bitbar to reboot).

the working machines logs can be seen at: https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630
the patches described in comment 2 are triggered here: https://github.com/mozilla-releng/OpenCloudConfig/blob/master/userdata/MaintainSystem.ps1#L146-L164
maintain system runs at each instance boot and invokes the patches in the gist. for instances that have already been patched, they don't do much more than just [log their state](https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630 Invoke-OccReset). when the instances in the first line of this comment come online, they will begin to patch themselves as well however, they won't be fully operational until their public keys have been added to the [encryption list](https://github.com/mozilla-releng/OpenCloudConfig/tree/master/keys) allowing them to collect their patched and encrypted gw configs. i will monitor for these instances coming to life when bitbar reboots them and add their keys.

the elephant in the room is that even once they are patched, they won't take tasks until we understand what is causing [the generic-worker panic](https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630%20program%3Ageneric%20%28panic%20OR%20%22WTSQueryUserToken%3A%20An%20attempt%20was%20made%20to%20reference%20a%20token%20that%20does%20not%20exist%22%20OR%20exiting%29).

my suspicion is that arm64 windows instances have installed a windows update which prevents the gw task user creation routine from running as intended and as used to work fine. if this suspicion is correct, the only solutions i can think of are to somehow roll back the windows update that caused the problem or to patch generic-worker to use a different mechanism when creating task users. of course, we first need to establish what is causing the problem and if it was actually a windows update.

we haven't yet found a resolution to get the yogas taking tasks. the machines are mostly up and running (with the exception of 002, 004, 005, 008, 013, 019, 028, 030, 031, 032, 035, which i have asked bitbar to reboot).

the working machines logs can be seen at: https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630
the patches described in comment 2 are triggered here: https://github.com/mozilla-releng/OpenCloudConfig/blob/master/userdata/MaintainSystem.ps1#L146-L164
maintain system runs at each instance boot and invokes the patches in the gist. for instances that have already been patched, they don't do much more than just [log their state](https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630%20Invoke-OccReset). when the instances in the first line of this comment come online, they will begin to patch themselves as well however, they won't be fully operational until their public keys have been added to the [encryption list](https://github.com/mozilla-releng/OpenCloudConfig/tree/master/keys) allowing them to collect their patched and encrypted gw configs. i will monitor for these instances coming to life when bitbar reboots them and add their keys.

the elephant in the room is that even once they are patched, they won't take tasks until we understand what is causing [the generic-worker panic](https://my.papertrailapp.com/events?q=system%3At-lenovoyogac630%20program%3Ageneric%20%28panic%20OR%20%22WTSQueryUserToken%3A%20An%20attempt%20was%20made%20to%20reference%20a%20token%20that%20does%20not%20exist%22%20OR%20exiting%29).

my suspicion is that arm64 windows instances have installed a windows update which prevents the gw task user creation routine from running as intended and as used to work fine. if this suspicion is correct, the only solutions i can think of are to somehow roll back the windows update that caused the problem or to patch generic-worker to use a different mechanism when creating task users. of course, we first need to establish what is causing the problem and if it was actually a windows update.