1328640 - Add hardware support to the OpenCloudConfig

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Comment 1

[Mark Cornmesser [:markco]]

Currently trying to get this to work through he opencloudconfig located at https://github.com/mozilla-releng/OpenCloudConfig

The list of prerequisites thus far:
Framework management 4.5 
.net Framework 4.0
Powershell 5.0 
And create a firewall rule allowing Windows Remote Management HTTP-In.

Comment 2

[Mark Cornmesser [:markco]]

I have hit snag on running rundsc.ps1. It errors out when it goes to run  DynamicConfig.ps1:

PSDesiredStateConfiguration\Configuration : Unable to connect to the remote server
At C:\Users\Administrator\AppData\Local\Temp\DynamicConfig.ps1:8 char:1
+ Configuration DynamicConfig {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Configuration], WebExcepti
   on
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Configuration

Which results to the following message in the event viewer: 

Log Name:      System
Source:        Microsoft-Windows-WinRM
Date:          1/30/2017 5:38:24 PM
Event ID:      10148
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      T-W864-IX-011
Description:
The WinRM service is listening for WS-Management requests. 

 User Action 
 Use the following command to see the specific IPs on which WinRM is listening: 

 winrm enumerate winrm/config/listener

However, the winrm is is listening and listening on what appears to be the correct IPs:

C:\Users\Administrator>winrm quickconfig
WinRM service is already running on this machine.
WinRM is already set up for remote management on this computer.


C:\Users\Administrator>winrm enumerate winrm/config/listener
Listener
    Address = *
    Transport = HTTP
    Port = 5985
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 10.26.40.41, 127.0.0.1, 169.254.145.172, ::1, fe80::5efe:10.26
.40.41%12, fe80::28c3:6c19:3589:c5f1%11, fe80::f57f:e461:16e3:91ac%13


Q: Grenade: DO you guys have any ideas on what could cause this kind of behavior? Any suggestions on how to move forward?

Comment 3

[Mark Cornmesser [:markco]]

Correction: The WinRM service is not listening for WS-Management requests.

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

couple of thoughts pop into my head.
run `Enable-PSRemoting -Force` in the powershell console. it's supposed to enable stuff. also, is the ps shell an elevated one? it needs to be. i can see that it's the admin account, but it also needs to be an elevated ps console session. failing that, let me know how to connect to the instance and i will attempt to debug live.

Comment 5

[Q]

Worked in IRC to resolve cancelling need info.

Comment 6

[Mark Cornmesser [:markco]]

The "The WinRM service is not listening for WS-Management requests" error was irrelevant. 

(Invoke-WebRequest -Uri 'http://169.254.169.254/latest/meta-data/public-keys' -UseBasicParsing).Content was the line cuasing the error in comment 2.

Comment 7

[Mark Cornmesser [:markco]]

Worked around the issue in comment 6 with a conditional statement based on the existence of the EC2 service.

Also it turns out we do not need to make any changes to the firewall configuration prior to running rundsc.ps1.

Currently I have gotten to a point where dsc is applying to the machine. Now I need to chase down the various errors that occur.

Comment 8

[Mark Cornmesser [:markco]]

Most of the initial errors from the manifest (gecko-t-win7-32-hw.json) regards there not being a y drive. This easily rectify with pointing everything to the C: drive. 

The other notable error:
VERBOSE: [T-W864-IX-011]: LCM:  [ End    Set      ]  [[Script]GpgKeyImport]  in 23.7260 seconds.
PowerShell DSC resource MSFT_ScriptResource  failed to execute Set-TargetResource functionality with error message: 
Exception calling "DownloadString" with "1" argument(s): "Unable to connect to the remote server" 
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : localhost
PowerShell DSC resource MSFT_ScriptResource  failed to execute Set-TargetResource functionality with error message:
Exception calling "DownloadString" with "1" argument(s): "Unable to connect to the remote server"
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : localhost

Comment 9

[Mark Cornmesser [:markco]]

From this code (where "$meta_data = https://github.com/markcor/OpenCloudConfig/tree/master/userdata/meta_data"):
 
 
 
$instancekey = (Invoke-WebRequest -Uri "$meta_data/latest/meta-data/public-keys" -UseBasicParsing).Content
  if ($instancekey.StartsWith('0=aws-provisioner-v1-managed:')) {
    # provisioned worker
    $workerType = $instancekey.Split(':')[1]
  } else {
    # ami creation instance
    $workerType = $instancekey.Replace('0=mozilla-taskcluster-worker-', '')
  }
  Write-host $workerType
  if ($workerType) {
    $manifest = (Invoke-WebRequest -Uri ('https://raw.githubusercontent.com/mozilla-releng/OpenCloudConfig/master/userdata/Manifest/{0}.json?{1}' -f $workerType, [Guid]::NewGuid()) -UseBasicParsing | ConvertFrom-Json)
  } else {
    switch -wildcard ((Get-WmiObject -class Win32_OperatingSystem).Caption) {
      'Microsoft Windows 7*' {
        $manifest = (Invoke-WebRequest -Uri ('https://raw.githubusercontent.com/markcor/OpenCloudConfig/master/userdata/Manifest/gecko-t-win7-32-hw.json?{0}' -f [Guid]::NewGuid()) -UseBasicParsing | ConvertFrom-Json)
      }
      'Microsoft Windows 10*' {
        $manifest = (Invoke-WebRequest -Uri ('https://raw.githubusercontent.com/mozilla-releng/OpenCloudConfig/master/userdata/Manifest/gecko-t-win10-64-hw.json?{0}' -f [Guid]::NewGuid()) -UseBasicParsing | ConvertFrom-Json)
      }
      default {
        $manifest = ('{"Items":[{"ComponentType":"DirectoryCreate","Path":"$env:SystemDrive\\log"}]}' | ConvertFrom-Json)
      }
    }
}
 
 
 
I am getting this error. 
 
PSDesiredStateConfiguration\Configuration : 400: Invalid request
At C:\Users\Administrator\AppData\Local\Temp\DynamicConfig.ps1:8 char:1
+ Configuration DynamicConfig {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Configuration], WebExcepti
   on
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Configuration
Errors occurred while processing configuration 'DynamicConfig'.
At
C:\windows\system32\WindowsPowerShell\v1.0\Modules\PSDesiredStateConfiguration\PSDesiredStateConfiguration.psm1:3705
char:5
+     throw $ErrorRecord
+     ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (DynamicConfig:String) [], InvalidOperationException
    + FullyQualifiedErrorId : FailToProcessConfiguration

Comment 10

[Mark Cornmesser [:markco]]

Attachment: Add support to rundcs.ps1 and DynamicConfig.ps1

I scrapped everything that had to deal with a faux metadata in the datacenter. 

Any thoughts or suggestions

Comment 11

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8836839 [details] [review]
Add support to rundcs.ps1 and DynamicConfig.ps1

loogs good. the only thing i think it needs is a block to set $LocationType in DynamicConfig.ps1 (exactly like the one you have in rundsc.ps1) since the vars aren't shared between the scripts.

Comment 12

[Mark Cornmesser [:markco]]

Will land with: https://github.com/mozilla-releng/OpenCloudConfig/pull/49 (Bug 1338324).

Comment 13

[Sebastian Pacurar [:spacurar]]

I noticed in slave health that t-w864-ix-011 and t-w864-ix-012 machines are disabled from slavealloc, and that in the comments they are both pointing to this bug.
Do you still need the machines for further tests or can I place them back in production ?

Comment 14

[Mark Cornmesser [:markco]]

(In reply to Sebastian Pacurar [:spacurar] from comment #13)
> I noticed in slave health that t-w864-ix-011 and t-w864-ix-012 machines are
> disabled from slavealloc, and that in the comments they are both pointing to
> this bug.
> Do you still need the machines for further tests or can I place them back in
> production ?

We will still needs these for few more weeks.

Comment 15

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Markco do you still need t-w864-ix-011 and t-w864-ix-012, or can we place them back in production?

Comment 16

[Mark Cornmesser [:markco]]

Yes. It may be another week or 2.

Comment 17

[Alin Selagea [:aselagea]]

Just want to check if t-w864-ix-011 and t-w864-ix-012 are still needed :-)

Comment 18

[Mark Cornmesser [:markco]]

(In reply to Alin Selagea [:aselagea][:buildduty] from comment #17)
> Just want to check if t-w864-ix-011 and t-w864-ix-012 are still needed :-)

I am finally done with these machines. I have kicked off a reinstall on both of them.