1257215 - Integrate local ephemeral disks into windows AWS testers

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Q]

1) Add a second device in the config files and verified the device path is correct

2) Modify user data to do the following:
 A) Detect if there is a second disk
  1) If the disk has a drive letter remove it
  2) Make sure c:\slave\test\build is empty folder
     a) If not then delete and create the folder ( with inheratable perms on) 
     b) Then mount the disk under c:\slave\test\build ( this can be done in with a diskpart script or via powershell)

 B) If no second disk exists make sure c:\slave\test\build is empty folder ( not a mount link)

Comment 1

[Q]

This should apply to all testers spot and ec2:
g-w732
t-w732
t-w10
g-w10
t-xp32

Comment 2

[Q]

brute force method with out checking etc:

rmdir /S /Q c:\slave\test\build
mkdir c:\slave\test\build
diskpart /s c:\changedrive.txt

where c:\changedrive.txt contains:
select volume 1
REMOVE LETTER=D
assign mount=C:\slave\test\build

Comment 3

[Q]

Looks like the disktoolks like get-disk, set-disk, etc cmdlets are native to win8/2012 and up so it may need to be a disk-part script that is dynamically built and fired to be cross compatible.

Comment 4

[Q]

It may also be beneficial to test for multiple disks in larger instance types and create a sw raid for this volume before mounting:

http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/raid-config.html#windows-raid

Comment 5

[Q]

We have to roll this into user data. It should be a quick test to see if we have local ephemeral disks attached and if it has a drive letter assigned or not. Then make a diskpart script and fire it. Right now is should only fire for testers.

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: https://github.com/mozilla/build-cloud-tools/pull/202

> Tested on Server 2012. To test on Win 7, etc (from a powershell prompt):
> New-Item -ItemType Directory -Force -Path C:\slave\test\build
> Set-ExecutionPolicy RemoteSigned -f
> Invoke-Expression (New-Object Net.WebClient).DownloadString('https://gist.github.com/grenade/17751e9ce0e3e6656d19')

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

> test script should have read:

> New-Item -ItemType Directory -Force -Path C:\slave\test\build
> Set-ExecutionPolicy RemoteSigned -f
> Invoke-Expression (New-Object Net.WebClient).DownloadString('https://gist.githubusercontent.com/grenade/17751e9ce0e3e6656d19/raw/9204bcbee48fb46a0fe63a2136df14c328ef3a86/test-mount.ps1')

Comment 8

[Q]

Test script fails on windows 7/ xp due to commandlets missing. Trying to upgrade powershell on windows 7 via chocolaty to ge the commandlets ran into issues with the fact that chocolaty rewraps an msu file which requires windows update to be enabled to install. We disable windows update on testers. After temporarily enabling windows update service  the msu was able to install but not upgrade power shell due to dependencies on a management framework upgrade which conflicts with several other installs on windows 7. Powershell will not upgrade on windows XP which will need the same tool. Unfortunately it looks like we will need legacy tool support in the scripts.

Comment 9

[Rob Thijssen [:grenade (EET/UTC+0300)]]

understood, will try again/harder...

Comment 10

[Q]

Per conversations in IRC. If and only if upgrades to powershell will not affect sub systems needed for tests an upgrade will be done to accommodate powershell native scripts. However,this will strand XP use in the cloud. If XP is needed I will work on a broader tool set that will accommodate backward compatibility.

Comment 11

[Q]

For documentation purposes:

Testers will mount the disks under C:\slave\test\build
and builders will mount under c:\build and require either recopy or symlinking of secrets files located there.

Comment 12

[Q]

grenade, any thoughts on adding raid 0 support if multiple ephemeral disks are attached?

Comment 13

[Amy Rich [:arr] [:arich]]

We will not put XP in the cloud, so this is not a concern.

Comment 14

[Q]

Given the track record of "we will kill XP" at Mozilla I will hold that if it is needed I will work on the tolls to do so but put no effort in until then.

Comment 15

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: https://github.com/mozilla/build-cloud-tools/pull/202

rewrite using diskpart. tested on:
- win 7 with 1 ephemeral disk (for some reason ec2 wouldn't let me have 2)
- win 2012 with 2 ephemeral disks (3 disks, 4 volumes: sys, C:, Y:, Z: were converted to 3 volumes: sys, C:, C:\mnt)

Comment 16

[Q]

Wow thanks for the quick work! 

As far as the disks on win 7 vs 2008:
gp2.2xl instances only have one ephemeral disk but c3.2xl instances have 2 the aws meta data should have how many disks are available.

Comment 17

[Q]

(In reply to Rob Thijssen (:grenade - GMT) from comment #15)
I will put notes on request but just quick in for here for my own reference: 

This static mounts to c:\mnt which can be quick fixed
for some reason str.Tolower.StartsWith('str') doesn't work however str.StartsWith('str', 1) does work

If the drive does not have a drive letter ( a real possibility) the script fails because it is limiting the wmi query to include drive letters  and diskpart will fail on the  "remove all dismount" step causing the whole thing to fail.

grenade's patch addresses the volume length check logic issues:

https://github.com/MozRelOps/OpenCloudConfig/commit/78c75cfd91e75f452980dff6844d01a5227f08d6#diff-fcc5ddda5838b242143a899a3fe5481eL20