992378 - Need a bld-lion-r5 to test build times with SSD

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Jake Watkins [:dividehex]]

We will be replacing the non-ssd drives with a single ssd to test build times.

Comment 1

[Jordan Lund (:jlund)]

I have disabled bld-lion-r5-087 and rebooted it.

Looks like you have access to the machine so I won't file a vpn access bug :)

IIUC, If we are replacing the storage with an SSD, we will need to wipe the current non-ssd drive. Should I file an IT bug for that or will I leave that to you?


I am assigning the machine to you. Please mark this bug as resolved and comment that it can go back in production once you are finished with it forever :)

Comment 2

[Jake Watkins [:dividehex]]

(In reply to Jordan Lund (:jlund) from comment #1)
> IIUC, If we are replacing the storage with an SSD, we will need to wipe the
> current non-ssd drive. Should I file an IT bug for that or will I leave that
> to you?

If it is decided the ssd drive is a permanent replacement, DCOPs will properly dispose of the non-ssd drives.  They have internal procedures for that. 

Thanks for that loaner :}

Comment 3

[Massimo Gervasini [:massimo]]

Hi dividehex,

do you have any data about the build times on mac with the ssd?

Comment 4

[Jake Watkins [:dividehex]]

(In reply to Massimo Gervasini [:mgerva] from comment #3)
> Hi dividehex,
> 
> do you have any data about the build times on mac with the ssd?

Hi :mgerva,
Getting build times is not part of my work flow.  That would fall on someone in releng.  I will be running benchmarking tests on it today. But, we should find out who should be doing the build time testing also.

Comment 5

[Jake Watkins [:dividehex]]

:mgerva,  I'm going to turn this over to you for testing of build times until the benchmarking tests in bug1001482 are defined.  Please send this slave back to me when you are done.
Thanks

Comment 6

[Massimo Gervasini [:massimo]]

Attachment: [buildbot-configs] Bug 992378 - add bld-lion-r5-087 to try; testing build times with SSD.patch

Adding bld-lion-r5-087 to try

Comment 7

[Chris Cooper [:coop] (he/him)]

Merged and deployed to production.

Comment 8

[Massimo Gervasini [:massimo]]

bld-lion-r5-087 back is in prod/try - waiting for it to get some jobs

Comment 9

[Aki Sasaki (not active)]

In production

Comment 10

[Chris AtLee [:catlee]]

Timed out after 4h30 in the compile step in both its jobs it took today. - disabled in slavealloc.

Comment 11

[Massimo Gervasini [:massimo]]

This machine timed out on 3 'compile' steps in a row. I have done some quick checks: memory, disk space and general settings look ok. I have no idea why this machine spends so much time on this step. 

dividehex, was there any change at the operating system level when we installed the ssd?

Comment 12

[Jake Watkins [:dividehex]]

I not sure why compile times would be longer in this case.  As far as I can tell, TRIM is enabled.  It was done here - https://bugzilla.mozilla.org/show_bug.cgi?id=992364#c4

System profiler also reports it being enabled.  The only other thing I can think of is the root partition being misaligned but that is rather unlikely since diskutil should be smart enough to align properly.

I did find that fio had left a lot (100k+) of empty file on the disk but still not sure if that would have caused the compile to timeout.  They have been deleted.


        Samsung SSD 840 EVO 500GB:

          Capacity: 500.11 GB (500,107,862,016 bytes)
          Model: Samsung SSD 840 EVO 500GB
          Revision: EXT0BB6Q
          Serial Number: S1DHNSAF312828W
          Native Command Queuing: Yes
          Queue Depth: 32
          Removable Media: No
          Detachable Drive: No
          BSD Name: disk0
          Medium Type: Solid State
--->      TRIM Support: Yes
          Bay Name: Upper
          Partition Map Type: GPT (GUID Partition Table)
          S.M.A.R.T. status: Verified
          Volumes:
            disk0s1:
              Capacity: 209.7 MB (209,715,200 bytes)
              BSD Name: disk0s1
              Content: EFI
            Macintosh HD:
              Capacity: 499.76 GB (499,763,888,128 bytes)
              Available: 471.82 GB (471,819,001,856 bytes)
              Writable: Yes
              File System: Journaled HFS+
              BSD Name: disk0s2
              Mount Point: /
              Content: Apple_HFS

Comment 13

[Chris AtLee [:catlee]]

As a comparison, existing machines complete 95% of these type of jobs in under an hour, and 50% in less than 30 minutes.

Comment 14

[Mike Hommey [:glandium]]

(In reply to Chris AtLee [:catlee] from comment #13)
> As a comparison, existing machines complete 95% of these type of jobs in
> under an hour, and 50% in less than 30 minutes.

But they use sccache.

Comment 15

[Justin Wood (:Callek)]

status? [its a loan-bug with no update for > a month :-) ]

Comment 16

[Amy Rich [:arr] [:arich]]

Project was dropped by relops for Q2 because the results were not as anticipated (and tracking down why requires additional work from both relops and releng). I asked if a releng intern wanted to pick it up and haven't heard a definitive yes/no.

Comment 17

[Chris AtLee [:catlee]]

Shall we restore the original disk, and toss it back into the production pool then?

Comment 18

[Amy Rich [:arr] [:arich]]

catlee: if that's how you'd like to proceed, sure, we can do that. Dcops should still have the drives to put back in and we can change the imaging profile so that it's the same as the others. Let me know if this is definitely the path you'd like to take.

Comment 19

[Amy Rich [:arr] [:arich]]

Per catlee: we're going to put the disks back into this and turn it into a mountain lion test machine.

Comment 20

[Amy Rich [:arr] [:arich]]

Machine has been repurposed. Coop: you should add this one in with the other 10 that just got reimaged.