Disable finder, etc. on OSX machines

RESOLVED INCOMPLETE

Status

task
RESOLVED INCOMPLETE
4 years ago
a year ago

People

(Reporter: catlee, Unassigned)

Tracking

Details

Attachments

(1 attachment)

Reporter

Description

4 years ago
I was on an OSX build slave and wondered why it was sluggish. It looks like there's a service called 'fseventsd' taking up quite a bit of CPU time.
Reporter

Comment 2

4 years ago
[root@bld-lion-r5-025.try.releng.scl3.mozilla.com ~]# find /.fseventsd/ -type f | wc -l
  295503
[root@bld-lion-r5-025.try.releng.scl3.mozilla.com ~]# du -ms /.fseventsd/
11708   /.fseventsd/
Reporter

Comment 3

4 years ago
also noticed this:

Jun 12 13:15:47 bld-lion-r5-025 fseventsd[34]: Logging disabled completely for device:1: /Volumes/Boot OS X

Looks like '/Volumes/Boot OS X' is mounted for some time after boot, and then unmounted....

maybe from this:

Jun 12 06:37:20 bld-lion-r5-027 com.apple.kextcache: / locked; waiting for lock.
Jun 12 06:37:33 bld-lion-r5-027 com.apple.kextcache: Created prelinked kernel /System/Library/Caches/com.apple.kext.caches/Startup/kernelcache.
Jun 12 06:37:33 bld-lion-r5-027 com.apple.kextcache: Lock acquired; proceeding.
Jun 12 06:37:33 bld-lion-r5-027 com.apple.kextcache: /System/Library/Caches/com.apple.kext.caches/Startup/kernelcache out of date.
Jun 12 06:37:36 bld-lion-r5-027 fseventsd: Logging disabled completely for device:1: /Volumes/Boot OS X
Jun 12 06:37:57 bld-lion-r5-027 com.apple.kextcache: Successfully updated helper partition disk1s3.
Jun 12 06:37:59 bld-lion-r5-027 fseventsd: Logging disabled completely for device:1: /Volumes/Boot OS X
Jun 12 06:38:16 bld-lion-r5-027 com.apple.kextcache: Successfully updated helper partition disk0s3.
Reporter

Comment 4

4 years ago
opendirectoryd also seems to take a lot of CPU from time to time
Reporter

Comment 5

4 years ago
I've had this running on bld-lion-r5-025 for a few days. I'm not 100% sure if it's working, but it doesn't seem to be hurting either.

I'd like to deploy this and let it bake for a few days and see if it makes a meaningful dent in build times.
Attachment #8622664 - Flags: review?(dustin)
Assignee: nobody → catlee
Attachment #8622664 - Flags: review?(dustin) → review+
Reporter

Updated

4 years ago
Attachment #8622664 - Flags: checked-in+
Reporter

Comment 6

4 years ago
This didn't seem to make any difference. We should back this out.
I can't find that there's any way to globally disable fseventsd. That's a bummer.
Reporter

Comment 8

4 years ago
so I took a look at fseventds time across a bunch of our machines, and there's a very strong correlation with total directory size. I think we could see improvement by purging data out of this directory, on boot, or maybe as part of runner
Reporter

Comment 9

4 years ago
my hypothesis here is that we do a lot of operations that generate a huge amount of filesystem events that need to be logged (e.g. clobbers). as the directory grows, the OS needs to spend more time adding/removing entries from the growing directory file, and then writing it back to disk.

the directory sizes for /.fseventsd on our build machines range from 

28,084 to 9,936,670 bytes, with the median being 1,773,678 bytes

NB this is the size of the directory file itself (ls -ld /.fseventsd), not the size of the directory's contents.

I'm purging entries older than 7 days, and will see if this improves fseventsd time.
Reporter

Comment 10

4 years ago
prior to purging old entries, the median total cpu time of fseventsd was 488s over 3,696 samples.
post purging, the median time is 172s, over 11,284 samples.

I'd like to purge this directory regularly, either as part of runner or puppet.
I wonder if we could file a radar bug with Apple to find out if there's a way to disable fseventsd or limit the amount of data it saves?
Reporter

Comment 12

3 years ago
Subsequent cleanups didn't seem to have the same impact on cpu time. I'm not really sure what's going on here, but I can't fix it.
Assignee: catlee → nobody
Reporter

Updated

a year ago
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → INCOMPLETE
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.