puppet should manage ccache size on r5 osx builders

RESOLVED FIXED

Status

RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: catlee, Assigned: kmoir)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Assignee: jhford → kmoir
(Assignee)

Comment 1

6 years ago
Created attachment 634383 [details] [diff] [review]
patch

Not sure where the DMGs for PuppetAgain should reside, guessed puppet:///repos/DMGs/ would be a good place
(Assignee)

Updated

6 years ago
Attachment #634383 - Flags: review?(bugspam.Callek)
Comment on attachment 634383 [details] [diff] [review]
patch

feedback- for the mere fact that it doesn't do what this bug is filed for (I can certainly review for 10.8 machines though. -- and I can see how you could have got mixed up)

This bug is about the r5 builders, which are defined in the (old) puppet at http://hg.mozilla.org/build/puppet-manifests and is not on the same puppetmaster as our "PuppetAgain" work.

Also this code for 10.8 of course does not manage the ccache size, and I don't *think* we have the current plan to need ccache on our 10.8 machines.
Attachment #634383 - Flags: feedback-
(Assignee)

Comment 3

6 years ago
Yes, I misread the intent of the bug.  The ccache dmg is installed on Lion machines by the modules in the puppet-manifests repo.  However, this isn't needed for 10.8 machines because they are just test machines,  not builders.
Comment on attachment 634383 [details] [diff] [review]
patch

Review of attachment 634383 [details] [diff] [review]:
-----------------------------------------------------------------

Kim, is this patch still relevant? I note again that this bug was relating to the old puppet system/repo. But if it is relevant for PuppetAgain, I'll hand it over to dustin
Attachment #634383 - Flags: review?(bugspam.Callek) → review?(dustin)
(Assignee)

Comment 5

6 years ago
No I don't think this patch is still relevant.  I'll make a new one.
Attachment #634383 - Flags: review?(dustin)
(Reporter)

Updated

6 years ago
Blocks: 775305
(Assignee)

Comment 6

6 years ago
Created attachment 644343 [details] [diff] [review]
patch

I've tested this on the staging server with a r5 mac slave and it works.  The linux machines currently have their ccache maximum set to 5G.

notice: //Node[build]/base/osx_lion/Exec[increasemaxccachesize]/returns: executed successfully
notice: Finished catalog run in 6.59 seconds
r5-mini-006:~ cltbld$  /usr/local/bin/ccache -s
cache directory                     /builds/ccache
cache hit (direct)                     0
cache hit (preprocessed)               0
cache miss                             0
files in cache                     19182
cache size                         929.4 Mbytes
max cache size                       5.0 Gbytes
r5-mini-006:~ cltbld$ env | grep CCACHE
CCACHE_DIR=/builds/ccache
Attachment #634383 - Attachment is obsolete: true
Attachment #644343 - Flags: review?(catlee)
(Reporter)

Comment 7

6 years ago
Comment on attachment 644343 [details] [diff] [review]
patch

Review of attachment 644343 [details] [diff] [review]:
-----------------------------------------------------------------

::: os/osx_lion.pp
@@ +204,5 @@
> +               command=> "/usr/local/bin/ccache -M 5G",
> +               environment => ["CCACHE_DIR=/builds/ccache", "CCACHE_COMPRESS=1"],
> +               user => "cltbld",
> +               group => "admin",
> +               require => [ Package["ccache-3.1.7.dmg"], File["/builds/ccache"]];

nit: looks like there's some indentation misalignment here?

this should be at least 10G. These machines have 1T of space, so no need to skimp out here.

r+ with fixed indentation and MOAR SPACE!
Attachment #644343 - Flags: review?(catlee) → review+
(Assignee)

Comment 8

6 years ago
Updated patch http://hg.mozilla.org/build/puppet-manifests/rev/cdaaf3ac8b02 landed and puppet servers updated.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED

Updated

6 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 9

6 years ago
Created attachment 737625 [details] [diff] [review]
fix ccache issue

For some reason this was causing even production machines to output an error.
I don't know why would the machines still initialize buildbot if puppet would fail:

You can see it on the log:
bld-lion-r5-041:~ root# /usr/bin/puppetd --onetime --no-daemonize --logdest console   --server scl3-production-puppet.srv.releng.scl3.mozilla.com
notice: Starting catalog run
notice: //Node[build]/base/osx_lion/buildslave::cleanup/Exec[find /tmp/* -mmin +15 -print | xargs -n1 rm -rf]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[set-time-server]/returns: executed successfully
notice: //Node[build]/ntp/File[/etc/ntp.conf]/checksum: checksum changed '{md5}454a1f30f9c7ec4222bdc2c83abdaec3' to '{md5}c086d264d534ecb48ad1221839c37583'
notice: //Node[build]/ntp/File[/etc/ntp.conf]/content: content changed '{md5}c086d264d534ecb48ad1221839c37583' to '{md5}454a1f30f9c7ec4222bdc2c83abdaec3'
notice: //Node[build]/ntp/Exec[restart-ntp]: Triggering 'refresh' from 2 dependencies
notice: //Node[build]/base/osx_lion/Exec[turn-off-sleep]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[remove-index]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[turn-on-auto-power]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disallow-sleep-button]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-indexing]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[verify-xcode]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[restart-on-panic]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-updater]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[turn-on-wol]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[clean-saved-state]/returns: executed successfully
err: //Node[build]/base/osx_lion/Exec[setmaxccachesize]/returns: change from notrun to 0 failed: /usr/local/bin/ccache -M 10G returned  instead of 0 at /etc/puppet/manifests/os/osx_lion.pp:210
notice: //Node[build]/base/osx_lion/Exec[disable-bluetooth]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-screensaver]/returns: executed successfully
notice: Finished catalog run in 8.06 seconds
bld-lion-r5-041:~ root# /usr/bin/puppetd --onetime --no-daemonize --logdest console   --server scl3-production-puppet.srv.releng.scl3.mozilla.com
notice: Starting catalog run
notice: //Node[build]/base/osx_lion/Exec[set-time-server]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-bluetooth]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-screensaver]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-updater]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[turn-off-sleep]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disable-indexing]/returns: executed successfully
notice: //Node[build]/base/osx_lion/buildslave::cleanup/Exec[find /tmp/* -mmin +15 -print | xargs -n1 rm -rf]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[restart-on-panic]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[turn-on-wol]/returns: executed successfully
notice: //Node[build]/ntp/File[/etc/ntp.conf]/checksum: checksum changed '{md5}454a1f30f9c7ec4222bdc2c83abdaec3' to '{md5}c086d264d534ecb48ad1221839c37583'
notice: //Node[build]/ntp/File[/etc/ntp.conf]/content: content changed '{md5}c086d264d534ecb48ad1221839c37583' to '{md5}454a1f30f9c7ec4222bdc2c83abdaec3'
notice: //Node[build]/ntp/Exec[restart-ntp]: Triggering 'refresh' from 2 dependencies
notice: //Node[build]/base/osx_lion/Exec[turn-on-auto-power]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[verify-xcode]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[setmaxccachesize]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[disallow-sleep-button]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[remove-index]/returns: executed successfully
notice: //Node[build]/base/osx_lion/Exec[clean-saved-state]/returns: executed successfully
notice: Finished catalog run in 11.53 seconds
Attachment #737625 - Flags: review?(kmoir)
(Assignee)

Updated

6 years ago
Attachment #737625 - Flags: review?(kmoir) → review+
Puppet's exit statuses weren't very good back then.
(Assignee)

Updated

5 years ago
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.