572333 - Linux Puppet slaves are pulling down all RPMs on every boot

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Nick discovered that despite attempts to avoid it, all of our Linux machines are pulling down all of the RPMs every time they run. This is happening because the package {} check in functions.pp checks for existence of the package on every run. I tried to avoid this by making it require the "check-for-$name" check before running the package, but that doesn't prevent the package {} check from running as I thought it did.

Using ensure => installed instead of ensure => latest on the package {} check may fix this. If that doesn't work we can try messing with the slave-side Puppet code.

Comment 1

[[e-mail] (:bhearsum)]

ensure => latest is the culprit here.

ensure => installed works around the issue, but doesn't allow us to upgrade.

ensure => $version_number worked well for both not downloading on each run and allowing upgrades to happen, in my limited test. I'm going to do a more extensive test before posting a patch.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: fix rpm checks

Here's the fix, using ensure => version.

Here's some output with --evaltrace --debug:
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[jdk1.6]/Exec[check-for-jdk1.6]: Evaluated in 0.76 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[jdk1.6]/Package[jdk1.6]: Evaluated in 0.11 seconds
debug: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[python26]/Exec[check-for-python26]: Executing check '/bin/bash -c '! /bin/rpm -ql python26''
debug: Executing '/bin/bash -c '! /bin/rpm -ql python26''
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[python26]/Exec[check-for-python26]: Evaluated in 0.44 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[python26]/Package[python26]: Evaluated in 0.03 seconds
debug: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[mercurial-py26]/Exec[check-for-mercurial-py26]: Executing check '/bin/bash -c '! /bin/rpm -ql mercurial-py26''
debug: Executing '/bin/bash -c '! /bin/rpm -ql mercurial-py26''
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[mercurial-py26]/Exec[check-for-mercurial-py26]: Evaluated in 0.61 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[mercurial-py26]/Package[mercurial-py26]: Evaluated in 0.04 seconds
debug: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[virtualenv]/Exec[check-for-virtualenv]: Executing check '/bin/bash -c '! /bin/rpm -ql virtualenv''
debug: Executing '/bin/bash -c '! /bin/rpm -ql virtualenv''
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[virtualenv]/Exec[check-for-virtualenv]: Evaluated in 0.55 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[virtualenv]/Package[virtualenv]: Evaluated in 0.03 seconds
debug: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc433]/Exec[check-for-gcc433]: Executing check '/bin/bash -c '! /bin/rpm -ql gcc433''
debug: Executing '/bin/bash -c '! /bin/rpm -ql gcc433''
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc433]/Exec[check-for-gcc433]: Evaluated in 2.68 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc433]/Package[gcc433]: Evaluated in 0.03 seconds
debug: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc411]/Exec[check-for-gcc411]: Executing check '/bin/bash -c '! /bin/rpm -ql gcc411''
debug: Executing '/bin/bash -c '! /bin/rpm -ql gcc411''
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc411]/Exec[check-for-gcc411]: Evaluated in 0.48 seconds
info: //Node[moz2-linux-slave17.build.mozilla.org]/staging-buildslave/devtools/Install_rpm[gcc411]/Package[gcc411]: Evaluated in 0.04 seconds


I tested by upgrading and then downgrading android-ndk.

So, the package checks still run, but they don't do any downloading; they just check the local package database (not shown in the output, unfortunately). I double checked this by looking at the access http access logs on staging-puppet. The only entries were for the upgrade and downgrade of android.

Comment 3

[Mike Taylor [:bear]]

Comment on attachment 451725 [details] [diff] [review]
fix rpm checks

looks good.  great find on nick's part!

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 451725 [details] [diff] [review]
fix rpm checks

changeset:   180:f0a34521189c

I had a bit of merging in devtools.pp due to Android changes.

Comment 5

[[e-mail] (:bhearsum)]

This landed, I tested a slave, and have been watching the logs on production-puppet. Haven't seen any issues.