Last Comment Bug 1310120 - puppet reinstalls scriptworker on every run
: puppet reinstalls scriptworker on every run
Status: RESOLVED FIXED
:
Product: Release Engineering
Classification: Other
Component: General Automation (show other bugs)
: unspecified
: Unspecified Unspecified
-- normal (vote)
: ---
Assigned To: Aki Sasaki [:aki]
: Chris AtLee [:catlee]
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-10-14 01:11 PDT by Rail Aliiev [:rail] ⌚️ET
Modified: 2016-10-18 08:59 PDT (History)
2 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
puppetfix (810 bytes, patch)
2016-10-17 17:02 PDT, Aki Sasaki [:aki]
mtabara: review+
Details | Diff | Splinter Review

Description User image Rail Aliiev [:rail] ⌚️ET 2016-10-14 01:11:29 PDT
... and restarts it. I discovered this while I was looking at the signing worker logs in papertrail. Additionally, signing-linux-1 generated ~1.3M events vs ~20K generated by others. Probably related to the initial setup?


Oct 14 00:57:01 signing-linux-1.srv.releng.use1.mozilla.com CROND: (root) CMD (. /usr/local/bin/proxy_reset_env.sh && /usr/bin/puppet agent --enable >/dev/null 2>/dev/null && /usr/bin/puppet agent --onetime --no-daemonize --logdest=syslog >/dev/null 2>&1)
Oct 14 00:57:01 signing-linux-1.srv.releng.use1.mozilla.com puppet-agent: Enabling Puppet.
Oct 14 00:57:40 signing-linux-1.srv.releng.use1.mozilla.com crontab: (root) LIST (root)
Oct 14 00:57:47 signing-linux-1.srv.releng.use1.mozilla.com puppet-agent: (/Stage[main]/Signing_scriptworker/Python35::Virtualenv[/builds/scriptworker]/Python35::Virtualenv::Package[/builds/scriptworker||frozendict==1.0]/Exec[pip /builds/scriptworker||frozendict==1.0]/returns) executed successfully


...


Oct 14 00:57:57 signing-linux-1.srv.releng.use1.mozilla.com puppet-agent: (/Stage[main]/Signing_scriptworker/Python35::Virtualenv[/builds/scriptworker]/Python35::Virtualenv::Package[/builds/scriptworker||slugid==1.0.7]/Exec[pip /builds/scriptworker||slugid==1.0.7]/returns) executed successfully
Oct 14 00:58:00 signing-linux-1.srv.releng.use1.mozilla.com puppet-agent: (/Stage[main]/Signing_scriptworker::Services/Exec[restart-scriptworker]) Triggered 'refresh' from 1 events
Oct 14 00:58:01 signing-linux-1.srv.releng.use1.mozilla.com puppet-agent: Finished catalog run in 28.40 seconds
Comment 1 User image Aki Sasaki [:aki] 2016-10-14 10:17:20 PDT
re: 1.3m vs 20k, I've been using signing-linux-1 as my debugging ground since signing scriptworker isn't tier1 yet, and it's been alive for longer.  Unless you mean that's within the last week or 24hrs or something.

However, reinstalling scriptworker every run is wrong, and should be fixed.
Comment 2 User image Aki Sasaki [:aki] 2016-10-14 10:37:53 PDT
https://hg.mozilla.org/build/puppet/rev/7d6f218c25e1b688b2997ee853de1e050900c178
bug 1310120 - puppet reinstalls scriptworker on every run. r=rail
Comment 3 User image Aki Sasaki [:aki] 2016-10-17 17:02:14 PDT
Created attachment 8801933 [details] [diff] [review]
puppetfix

Use the python3-compatible version of pip-check in the python35 module.
Comment 4 User image Mihai Tabara [:mtabara]⌚️GMT 2016-10-18 07:18:31 PDT
(In reply to Aki Sasaki [:aki] from comment #3)
> Created attachment 8801933 [details] [diff] [review]
> puppetfix
> 
> Use the python3-compatible version of pip-check in the python35 module.

Just to confirm, the fix is working great ;) 

I was working on my beetmover puppet patches this morning and hit this problem as the python pip check would not install any dep packages in my virtualenv. After tweaking on my local environment with Aki's patch, it worked like a charm.
Comment 5 User image Aki Sasaki [:aki] 2016-10-18 08:41:46 PDT
https://hg.mozilla.org/build/puppet/rev/cf679eab495015b386ce803b0f3b69faa7244cf3
bug 1310120 - use the py3 compatible pip_check in py35 virtualenv. r=mtabara
Comment 6 User image Aki Sasaki [:aki] 2016-10-18 08:59:22 PDT
Awesome.  Fix is rolled out!

Note You need to log in before you can comment on or make changes to this bug.