Chief deploy runs rsync_project before running compiling *.pyx

RESOLVED FIXED

Status

Tree Management
Treeherder
P2
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: emorley, Assigned: emorley)

Tracking

Details

Attachments

(1 attachment)

(Assignee)

Description

4 years ago
eg:
http://treeherderadm.private.scl3.mozilla.com/chief/treeherder.prod/logs/master.1410211793

[2014-09-08 14:29:58] Running deploy
[2014-09-08 14:29:58] [localhost] running: /data/treeherder/deploy -n treeherder.mozilla.org
[2014-09-08 14:29:58] [localhost] finished: /data/treeherder/deploy -n treeherder.mozilla.org (0.129s)
[localhost] err: [2014-09-08 14:29:58] Running rsync_project
[localhost] err: [2014-09-08 14:29:58] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/treeherder/src/treeherder.mozilla.org/ /data/treeherder/www/treeherder.mozilla.org/
[localhost] err: [2014-09-08 14:29:58] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/treeherder/src/treeherder.mozilla.org/ /data/treeherder/www/treeherder.mozilla.org/ (0.095s)
[localhost] err: [2014-09-08 14:29:58] Finished rsync_project (0.095s)
[2014-09-08 14:29:58] [localhost] running: /sbin/service celerybeat restart
[2014-09-08 14:30:09] [localhost] finished: /sbin/service celerybeat restart (11.201s)
[localhost] out: Restarting celerybeat: celerybeat: stopped
[localhost] out: celerybeat: started
[localhost] out: [  OK  ]
[2014-09-08 14:30:09] [localhost] running: cd /data/treeherder/src/treeherder.mozilla.org/treeherder-service && python2.6 manage.py collectstatic --noinput --settings treeherder.settings
[2014-09-08 14:30:09] [localhost] finished: cd /data/treeherder/src/treeherder.mozilla.org/treeherder-service && python2.6 manage.py collectstatic --noinput --settings treeherder.settings (0.513s)
[localhost] out: 
[localhost] out: 0 static files copied, 106 unmodified.
[2014-09-08 14:30:09] [localhost] running: cd /data/treeherder/src/treeherder.mozilla.org/treeherder-service && python2.6 setup.py build_ext --inplace
[2014-09-08 14:30:13] [localhost] finished: cd /data/treeherder/src/treeherder.mozilla.org/treeherder-service && python2.6 setup.py build_ext --inplace (3.322s)
[localhost] out: Compiling treeherder/log_parser/parsers.pyx because it changed.
[localhost] out: Cythonizing treeherder/log_parser/parsers.pyx

This means that we have to deploy twice before the changes to the pyx files are reflected in production (first time rsyncs the old files, then compiles, then second time rsyncs the files compiled prior).
(Assignee)

Comment 1

4 years ago
So we do:

@task
def deploy(ctx):
    checkin_changes(ctx)
    deploy_admin_node(ctx)

And checkin_changes() runs the IT deploy script which does the rsync, and deploy_admin_node() does the compile after.

We should probably just break the compile out of deploy_admin_node().
(Assignee)

Updated

4 years ago
Assignee: nobody → emorley
Status: NEW → ASSIGNED
(Assignee)

Comment 2

4 years ago
Created attachment 8486234 [details] [review]
service PR205: Collect static assets and rebuild pyx files before deploy
Attachment #8486234 - Flags: review?(jeads)

Comment 3

4 years ago
Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/423446402c10eaf2deb85b9bd36612ceea55c102
Bug 1064646 - Comment and PEP8 cleanup

https://github.com/mozilla/treeherder-service/commit/7775a57b99df6122e85053c201e988dcf94d2511
Bug 1064646 - Collect static assets and rebuild pyx files before deploy

Previously changes were made to the srcdir after the IT deploy script
had already run, so were not rsynced until the next deploy push.
(Assignee)

Comment 4

4 years ago
I'll file another bug for the general cleanup of this file.
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Comment 5

4 years ago
Comment on attachment 8486234 [details] [review]
service PR205: Collect static assets and rebuild pyx files before deploy

iirc this was r+'d over IRC or in RL
Attachment #8486234 - Flags: review?(jeads)
You need to log in before you can comment on or make changes to this bug.