876844 - [hacksmo] set up chief

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Luke Crouch [:groovecoder]]

We just merged https://github.com/mozilla/mozhacks/pull/13 and we'd like to push it. But we'd like to set up chief for hacks.mozilla.org so we can push more quickly without having to bother webops each time.

Comment 1

[Luke Crouch [:groovecoder]]

note: we also want to set up chief for hacks.allizom.org

Comment 2

[Luke Crouch [:groovecoder]]

Not bumping to 'major' just yet, but this is blocking another pull we need to merge and push.

Comment 3

[Brandon Burton [:solarce]]

Working on, should be done today, would you like me to push the code in the mean time? Setup time for chief is a couple hours per environment

Comment 4

[Luke Crouch [:groovecoder]]

The push can wait, I'd like to use it to test the whole chief setup actually. Thanks.

Comment 5

[Chris Turra [:cturra]]

grabbing this bug. i just made a pull request to add the chief update script to the mozhacks github repo.

  https://github.com/mozilla/mozhacks/pull/15

Comment 6

[Luke Crouch [:groovecoder]]

looks good to me - merged. when can we test on hacks.allizom.org?

Comment 7

[Chris Turra [:cturra]]

thnx for merging this. give me a little bit to complete the setup and testing.

Comment 8

[Chris Turra [:cturra]]

chief in stage should be good to go. you can access it at:

  http://genericadm.private.phx1.mozilla.com/chief/hacks.stage

i will provide you with the password privately in an email. share with those who should have access to push to stage.

Comment 9

[Luke Crouch [:groovecoder]]

tried pushing and got a failure:

http://genericadm.private.phx1.mozilla.com/chief/hacks.stage/logs/640fe38c927587ec0c1682fc5443989ac4d5f135.1369865214

Comment 10

[Chris Turra [:cturra]]

i guess the failure is misleading, but looks to be telling you that there is nothing to deploy.

 [localhost] err: [localhost] out: # On branch master
 [localhost] err: [localhost] out: nothing to commit (working directory clean)


i also confirmed that on the admin host:

[[e-mail]1 wp-content-git]# git log -1
commit 640fe38c927587ec0c1682fc5443989ac4d5f135
Merge: 3c62f3c b0361a7
Author: luke crouch <[e-mail]>
Date:   Wed May 29 13:54:37 2013 -0700

    Merge pull request #15 from cturra/master

    adding chief deploy scripts to repo

Comment 11

[Luke Crouch [:groovecoder]]

Hmm ... for MDN we sometimes deploy the same refspec in order to trigger changes like l10n updates or cache clears, etc. Can we do the same thing here, or do we need to make changes to trigger a push?

Comment 12

[Chris Turra [:cturra]]

works for me. i made a change to the chief setup this morning to allow you to push the same refspec :)

 http://genericadm.private.phx1.mozilla.com/chief/hacks.stage/history

Comment 13

[Luke Crouch [:groovecoder]]

Ran into the error again?

http://genericadm.private.phx1.mozilla.com/chief/hacks.stage/logs/640fe38c927587ec0c1682fc5443989ac4d5f135.1369941165

(Also, can we have this chief instance broadcast to #mdndev like the MDN chief instance does?)

Comment 14

[Chris Turra [:cturra]]

i believe i know what the difference between the environments is that is causing this. the stderr you're seeing is from the deploy script having nothing to commit to the local git repo - which is used for the deployment of code to the web heads. in developer, since locale data and other files are updated on every deploy, there are changed files and thus the local git commit is successful.

i have made a minor tweak to the update.py script, which simply writes the `date` at deploy time to a text file. this should result in a modified file and the local git commits to work as expected. pull request found at:

  https://github.com/mozilla/mozhacks/pull/16

Comment 15

[Luke Crouch [:groovecoder]]

merged.

Comment 16

[Chris Turra [:cturra]]

looks like it worked :) 

  http://genericadm.private.phx1.mozilla.com/chief/hacks.stage/history


i will look into the irc bot for you and report back when i have more.

Comment 17

[Luke Crouch [:groovecoder]]

Yup, stage looks good. Let's do prod now too.

Comment 18

[Chris Turra [:cturra]]

(In reply to Luke Crouch [:groovecoder] from comment #17)
> Yup, stage looks good. Let's do prod now too.

as discussed on irc, this is now complete (and we had success on the first chief push). 

  http://genericadm.private.phx1.mozilla.com/chief/hacks.prod/history


* remaining action items from me: complete the irc bot setup for stage and prod chief deploys.

Comment 19

[Chris Turra [:cturra]]

chief irc bots setup for both stage and prod. below is a test run in stage. :)

11:29 -!- hacksstagepush [[e-mail]] has joined #mdndev
11:31 -!- hacksprodpush [[e-mail]] has joined #mdndev

11:32 < hacksstagepush> whoa, cturra is pushing hacks-stage 2cd8acb5dbaf3e98825f076305ef7b417c979edc
11:32 < hacksstagepush> watching http://genericadm.private.phx1.mozilla.com/chief/hacks.stage/logs/2cd8acb5dbaf3e98825f076305ef7b417c979edc.1370025121
11:32 < hacksstagepush> the push is now going to the webheads!! (2cd8acb5dbaf3e98825f076305ef7b417c979edc cturra)
11:32 < hacksstagepush> Finished: rsync_project (0.087s), commit_www (5.862s), checkin_changes (5.988s), update (5.988s)
11:32 < hacksstagepush> cturra pushed hacks-stage 2cd8acb5dbaf3e98825f076305ef7b417c979edc
11:32 < hacksstagepush> Finished: deploy_app (5.728s), deploy (5.728s)