1180028 - Use woo_mailer.py from the webtools orangefactor environment not mcote's user directory

Status: RESOLVED FIXED

(Tree Management Graveyard :: OrangeFactor, defect, P2)

Description

[Ed Morley [:emorley]]

Currently we have two orangefactor src clones (plus dep repos) and two virtualenvs. One in webtools (used for the site) and one in mcote's user directory, used only by the cron that runs woo_mailer.py

In bug 1179821 we want to add another script that comments on bugs.
It's a pain to try and keep these two src dirs and virtualenvs in sync (and doubles the amount I have to deploy each time), so let's combine them.

Comment 1

[Ed Morley [:emorley]]

webtools:

0 0 * * *   /home/webtools/apps/gofaster_dashboard/bin/fetch-and-process-builddata.sh
0 0,4,8,12,16,18,20 * * * /home/webtools/apps/orangefactor/bin/python /home/webtools/apps/orangefactor/src/bzcache/bzcache/bz_cache_refresh.py

mcote:

0 8 * * tue     (/home/mcote/woo_mailer.sh 2>&1) >> /home/mcote/woo_mailer.log

[[e-mail]3 orangefactor]$ hg status
? test_mailer.py

[[e-mail]3 orangefactor]$ cat test_mailer.py
# This Source Code is subject to the terms of the Mozilla Public License
# version 2.0 (the "License"). You can obtain a copy of the License at
# http://mozilla.org/MPL/2.0/.

from sendemail import sendemail

def main():
    from_address = "Mozilla A-Team <auto-tools@<snip>.com>"
    mail_dest = ["mcote@<snip>.com"]
    subject = "Test message"
    #mail_username = "automation@<snip>.com"
    mail_username = None
    #mail_password = "<snip>"
    mail_password = None
    text = "This is a test message.\n"
    mail_server = "smtp.mozilla.org"
    #mail_server = "mail.mozilla.com"
    mail_port = 25
    mail_ssl = False

    sendemail(from_addr=from_address, to_addrs=mail_dest,
              subject=subject, username=mail_username,
              password=mail_password, text_data=text,
              server=mail_server, port=mail_port, use_ssl=mail_ssl)

if __name__ == '__main__':
    main()

[[e-mail]3 orangefactor]$ hg status -i | grep -v '.pyc'
I woo_mailer.conf

[[e-mail]3 orangefactor]$ cat woo_mailer.conf
[report]
from = Mozilla A-Team <auto-tools@<snip>.com>

[email]
dest = auto-tools@<snip>.com, dev-tree-alerts@<snip>.org
admin = mcote@<snip>.com
server = smtp.mozilla.org
ssl = false
port = 25
#username = automation@<snip>.com
#password = <snip>

[woo]
local_server_url = http://localhost/orangefactor/api
external_server_url = http://brasstacks.mozilla.com/orangefactor

Comment 2

[Ed Morley [:emorley]]

[[e-mail]3 ~]$ cat woo_mailer.sh
OFHOME=/home/mcote/orangefactor
cd $OFHOME/src/orangefactor
echo "Running in test mode."
$OFHOME/bin/python woo_mailer.py -t
echo "Running in production mode."
$OFHOME/bin/python woo_mailer.py

[[e-mail]3 ~]$ cat woo_mailer_test.sh
OFHOME=/home/mcote/orangefactor
cd $OFHOME/src/orangefactor
$OFHOME/bin/python woo_mailer.py -t

Comment 3

[Ed Morley [:emorley]]

[[e-mail]3 ~]# cp --no-preserve=ownership /home/mcote/woo_mailer.sh /home/mcote/woo_mailer_test.sh /hom e/webtools/apps/orangefactor/
[[e-mail]3 ~]# chown webtools:webtools /home/webtools/apps/orangefactor/woo_mailer*.sh
[[e-mail]3 ~]# ls -l /home/webtools/apps/orangefactor/
total 32
drwxr-x--- 2 webtools webtools 4096 Jul  2 16:10 bin
-rwxr-x--- 1 webtools webtools  110 Jan 30  2012 deploy.sh
drwxr-x--- 2 webtools webtools 4096 Jan 23  2012 include
drwxr-x--- 3 webtools webtools 4096 Jan 23  2012 lib
lrwxrwxrwx 1 webtools     2325   36 Jan 23  2012 lib64 -> /home/webtools/apps/orangefactor/lib
-rw-r----- 1 webtools webtools   60 Aug  7 06:27 pip-selfcheck.json
drwxr-x--- 5 webtools webtools 4096 Jun  4  2014 src
-rwxr-x--x 1 webtools webtools  194 Aug  7 07:09 woo_mailer.sh
-rwxr-x--- 1 webtools webtools   96 Aug  7 07:09 woo_mailer_test.sh


[[e-mail]3 ~]# cp /home/mcote/orangefactor/src/orangefactor/woo_mailer.conf /home/webtools/apps/orangefactor/src/orangefactor/
[[e-mail]3 ~]# chown webtools:webtools /home/webtools/apps/orangefactor/src/orangefactor/woo_mailer.conf
[[e-mail]3 ~]# ls -l /home/webtools/apps/orangefactor/src/orangefactor/woo_mailer.conf
-rw-r----- 1 webtools webtools 393 Aug  7 07:14 /home/webtools/apps/orangefactor/src/orangefactor/woo_mailer.conf


[[e-mail]3 orangefactor]$ cd ~/apps/orangefactor/
[[e-mail]3 orangefactor]$ sed woo_mailer.sh woo_mailer_test.sh -i -e 's_mcote_webtools/apps_'
[[e-mail]3 orangefactor]$ cat woo_mailer.sh
OFHOME=/home/webtools/apps/orangefactor
cd $OFHOME/src/orangefactor
echo "Running in test mode."
$OFHOME/bin/python woo_mailer.py -t
echo "Running in production mode."
$OFHOME/bin/python woo_mailer.py

[[e-mail]3 orangefactor]$ cat woo_mailer_test.sh
OFHOME=/home/webtools/apps/orangefactor
cd $OFHOME/src/orangefactor
$OFHOME/bin/python woo_mailer.py -t


[[e-mail]3 ~]$ (/home/webtools/apps/orangefactor/woo_mailer.sh 2>&1) >> /home/webtools/apps/orangef actor/woo_mailer.log
[[e-mail]3 ~]$ tail -n 2 ~/apps/orangefactor/woo_mailer.log
Sending email...
Done!


-> All seems to work, so switched the crons over.

[[e-mail]3 ~]# crontab -u mcote -l
0 8 * * tue     (/home/mcote/woo_mailer.sh 2>&1) >> /home/mcote/woo_mailer.log
[[e-mail]3 ~]# crontab -u webtools -l
0 0 * * *   /home/webtools/apps/gofaster_dashboard/bin/fetch-and-process-builddata.sh
0 0,4,8,12,16,18,20 * * * /home/webtools/apps/orangefactor/bin/python /home/webtools/apps/orangefactor/src/bzcache/bzcache/bz_cache_refresh.py

[[e-mail]3 ~]# crontab -u webtools -e
crontab: installing new crontab
[[e-mail]3 ~]# crontab -u webtools -l
0 0 * * *   /home/webtools/apps/gofaster_dashboard/bin/fetch-and-process-builddata.sh
0 0,4,8,12,16,18,20 * * * /home/webtools/apps/orangefactor/bin/python /home/webtools/apps/orangefactor/src/bzcache/bzcache/bz_cache_refresh.py
0 8 * * tue     (/home/webtools/apps/orangefactor/woo_mailer.sh 2>&1) >> /home/webtools/apps/orangefactor/woo_mailer.log

[[e-mail]3 ~]# crontab -u mcote -r
[[e-mail]3 ~]# crontab -u mcote -l
no crontab for mcote


Rather than removing the old src checkouts in mcote's home directory, I've just renamed them for now, so we can check they are definitely no longer required:

[[e-mail]3 ~]# su mcote
[[e-mail]3 root]$ cd ~/
[[e-mail]3 ~]$ ll
total 232288
drwxr-x--- 3 mcote mcote      4096 Apr 22 02:33 nginx
drwxr-x--- 6 mcote mcote      4096 Jul  2 16:14 orangefactor
-rw-r----- 1 mcote mcote       361 Jan 30  2012 updatecache.py
-rw-r--r-- 1 mcote mcote 237601636 Aug  4 08:01 woo_mailer.log
-rwxr-x--x 1 mcote mcote       194 Jun 25  2013 woo_mailer.sh
-rwxr-x--- 1 mcote mcote        96 Jul 18  2013 woo_mailer_test.sh
[[e-mail]3 ~]$ mv ~/orangefactor/ ~/orangefactor_old
[[e-mail]3 ~]$ mv ~/nginx/ ~/nginx_old
[[e-mail]3 ~]$ mkdir old_scripts
[[e-mail]3 ~]$ mv updatecache.py woo_mailer.sh woo_mailer_test.sh old_scripts/
[[e-mail]3 ~]$ ll
total 232280
drwxr-x--- 3 mcote mcote      4096 Apr 22 02:33 nginx_old
drwxr-x--- 2 mcote mcote      4096 Aug  7 07:39 old_scripts
drwxr-x--- 6 mcote mcote      4096 Jul  2 16:14 orangefactor_old
-rw-r--r-- 1 mcote mcote 237601636 Aug  4 08:01 woo_mailer.log


[[e-mail]3 ~]# /etc/init.d/orangefactor stop; /etc/init.d/orangefactor start; /etc/init
.d/nginx reload
stopping orangefactor                                      [  OK  ]
starting orangefactorspawn-fcgi: child spawned successfully: PID: 30985
                                                           [  OK  ]
Reloading nginx:                                           [  OK  ]