617110 - Push Firefox Input 2.1 to production

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Aakash Desai [:aakashd]]

We need a push of input.stage.mozilla.com to production on Tuesday, 12/9.

The only step to perform now (thanks to the amazing shyam) is to run:

/root/bin/input_update.sh 


We're flexible on time, but suggest 4pm. Also, let's go with the safe bet and announce that it needs downtime. We're adding a new submission method and it'd be good to not take any chances.

Comment 1

[Shyam Mani [:fox2mike]]

Tuesday or Thursday? Tue is 7th, Thu is 9th :)

Comment 2

[Aakash Desai [:aakashd]]

Thursday; Copy/paste fail from the previous 2.0 push bug :(

Comment 3

[Shyam Mani [:fox2mike]]

Gotcha.

Comment 4

[Fred Wenzel [:wenzel]]

We're going to have to move this push to Tuesday the 14th instead. Will that work for you, Shyam?

Comment 5

[Shyam Mani [:fox2mike]]

Fred,

We have a big SUMO push on the 14th, anything earlier or later is probably better for everyone. Thoughts?

Comment 6

[Shyam Mani [:fox2mike]]

Moving to the 21st after talking with Fred and Ryan.

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #6)
> Moving to the 21st after talking with Fred and Ryan.

Which means that Rebecca will either have to agree to do the release alone (and it'd be her first time -- I'm on vacation), or we can do it when I get back, starting on the 27th.  Rebecca, it's up to you, since the release date was changed.

Comment 8

[Aakash Desai [:aakashd]]

Fx 4 beta 8 and 3 look to ship on the 22nd, so there isn't any real lee-way here. Stephen, can you make sure that Rebecca is ready for the release on Tuesday?

Comment 9

[Shyam Mani [:fox2mike]]

Just a note - If I'm not awake for this, push back to the queue and set it to critical, it'll page oncall.

Comment 10

[Fred Wenzel [:wenzel]]

I am done tagging:

- last_known_good branch points to the old prod (== 2.0.1)
- tag 2.1 points to revision 089c81a
- prod branch points to 2.1

Before updating, let's fetch a complete database dump to roll back to in case of any problems.

Then run the update script.

After this successfully completes, update the Sphinx configs on our search servers with the latest one (presumably, that involves something like a ``git pull`` on these boxes).

Comment 11

[Shyam Mani [:fox2mike]]

(In reply to comment #10)
> I am done tagging:
> 
> - last_known_good branch points to the old prod (== 2.0.1)
> - tag 2.1 points to revision 089c81a
> - prod branch points to 2.1
> 
> Before updating, let's fetch a complete database dump to roll back to in case
> of any problems.

Done. -rw-r--r-- 1 root root 298107435 Dec 21 16:45 input_mozilla_com-20101221-164508.sql

> Then run the update script.

[root@mradm02 ~]# /root/bin/input_update.sh 
Updating vendor...
Already up-to-date.
Updating reporter...
remote: Counting objects: 579, done.
remote: Compressing objects: 100% (331/331), done.
remote: Total 456 (delta 309), reused 178 (delta 107)
Receiving objects: 100% (456/456), 170.06 KiB, done.
Resolving deltas: 100% (309/309), completed with 49 local objects.
From git://github.com/fwenzel/reporter
   61a3aea..5aed9c8  last_known_good -> origin/last_known_good
   fbd440a..089c81a  master     -> origin/master
 + 5aed9c8...089c81a prod       -> origin/prod  (forced update)
 * [new branch]      stable-feedback -> origin/stable-feedback
 * [new tag]         2.1        -> 2.1
M       locale
Previous HEAD position was 5aed9c8... 2.0.1 reverts
HEAD is now at 089c81a... Do not throw AssertionError if page <= 0 is requested. Bug 620296.
remote: Counting objects: 429, done.
remote: Compressing objects: 100% (184/184), done.
remote: Total 372 (delta 140), reused 274 (delta 87)
Receiving objects: 100% (372/372), 383.09 KiB, done.
Resolving deltas: 100% (140/140), completed with 2 local objects.
From git://github.com/fwenzel/reporter-locales
   de7baac..a0d2ac1  master     -> origin/master
Submodule path 'locale': checked out 'a0d2ac1e6b913479f6408a8a214a6ad9bc770ed4'
Running migration 8:
ALTER TABLE theme ENGINE=InnoDB;
ALTER TABLE theme_item ENGINE=InnoDB;

That took 0.46 seconds
################################################## 

WARNING:product_details:Requested product details file firefox_versions not found!
WARNING:product_details:Requested product details file mobile_details not found!
/data/input/www /data/input
[master 201d9c6] deploy input
 127 files changed, 6420 insertions(+), 3865 deletions(-)
 create mode 100644 django/input.mozilla.com/reporter/apps/dashboard/templates/dashboard/devices.html
 create mode 100644 django/input.mozilla.com/reporter/apps/dashboard/templates/dashboard/manufacturers.html
 create mode 100644 django/input.mozilla.com/reporter/apps/feedback/cron.py
 rewrite django/input.mozilla.com/reporter/apps/input/__init__.pyc (97%)
 create mode 100644 django/input.mozilla.com/reporter/apps/themes/templates/themes/theme.html
 rewrite django/input.mozilla.com/reporter/locale/de/LC_MESSAGES/messages.mo (87%)
 rewrite django/input.mozilla.com/reporter/locale/el/LC_MESSAGES/messages.mo (92%)
 rewrite django/input.mozilla.com/reporter/locale/en_US/LC_MESSAGES/messages.mo (91%)
 rewrite django/input.mozilla.com/reporter/locale/es/LC_MESSAGES/messages.mo (93%)
 rewrite django/input.mozilla.com/reporter/locale/fy/LC_MESSAGES/messages.mo (60%)
 rewrite django/input.mozilla.com/reporter/locale/hu/LC_MESSAGES/messages.mo (88%)
 rewrite django/input.mozilla.com/reporter/locale/id/LC_MESSAGES/messages.mo (97%)
 rewrite django/input.mozilla.com/reporter/locale/it/LC_MESSAGES/messages.mo (93%)
 rewrite django/input.mozilla.com/reporter/locale/ko/LC_MESSAGES/messages.mo (92%)
 rewrite django/input.mozilla.com/reporter/locale/pl/LC_MESSAGES/messages.mo (96%)
 create mode 100644 django/input.mozilla.com/reporter/locale/ro/LC_MESSAGES/messages.mo
 copy django/input.mozilla.com/reporter/locale/{pl => ro}/LC_MESSAGES/messages.po (58%)
 rewrite django/input.mozilla.com/reporter/locale/ru/LC_MESSAGES/messages.mo (91%)
 rewrite django/input.mozilla.com/reporter/locale/sq/LC_MESSAGES/messages.mo (98%)
 rewrite django/input.mozilla.com/reporter/locale/zh_TW/LC_MESSAGES/messages.mo (94%)
 rewrite django/input.mozilla.com/reporter/media/css/mobile_old-min.css (62%)
 create mode 100644 django/input.mozilla.com/reporter/media/img/suggestion-box-idea-large.png
 create mode 100644 django/input.mozilla.com/reporter/media/img/suggestion-box-idea-medium.png
 create mode 100644 django/input.mozilla.com/reporter/media/img/suggestion-box-idea-small.png
 create mode 100644 django/input.mozilla.com/reporter/migrations/09-device-info.sql
 create mode 100644 django/input.mozilla.com/reporter/migrations/10-feedback_opinion_type.sql
/data/input
From git://mradm02/data/input/www
   8c6e555..201d9c6  master     -> origin/master
===pp-app-input01===

===pp-app-input02===

Restart Apache? [y/n] y
===pp-app-input01===

===pp-app-input02===

[root@mradm02 ~]#

Also, had to run 2 migrations by hand :

[root@mradm02 reporter]# schematic migrations/
Running migration 9:
ALTER TABLE `feedback_opinion` ADD `manufacturer` VARCHAR( 255 ) NOT NULL AFTER `locale` ,
ADD `device` VARCHAR( 255 ) NOT NULL AFTER `manufacturer` ;

ALTER TABLE `feedback_opinion` ADD INDEX ( `manufacturer` , `device` ) ;

That took 49.89 seconds
################################################## 

Running migration 10:
ALTER TABLE `feedback_opinion` ADD `type` SMALLINT UNSIGNED NOT NULL DEFAULT 1 AFTER `positive`;
UPDATE `feedback_opinion` SET `type` = 2 WHERE `positive` = 0;
ALTER TABLE `feedback_opinion` DROP `positive`;

That took 61.04 seconds
##################################################

> After this successfully completes, update the Sphinx configs on our search
> servers with the latest one (presumably, that involves something like a ``git
> pull`` on these boxes).

Updated and kicked by hand :

[root@pp-app-sphinx02 ~]# /data/bin/sphinx-reindex input
Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/data/sphinx/input/sphinx.conf'...
FATAL: failed to lock pid file '/data/sphinx/input/log/searchd.pid': Resource temporarily unavailable (searchd already running?)
Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/data/sphinx/input/sphinx.conf'...
indexing index 'opinions'...
collected 943134 docs, 64.9 MB
sorted 11.5 Mhits, 100.0% done
total 943134 docs, 64891090 bytes
total 20.246 sec, 3205013 bytes/sec, 46582.01 docs/sec
total 104 reads, 0.033 sec, 1047.7 kb/call avg, 0.3 msec/call avg
total 206 writes, 0.314 sec, 1210.2 kb/call avg, 1.5 msec/call avg
rotating indices: succesfully sent SIGHUP to searchd (pid=7210).

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED (pushed).  Thanks, Rebecca (and Ryan, Fred, etc.)