The default bug view has changed. See this FAQ.

Push SUMO 2.2.2 and SUMO mobile on Thursday, August 19

VERIFIED FIXED

Status

Infrastructure & Operations
WebOps: Other
--
minor
VERIFIED FIXED
7 years ago
4 years ago

People

(Reporter: paulc, Assigned: oremj)

Tracking

other
All
Other
Bug Flags:
needs-downtime +

Details

(Whiteboard: 08/19/2010 @ 4pm)

(Reporter)

Description

7 years ago
This is a smaller release than 2.2.1, with the important difference that we also need to push a minor mobile SUMO change, so I figured it's OK to have it in the same push bug.

Steps (we'll post if this is missing any):

mobile.sumo:
* svn up
* flush caches

desktop.sumo:
* git checkout 2.2.2
* schematic migrations/

Updated

7 years ago
Flags: needs-downtime+
Whiteboard: 08/19/2010 @ 7pm

Updated

7 years ago
Assignee: server-ops → shyam
The push meeting is scheduled for 1pm West Coast time- is it possible to move this back from 7pm [what I see on the push bug whiteboard]? I am not available for a 7pm push Thurs night as I'm not going to be home- but I could do it in the afternoon.

Comment 2

7 years ago
No.  The downtime windows are Tues & Thurs from 7-11pm.  On occasion we've started as early as 4pm but 1pm affects to many time zones.
(Reporter)

Comment 3

7 years ago
We don't expect any downtime for this push. The schematic migrations we have to run are adding some permissions, but those are not required for the site to run.

If 1pm is too early, Rebecca: does 4pm work?
(Reporter)

Comment 4

7 years ago
(In reply to comment #3)
> If 1pm is too early, Rebecca: does 4pm work?
Nevermind. 4pm is AMO :)

Comment 5

7 years ago
Last SUMO push didn't go smooth so I'm not willing to do this outside of a normal push window.  Also need to make sure we have the right people online incase something does go wrong.
(Reporter)

Comment 6

7 years ago
(In reply to comment #5)
> Last SUMO push didn't go smooth so I'm not willing to do this outside of a
> normal push window.  Also need to make sure we have the right people online
> incase something does go wrong.
Fair point. Is there any way we can do tomorrow (Wed) instead? Anytime after 2pm?
(Reporter)

Comment 7

7 years ago
Talked to mrz, moved to tomorrow @ 4pm.
Whiteboard: 08/19/2010 @ 7pm → 08/18/2010 @ 4pm

Comment 8

7 years ago
Paul, we talked but why can't this wait till next Tuesday's window?
(Reporter)

Comment 9

7 years ago
James scheduled this push for tomorrow at 1pm a week ago. Pushing next week puts us behind schedule. We plan to have another push this month.

We've discussed our availability and we cannot do tomorrow at 7pm because neither Rebecca (our QA lead) nor I can be here. We can do today, or tomorrow at 1pm (as originally scheduled). We would much rather push this week, so - if neither of those work, how can we get this out this week?

If it helps - as opposed to our previous pushes, this one doesn't have any new requirements to install (which were usually the problem), so the risks are lower.


As a side note, is pushing today a problem because of beta 4? (and isn't that going out next week?)
Assignee: shyam → server-ops
Regardless of what was originally requested, downtime windows are Tuesday/Thursday with any other window an emergency push.
We released 2.2 on Wed Aug 4th- and it wasn't an emergency push. Is the schedule for when releases are available written somewhere?

It seems like there is a better process possible. If the Thurs at 1pm window wasn't acceptable it should have come up earlier. Pushing out until next week just puts everyone behind.
mrz, moving this to next week would really slow us down -- if at all possible, can we please do this push tomorrow at 4pm like Paul is requesting?

As discussed in the 1.5.4/2.0 postmortem, we want to do SUMO pushes earlier on the day to ensure we don't end up in similar situations again. As long as push windows are in the evenings, we can't really use them for our pushes.
(In reply to comment #12)
> mrz, moving this to next week would really slow us down -- if at all possible,
> can we please do this push tomorrow at 4pm like Paul is requesting?

Thursday @ 4pm is fine. 

> As discussed in the 1.5.4/2.0 postmortem, we want to do SUMO pushes earlier on
> the day to ensure we don't end up in similar situations again. As long as push
> windows are in the evenings, we can't really use them for our pushes.

1pm is too early.  The trough -starts- at 2pm Pacific and isn't at the bottom until 6pm which is why the normal windows are 7-11pm.

1pm affects too many users.
1pm was my fault: I inadvertently specified 4pm Eastern instead of 4pm Pacific.

mrz, w/r/t having the right people online, 7pm PDT windows start at 10pm my time, making it difficult to be coherent and useful. Both AMO and SUMO have been pushing at 4pm Pacific for the past couple of months and it seems to be working much better.

I'll email infra@ and webdev@.
(Reporter)

Comment 15

7 years ago
So is this happening today or tomorrow? (either would have to be at 4pm)

Looks like AMO's not pushing tomorrow?
(Reporter)

Updated

7 years ago
Whiteboard: 08/18/2010 @ 4pm → 08/19/2010 @ 4pm
(Reporter)

Comment 16

7 years ago
Adding this to the steps to save us from going through bug 588469 again:

* restart celery
(In reply to comment #15)
> So is this happening today or tomorrow? (either would have to be at 4pm)

Thursday 19 August @ 4pm PDT.
(Assignee)

Updated

7 years ago
Assignee: server-ops → jeremy.orem+bugs
(Assignee)

Comment 18

7 years ago
[root@mradm02 mobile.support.mozilla.com]# svn up
U    webroot/tiki-register.php
U    webroot/lib/userslib.php
U    webroot/templates/styles/mozkb/tiki-register.tpl


[root@mradm02 prod]# git fetch
remote: Counting objects: 221, done.
remote: Compressing objects: 100% (145/145), done.
remote: Total 152 (delta 94), reused 1 (delta 0)
Receiving objects: 100% (152/152), 39.16 KiB, done.
Resolving deltas: 100% (94/94), completed with 39 local objects.
From http://github.com/jsocol/kitsune
   5536aca..65bd879  development -> origin/development
   b76cd10..4197f60  master     -> origin/master
 * [new branch]      wiki       -> origin/wiki
From http://github.com/jsocol/kitsune
 * [new tag]         2.2.2      -> 2.2.2
[root@mradm02 prod]# git checkout 2.2.2
Previous HEAD position was b76cd10... Merge branch 'development'
HEAD is now at 67a4192... [583106] Introduce 2 new permissions--view_in_forum and post_in_forum--making it possible to have hidden-to-the-public and read-only forums.



Running migration 30:
INSERT INTO authority_permission
    VALUES 
        (NULL,'forums_forum.thread_move_forum',
            (select id from django_content_type where app_label='forums' and model='forum'),1,NULL,1,47963,1,
            '2010-08-10 10:37:22','2010-08-10 10:39:57'),
        (NULL,'forums_forum.thread_move_forum',
            (select id from django_content_type where app_label='forums' and model='forum'),2,NULL,1,47963,1,
            '2010-08-10 10:37:22','2010-08-10 10:39:57'),
        (NULL,'forums_forum.thread_move_forum',
            (select id from django_content_type where app_label='forums' and model='forum'),3,NULL,1,47963,1,
            '2010-08-10 10:37:22','2010-08-10 10:39:57');

That took 0.07 seconds
################################################## 



Running migration 31:
SET @ct = (SELECT id FROM django_content_type WHERE app_label='forums' AND model='forum');
INSERT INTO auth_permission (name, content_type_id, codename) VALUES
    ('Can view restricted forums', @ct, 'view_in_forum'),
    ('Can post in restricted forums', @ct, 'post_in_forum');

That took 0.17 seconds
##################################################
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.