1163662 - Please re-deploy release 0.17.7 to loop-server STAGE - CONFIG UPDATES

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Comment 1

[Dean Wilson (:deanw)]

These stack now exists under
  loop.stage.mozaws.net
  
They have numerous internal changes from an infrastructure perspective -
We've changed the build pipeline around -alot-.
The datadog agent has been added to the build.
We've updated the AMI (and so all the base OS packages).
We now use the Amazon provided SSL policies.
Version information is exposed at the stack, ASG and instance level.

From a code/QA test suite perspective hopefully nothing has changed.

Comment 2

[Richard Pappalardo [:rpapa][:rpappalardo]]

------------------
SUMMARY
------------------

Stack check and loadtests - OK but....

'Start a conversation' button appears, but is (intermittently) inert for FF: Nightly, DevEd & Beta
By intermittently, I mean most of the time.  In one instance, I tried pushing the button, assumed it was inert because nothing happened but then 30 secs later, the conversation window opened.  I've seen this in the past when a loadtest was running in background but in this case, there was no loadtest.  Not sure what's up with this, but I'll look into it more tomorrow.

FYI: that I always restart browser with a fresh profile, but same result.

Note:
1 intermittent unit test error - alexis is aware of it:
AssertionError: expected 0 to deeply equal 600



------------------
STAGE VALIDATION
------------------

STAGE STACK - PACKAGE VERSION
$ sed -n '/version/p' /data/loop-server/package.json:
"version": "0.17.7",


https://loop.stage.mozaws.net:
{"name":"mozilla-loop-server","description":"The Mozilla Loop (WebRTC App) server","version":"0.17.7","homepage":"https://github.com/mozilla-services/loop-server/","endpoint":"https://loop.stage.mozaws.net","fakeTokBox":false,"fxaOAuth":true,"i18n":{"defaultLang":"en-US","lang":"en-US"}}



https://loop.stage.mozaws.net/push-server-config
{
  "pushServerURI" : "wss://loop-push1.stage.mozaws.net/"
}
and:
{
  "pushServerURI": "wss://loop-push2.stage.mozaws.net/"
}


https://loop.stage.mozaws.net/__heartbeat__:
{
  "storage" : true,
  "provider": true
}


$ curl -I https://loop.stage.mozaws.net

HTTP/1.1 200 OK
Date: Wed, 13 May 2015 20:38:52 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 273
Connection: keep-alive
Vary: Origin
ETag: W/"111-e944a68d"
Timestamp: 1431549532


------------------
LOADTEST
------------------

Users [80], Agents 5, 4 slaves, Duration 30 mins
https://loads.services.mozilla.com/run/d0b4e187-20d0-44bd-94dd-5c7560aecf15

Users [80], Agents 10, 4 slaves, Duration 30 mins
https://loads.services.mozilla.com/run/2086496c-6c55-4134-aac8-88606dd47000


k

Comment 3

[Richard Pappalardo [:rpapa][:rpappalardo]]

Met w/ deanw and bobm today to troubleshoot inert 'Start a Conversation' window

deanw to respin the 0.17.7 stack so that error logging will appear in new sentry.
Also, looking at logs, doesn't appear to be any error.
plan is to follow-up w/ natim,tarek on Mon.

Notes from Standard8 in regards to 'Start a Conversation' window:
9:35:49 AM <Standard8> deanw: it’ll always appear straight away. It will be disabled until 
a) push registration has completed and 
b) its registered with the loop-server
9:36:00 AM <Standard8> I believe its a slightly different shade when disabled

Comment 4

[Rémy Hubscher (:natim)]

I am not sure this is related but for websockets through ELB, there is this thread: http://stackoverflow.com/a/9210639/186202

Comment 5

[Richard Pappalardo [:rpapa][:rpappalardo]]

(In reply to Rémy Hubscher (:natim) from comment #4)
> I am not sure this is related but for websockets through ELB, there is this
> thread: http://stackoverflow.com/a/9210639/186202

Now that Dean has another stack -40- working (Thanks, Dean!) and all the URLs are rendering, we may want to revisit this.

Seems like we could either:

[1]. Rollback the Amazon provided SSL policies
  - if we do this and re-deploy, that might be the quickest way to isolate the problem to this issue.
[2]. Try what they suggest in StackOverflow: http://stackoverflow.com/a/9210639/186202
  - Dean, is it worth attempting this so we can use Amazon-provided SSL policies?
[3]. other?

What do you guys think?

------------------
STAGE RE-VERIFICATION (STACK 40)
------------------

https://loop.stage.mozaws.net/push-server-config:
{
  "pushServerURI": "wss://loop-push2.stage.mozaws.net/"
}

https://loop.stage.mozaws.net:
{
  "name": "mozilla-loop-server",
  "description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7",
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.stage.mozaws.net",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang": "en-US",
    "lang": "en-US"
  }
}

Comment 6

[Richard Pappalardo [:rpapa][:rpappalardo]]

FYI: Though loop-server seems to be up and running 'Start a Conversation' window is still inert with stack 40.

Also verified processes on stack: loop (node), stackdriver, circus, heka, nginx

Comment 7

[Rémy Hubscher (:natim)]

I still cannot access https://loop.stage.mozaws.net/push-server-config from here.

Comment 8

[Richard Pappalardo [:rpapa][:rpappalardo]]

------------------
SUMMARY
------------------

deanw fixed deployment to stage and 'Start a conversation' button is now working properly.
Thanks, Dean!

Placed several calls successfully between Nightly and DevEd w/ out issue.

Verifying STAGE.


------------------
STAGE VALIDATION
------------------

STAGE STACK - PACKAGE VERSION
$ sed -n '/version/p' /data/loop-server/package.json:
"version": "0.17.7",


PROCESSES:
Verified processes: loop (node), stackdriver, circus, heka, nginx


https://loop.stage.mozaws.net:
{
  "name": "mozilla-loop-server",
  "description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7",
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.stage.mozaws.net",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang": "en-US",
    "lang": "en-US"
  }
}

https://loop.stage.mozaws.net/push-server-config
{
  "pushServerURI": "wss://loop-push1.stage.mozaws.net/"
}


https://loop.stage.mozaws.net/__heartbeat__:
{
  "storage" : true,
  "provider": true
}


$ curl -I https://loop.stage.mozaws.net

HTTP/1.1 200 OK
Date: Thu, 21 May 2015 15:59:51 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 273
Connection: keep-alive
Vary: Origin
ETag: W/"111-e944a68d"
Timestamp: 1432223991


------------------
LOADTEST
------------------

Users [80], Agents 10, 4 slaves, Duration 5 mins
0.17.7 has already been load-tested, just doing a sanity check
https://loads.services.mozilla.com/run/2bc6ebdf-eb73-48e5-b282-2a3064553642

Comment 9

[Richard Pappalardo [:rpapa][:rpappalardo]]

In regards to removing the config option for load-balancing loop-push on STAGE
(wss://loop-push2.stage.mozaws.net/) I heard from the push team that this was never rolled out to PROD because it wasn't needed after all.

Here's the original bug.  You may want to update it noting that this has been removed then close it out.
Bug 1055139 - Loop client needs to provide hook for load balancing Simple Push servers