1021726 - heartbeat timeout

Status: VERIFIED FIXED

(Hello (Loop) :: Server, defect, P1)

Description

[Tarek Ziadé (:tarek)]

We should timeout after 5 seconds if the tok backend does not answer

Comment 1

[Rémy Hubscher (:natim)]

http://www.sitepoint.com/making-http-requests-in-node-js/ can help I will have a look.

Comment 2

[James Bonacci [:jbonacci]]

I am getting a response from Stage and Prod in less that 5 seconds (of course I have a fast connection).
What are the symptoms?

Comment 3

[Rémy Hubscher (:natim)]

Well, the __heartbeat__ works great when storage and partner answer.
When they don't the page is taking too long to answer and we don't get any answer with the 30s nginx timeout.

What we want to do is to set the redis.client timeout and partner.request timeout to a lower value (2s) so that in the worst case the page is taking 5s to answer

Comment 4

[Alexis Metaireau (:alexis)]

This is actually a configuration option we should set on the server. Letting the default behavior by default is probably what we want, but for prod we want something different here.

Benson, I'm adding you to this one, you would need to add the a "connect_timeout" setting of the storage "options" to something that's shorter than the expiration of nginx.

Comment 5

[Benson Wong [:mostlygeek]]

:alexis if it is a new configuration option please update the config.js schema. Once that is done I'll do the puppet work necessary to set this timeout.

Comment 6

[Alexis Metaireau (:alexis)]

that's a configuration option, but not required in all the cases, only when the backend is redis, it should go in the "settings" blob.

Comment 7

[Alexis Metaireau (:alexis)]

Attachment: link to github PR

Comment 8

[Alexis Metaireau (:alexis)]

https://github.com/mozilla-services/loop-server/commit/96ae6440a131e10e18feed15dfab873d02546361

Comment 9

[James Bonacci [:jbonacci]]

Verified in code and on Stage.