1021855 - Improve Loop-Server logging for troubleshooting

Status: VERIFIED FIXED

Description

[Tarek Ziadé (:tarek)]

Today we had to debug a server stack alive to understand why it was not working.

We had three configuration problems related to:

- CORS
- fakeTokbox
- https

We need to get better at troubleshooting this kind of issue by using :

- the information displayed at / and /__heartbeat__
- the errors found in the logs

Comment 1

[James Bonacci [:jbonacci]]

I was just about to open a bug like this - thanks for saving me the effort!...

Comment 2

[Alexis Metaireau (:alexis)]

I guess this is related to bug 984325, we need to check log errors are effectively captured.

Comment 3

[Alexis Metaireau (:alexis)]

Tarek, what do you have in mind specifically about having better loggin here? (it seems that the errors I found in the sentry deployment for stage is enough?).

Comment 4

[Tarek Ziadé (:tarek)]

Are those logs enough to detect the issues we had ? Would you have been able to troubleshoot them with no ssh access ?

Comment 5

[Alexis Metaireau (:alexis)]

That was useful for the problems we had so far.

However, I need to double check if what I found in the stdout of the process (collected by circus) was also sent to sentry, and if not, fix it.

Comment 6

[Alexis Metaireau (:alexis)]

Logs there seem to be enough. Closing.

Comment 7

[James Bonacci [:jbonacci]]

I agree.
/var/log/circus.log
/var/log/loop_err.log
/var/log/loop_out.log
/var/log/hekad/loop.stdout.log
/var/log/hekad/loop.stderr.log
/var/log/nginx/access.log
/var/log/nginx/error.log