1419549 - [ops infra socorro] logging

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

The new ops infrastructure for socorro needs working logging from all the nodes to the logging host.

Because nodes produce a lot of output, it'd be best if we could get logging broken up by node type. Processor logs in one file, crontabber logs in another file, webapp logs in a third file, etc.

This bug covers setting up logging and making sure developers can access it.

Comment 1

[Miles Crabill [:miles]]

Logging in new-stage is configured for all node types, including:
  - collector (antenna)
  - crontabber
  - elasticsearch
  - processor
  - webapp

The current recommendation for accessing logs is the rawlogs host. The socorro rawlogs hosts can be accessed through the logging bastion like so:

```ssh -A logging.bastion.us-east-1.stage.mozaws.net -t ssh socorro``` (for stage)
```ssh -A logging.bastion.us-west-2.prod.mozaws.net -t ssh socorro``` (for prod, not yet configured)

There are accounts for both devs on both the logging bastion and the rawlogs host. PM me if you have trouble with access!

See here for more info on logging access: https://mana.mozilla.org/wiki/display/SVCOPS/Logging+2.0+Access+Retention+and+Delays

Logs are broken up by node type and function, such as:
socorro.collector.docker.collector.log
socorro.collector.file.nginx_access.log
socorro.webapp.docker.webapp.log
socorro.webapp.file.nginx_access.log
... etc.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I tested this out just now and was able to log in through the bastion for -stage to the socorro server and view the logs. Yay!

Marking this as FIXED.