1406020 - [ops infra socorro] working processor in stage

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Miles Crabill [:miles]]

Requirements for a working processor in stage:
  - access to crash bucket in s3
  - access to elasticsearch cluster in stage (bug 1399576)
  - access to postgres database in stage (using rds, working afaict, bug 1397531)
  - access rmq cluster (not done, bug 1406003)
  - working crontabber in stage

Comment 1

[Miles Crabill [:miles]]

:willkg and I talked about how parts of socorro, such as the processor and submitter, use the configuration value 'resource.boto.temporary_file_system_storage_path' to store files temporarily downloaded.

Leaving a note here that this value, once set, will need to correspond to a directory that is volume mounted into the containers from the host with appropriate permissions.

Comment 2

[Miles Crabill [:miles]]

We've determined that the processor is working as intended, aside from its inability to access private symbols, covered in bug 1437928. Making that block this.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

We wrote a compare script to compare raw and processed crash data between -stage and -new-stage. This helps us determine whether -new-stage is processing the same way that -stage is.

Both environments are processing crashes the same way.

The processor can pull from the rabbitmq queues, mdsw (tmp directories, symbols, etc) is configured correctly and working, -new-stage is saving to the crash stores ... Everything looks good.

I think we're good here. Given that, marking as FIXED.