1409475 - make api token requirement for fetch_crash_data.py clearer

Status: RESOLVED FIXED

(Socorro :: Processor, task, P1)

Description

[Osmose [:osmose, :mkelly]]

I followed the steps from http://socorro.readthedocs.io/en/latest/components/processor.html#example-using-all-the-scripts to process a crash from prod (10d7bf20-a19c-48bd-9dfe-0174b0170918):

> $ ./docker/as_me.sh bash
> Starting socorro_elasticsearch_1 ...
> Starting socorro_localstack-s3_1 ... done
> Starting socorro_localstack-s3_1 ... done
> I have no name!@643423e77324:/app$ ./scripts/fetch_crash_data.py ./crashdata 10d7bf20-a19c-48bd-9dfe-0174b0170918
> Working on 10d7bf20-a19c-48bd-9dfe-0174b0170918...
> Fetching 10d7bf20-a19c-48bd-9dfe-0174b0170918
> I have no name!@643423e77324:/app$ ./scripts/socorro_aws_s3.sh sync ./crashdata/ s3://dev_bucket/
> S3 container bucket is dev_bucket
> upload: crashdata/v2/raw_crash/10d/20170918/10d7bf20-a19c-48bd-9dfe-0174b0170918 to s3://dev_bucket/v2/raw_crash/10d/20170918/10d7bf20-a19c-48bd-9dfe-0174b0170918
> upload: crashdata/v1/dump_names/10d7bf20-a19c-48bd-9dfe-0174b0170918 to s3://dev_bucket/v1/dump_names/10d7bf20-a19c-48bd-9dfe-0174b0170918
> I have no name!@643423e77324:/app$ ./scripts/add_crashid_to_queue.py socorro.normal 10d7bf20-a19c-48bd-9dfe-0174b0170918
> Configuration:
> host:         rabbitmq
> port:         5672
> user:         rabbituser
> password:     ********
> virtual_host: rabbitvhost
> queue:        socorro.normal
> # crashids:   1
> 
> Sending 10d7bf20-a19c-48bd-9dfe-0174b0170918 to socorro.normal....
> Done!
> I have no name!@643423e77324:/app$ exit
> exit
> 
> $ docker-compose up processor

Here's the output of the processor with most of the unimportant bits removed: https://gist.githubusercontent.com/Osmose/a1e828230805d3913b1485f04ddb78b7/raw/5f3d19b5dc8ec6392a3446387dc4468daa55e2e2/gistfile1.txt

As far as I can tell, it isn't running stackwalker on the dump, which leads to the missing json_dump key. I can't figure out if I'm missing anything in my steps or environment that will make the processor actually properly process the crash.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to look into now.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I think this is because you don't have an API token that has "View raw dumps" permission. There's some setup involved here:

http://socorro.readthedocs.io/en/latest/components/processor.html#scripts-fetch-crash-data-py

I haven't verified that that's the issue (i.e. process a crash with a dump successfully). I can when I get back later.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

One outcome of this could be to make this clearer in the docs. I'm definitely game for that.

Comment 4

[Osmose [:osmose, :mkelly]]

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #2)
> I think this is because you don't have an API token that has "View raw
> dumps" permission. There's some setup involved here:
> 
> http://socorro.readthedocs.io/en/latest/components/processor.html#scripts-
> fetch-crash-data-py
> 
> I haven't verified that that's the issue (i.e. process a crash with a dump
> successfully). I can when I get back later.

That was the issue, and setting that fixed it.

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #3)
> One outcome of this could be to make this clearer in the docs. I'm
> definitely game for that.

If possible, it might also be nice to make fetch_crash_data.py log a message when it fails to fetch all the data for a crash due to lack of permissions.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

That seems prudent. I'll tweak it now.

Comment 6

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/42211495a33e87290d69e1416b3d0f3044fd956e
fixes bug 1409475 - improve docs around API token requirement

https://github.com/mozilla-services/socorro/commit/69db3f233e288aa5b9cd4fc9211a55a56161e96b
Merge pull request #4045 from willkg/1409475-mdsw

fixes bug 1409475 - improve docs around API token requirement

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Tweaking the summary to match what we ended up doing.