1346851 - memory_measures is NOT returned in /api/ProcessedCrash?crash_id=XXX

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[Peter Bengtsson [:peterbe]]

If you open https://crash-stats.mozilla.com/api/ProcessedCrash/?crash_id=07794e51-0de1-4d3a-8c07-33ea72170309 in a browser where you have a logged in session cookie, the key `memory_measures` IS exposed. 
However, if you open it WITHOUT a session cookie, it's NOT exposed. 

For example...

curl https://crash-stats.mozilla.com/api/ProcessedCrash/?crash_id=07794e51-0de1-4d3a-8c07-33ea72170309 2>/dev/null | grep memory_measures

It's curious because 'memory_measures' is equally configured (as a SuperSearchField) as 'json_dump' and THAT does get exposed.

Equally curious, this is NOT reproducible with STAGE.

For example...

curl https://crash-stats.allizom.org/api/ProcessedCrash/?crash_id=f8693d2e-0c00-4eed-a955-a3c7d2170313 2>/dev/null | grep memory_measures

Comment 1

[Peter Bengtsson [:peterbe]]

The background of this is that we can't validate new changes to crash_reports.json when certain fields aren't getting into the combined crash object we intend to save to Telemetry. 
See https://github.com/mozilla/socorro/pull/3688

Comment 2

[Peter Bengtsson [:peterbe]]

Although I haven't fully experimentally proven this yet, I do suspect the problem is that the `ProcessedCrash.API_WHITELIST` is only materialized once. See https://github.com/mozilla/socorro/blob/c2ef5e69636f808a958fb9b170ab18b91d3c6d45/webapp-django/crashstats/api/views.py#L397-L401

Since it's a class attribute [0], it's global per the python process and not a per-wsgi-request thing. 
What `model.API_WHITELIST = model.API_WHITELIST.materialize()` does is that it overrides the `model.API_WHITELIST` from being a Lazy instance [1] to being tuple [2].

The simplest way to resolve this bug, temporarily, is to restart all web heads. Since stage was recently upgraded, its web heads started afresh after these new fields were added. 

The correct, long term, solution is to force ProcessedCrash.API_WHITELIST (and all classes with a similar solution) to reload after ANY Super Search Field is updated. 


[0] https://github.com/mozilla/socorro/blob/c2ef5e69636f808a958fb9b170ab18b91d3c6d45/webapp-django/crashstats/crashstats/models.py#L807
[1] https://github.com/mozilla/socorro/blob/c2ef5e69636f808a958fb9b170ab18b91d3c6d45/webapp-django/crashstats/crashstats/models.py#L105
[2] https://github.com/mozilla/socorro/blob/c2ef5e69636f808a958fb9b170ab18b91d3c6d45/webapp-django/crashstats/crashstats/models.py#L159

Comment 3

[Peter Bengtsson [:peterbe]]

About 1+ hours ago I restarted the web heads in prod. Now what didn't work before works:


▶ curl https://crash-stats.mozilla.com/api/ProcessedCrash/\?crash_id\=07794e51-0de1-4d3a-8c07-33ea72170309 2>/dev/null | grep memory_m
"memory_measures": {

Comment 4

[Peter Bengtsson [:peterbe]]

Attachment: Link to Github pull-request: https://github.com/mozilla/socorro/pull/3698

Comment 5

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/d39c566705661037c51016e424c19b68a5954590
fixes bug 1346851 - don't use API_WHITELIST class attributes (#3698)

* fixes bug 1346851 - don't use API_WHITELIST class attributes

* refactored how cache is cleared on supersearchfields crud