1838302 - reduce work required for datadictionary views

Status: RESOLVED FIXED

(Socorro :: Webapp, task, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Crash Stats is getting fuzzed by someone hitting the datadictionary views with all manner of variations. I'm pretty sure this is using more webapp CPU than it should.

This bug covers reviewing the datadictionary views and reducing the work required to render datadictionary requests.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6425: bug 1838302: cache computed data for datadictionary

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6425: "bug 1838302: cache computed data for datadictionary" in 15b6f9e.

I'll test this on stage.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6426: bug 1838302: fix cache key creation so they're valid

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6426: "bug 1838302: fix cache key creation so they're valid" in 7ceb6e6.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I threw together a locust script and ran it before and after the first set of changes were deployed.

Before:

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /documentation/datadictionary/dataset/annotation/field/%3Cstar%3E                    3200   3500   3600   3600   3800   3800   3800   3800   3800   3800   3800      7
GET      /documentation/datadictionary/dataset/annotation/field/AppInitDLLs                   3200   3400   3500   3600   6000   6000   6000   6000   6000   6000   6000      9
GET      /documentation/datadictionary/dataset/annotation/field/foo                           3500   3900   3900   4100   4100   4100   4100   4100   4100   4100   4100      5
GET      /documentation/datadictionary/dataset/annotation/field/submitted_timestamp           3800   4000   4100   4100   4300   5700   5700   5700   5700   5700   5700     11
GET      /documentation/datadictionary/dataset/processed/field/cpu_microcode_version          4900   4900   5000   5000   6600   6600   6600   6600   6600   6600   6600      7
GET      /documentation/datadictionary/dataset/processed/field/phc_base_address               4900   4900   5000   5000   5000   5000   5000   5000   5000   5000   5000      4
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           3700   4000   4400   4500   5000   5700   6600   6600   6600   6600   6600     43

After:

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /documentation/datadictionary/dataset/annotation/field/%3Cstar%3E                     720    820    870    910   1600   1600   1900   3400   3400   3400   3400     56
GET      /documentation/datadictionary/dataset/annotation/field/AppInitDLLs                    420    520    610    670    750    930   2600   2600   2600   2600   2600     42
GET      /documentation/datadictionary/dataset/annotation/field/foo                            670    810    880    910   1100   1500   3500   3500   3500   3500   3500     47
GET      /documentation/datadictionary/dataset/annotation/field/submitted_timestamp            310    410    520    560    910   1300   3100   3100   3100   3100   3100     43
GET      /documentation/datadictionary/dataset/processed/field/cpu_microcode_version           450    510    740   1300   2000   2200   3200   3200   3200   3200   3200     40
GET      /documentation/datadictionary/dataset/processed/field/phc_base_address                410    460    520    520   1500   2800   3100   3100   3100   3100   3100     41
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                            510    640    750    840   1300   1700   3100   3200   3500   3500   3500    269

Caching definitely dropped the timings down and a lot more requests are going through in the same minute of time (43 vs. 269). I'm assuming it drops the CPU usage down as well. Grafana isn't working, so I can't check.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6427: bug 1838302: fix cache keys for nested protected data

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6427: "bug 1838302: fix cache keys for nested protected data" in 8559f3c.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

I deployed this just now with bug #1839444. Marking as FIXED.