1570344 - Total-Records response header not always served

Status: RESOLVED WONTFIX

(Cloud Services :: Server: Remote Settings, defect)

Description

[Mathieu Leplatre [:leplatrem]]

It looks like CloudFront does some magic between GET/HEAD responses...

Let's craft a request that will miss the cache:

➜  ~ http "https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/fxmonitor-breaches/records?_limit=1"  
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Content-Length, Next-Page, Retry-After, Pragma, Alert, Cache-Control, Backoff, ETag, Expires, Last-Modified
Cache-Control: max-age=3600
Connection: keep-alive
Content-Length: 237
Content-Type: application/json
Date: Wed, 31 Jul 2019 15:05:39 GMT
ETag: "1564358474902"
Expires: Wed, 31 Jul 2019 16:05:39 GMT
Last-Modified: Mon, 29 Jul 2019 00:01:14 GMT
Next-Page: https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/fxmonitor-breaches/records?_limit=1&_token=eyJsYXN0X29iamVjdCI6eyJsYXN0X21vZGlmaWVkIjoxNTY0MzU4NDc0OTAyfSwib2Zmc2V0IjoyLCJub25jZSI6InBhZ2luYXRpb24tdG9rZW4tODU3ODJiMjUtMTdkMi00YzBiLTk5NWItNWFmNDM1MWRkYTY0In0%3D
Strict-Transport-Security: max-age=31536000
Via: 1.1 1a5ad5812efb2d9880e30a9b73fb3244.cloudfront.net (CloudFront)
X-Amz-Cf-Id: oDXoNBC_HHnoAmhGTEB66jc8rAGuxj1U87YLgxE8I-J97xP4HPsPFg==
X-Amz-Cf-Pop: CDG50-C1
X-Cache: Miss from cloudfront
X-Content-Type-Options: nosniff

{
    "data": [
        {
            "AddedDate": "2019-07-28T00:35:07Z",
            "BreachDate": "2016-01-01",
            "Domain": "anime-planet.com",
            "Name": "AnimePlanet",
            "PwnCount": 368507,
            "id": "dbee8cd7-ffc0-4b34-aee4-cca097ddc9b7",
            "last_modified": 1564358474902,
            "schema": 1564342888393
        }
    ]
}

Now, reach the same with HEAD:

➜  ~ http HEAD "https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/fxmonitor-breaches/records?_limit=1"
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Content-Length, Next-Page, Retry-After, Pragma, Alert, Cache-Control, Backoff, ETag, Expires, Last-Modified
Age: 11
Cache-Control: max-age=3600
Connection: keep-alive
Content-Length: 237
Content-Type: application/json
Date: Wed, 31 Jul 2019 15:05:39 GMT
ETag: "1564358474902"
Expires: Wed, 31 Jul 2019 16:05:39 GMT
Last-Modified: Mon, 29 Jul 2019 00:01:14 GMT
Next-Page: https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/fxmonitor-breaches/records?_limit=1&_token=eyJsYXN0X29iamVjdCI6eyJsYXN0X21vZGlmaWVkIjoxNTY0MzU4NDc0OTAyfSwib2Zmc2V0IjoyLCJub25jZSI6InBhZ2luYXRpb24tdG9rZW4tODU3ODJiMjUtMTdkMi00YzBiLTk5NWItNWFmNDM1MWRkYTY0In0%3D
Strict-Transport-Security: max-age=31536000
Via: 1.1 898f6aa61dbcef9a01708275308d405e.cloudfront.net (CloudFront)
X-Amz-Cf-Id: L6_POZhJfy97MfRJzUr7922Ihihpbn84JQzypm38GkaI8ymuLQV8gQ==
X-Amz-Cf-Pop: MAD51-C1
X-Cache: Hit from cloudfront
X-Content-Type-Options: nosniff

We get a Hit.

If we craft a HEAD request that miss the cache, we get the Total-Records header as expected:

➜  ~ http HEAD "https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/fxmonitor-breaches/records?_limit=24"
HTTP/1.1 200 OK
Access-Control-Expose-Headers: ETag, Total-Records, Next-Page, Total-Objects, Pragma, Backoff, Alert, Last-Modified, Content-Length, Cache-Control, Retry-After, Expires
Cache-Control: max-age=3600
Connection: keep-alive
Content-Length: 11
Content-Type: application/json
Date: Wed, 31 Jul 2019 15:07:12 GMT
ETag: "1564358474902"
Expires: Wed, 31 Jul 2019 16:07:12 GMT
Last-Modified: Mon, 29 Jul 2019 00:01:14 GMT
Strict-Transport-Security: max-age=31536000
Total-Objects: 371
Total-Records: 371
Via: 1.1 3682ae673ce1091d4547ffdcc74acbce.cloudfront.net (CloudFront)
X-Amz-Cf-Id: x5Bkfc3_MxnPWgF2S-FAW4_2hFez37WYq2NAHKl6mpUc4CmevDX_Tg==
X-Amz-Cf-Pop: MAD51-C1
X-Cache: Miss from cloudfront
X-Content-Type-Options: nosniff

Comment 1

[Mathieu Leplatre [:leplatrem]]

This is not really an issue for clients, and I don't think we can do much since this is a CloudFront behaviour....