1752022 - Twitch streaming video gets written to the disk cache

Status: RESOLVED INVALID

(Core :: Networking: Cache, task)

Description

[Jeff Muizelaar [:jrmuizel]]

Attachment: Cache entry information.htm

This is probably Twitch's fault but it would be good to confirm and provide them with a suggestion for how to fix it or otherwise solve the problem they're trying to solve without having to write to disk.

I've attached a sample cache entry

Comment 1

[Valentin Gosu [:valentin] (he/him)]

With no cache-control headers present, we will default to caching the response.

Youtube seems to also write files into the disk cache.
However, it would be good for twitch to add some cache-control headers to their responses.
Here's a sample of the response headers from a youtube video request:

last-modified: Wed, 26 Jan 2022 07:27:27 GMT
content-type: audio/webm
date: Wed, 26 Jan 2022 10:30:31 GMT
expires: Wed, 26 Jan 2022 10:30:31 GMT
cache-control: private, max-age=21296
accept-ranges: bytes
content-length: 287408
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
client-protocol: quic
access-control-allow-origin: https://www.youtube.com
access-control-allow-credentials: true
timing-allow-origin: https://www.youtube.com
access-control-expose-headers: Client-Protocol, Content-Length, Content-Type, X-Bandwidth-Est, X-Bandwidth-Est2, X-Bandwidth-Est3, X-Bandwidth-App-Limited, X-Bandwidth-Est-App-Limited, X-Bandwidth-Est-Comp, X-Bandwidth-Avg, X-Head-Time-Millis, X-Head-Time-Sec, X-Head-Seqnum, X-Response-Itag, X-Restrict-Formats-Hint, X-Sequence-Num, X-Segment-Lmt, X-Walltime-Ms
vary: Origin
cross-origin-resource-policy: cross-origin
x-content-type-options: nosniff
server: gvs 1.0

If they want to disable disk cache completely, they can use no-store, but that might make things slower if people are seeking through a video a lot. But I'm not totally sure about the implications for live-video vs recorded video.

In any case, all of the available caching headers are documented here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

Let me know if you think this bug should stay open instead.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Would setting no-store affect CDN behaviour? You can't seek in a Twitch live stream so it's pretty unlikely that an individual client would re-request a particular chunk of the video but lots of client will request the same chunk so you'd want each chunk to be cached on a CDN.

Comment 3

[Valentin Gosu [:valentin] (he/him)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #2)

Would setting no-store affect CDN behaviour?

I think it would. But I think they can use s-maxage to control that separately.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

So something like Cache-Control: max-age=0, s-maxage=604800? Do we avoid writing resources with a max-age of 0 to disk?

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Unfortunately only no-store causes us to skip disk cache (from what I can tell). I'm not sure if that's compatible with CDNs.
Cloudflare for example uses separate headers for CDN caching decisions

Comment 6

[Jeff Muizelaar [:jrmuizel]]

I reached out to Twitch and they said they would improve the cache headers.