Accept-Encoding syntax causes upstream cache inefficiency




Networking: HTTP
7 years ago
7 years ago


(Reporter: Mark Nottingham, Assigned: Robert Sayre)



Firefox Tracking Flags

(blocking2.0 final+)



(1 attachment)



7 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-au) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16
Build Identifier: 

Firefox 3.6.3 sends the following Accept-Encoding header:

  Accept-Encoding: gzip,deflate

However, Safari and IE8 both send:

  Accept-Encoding: gzip, deflate

Note the difference in whitespace.

Although HTTP allows caches to canonicalise headers before comparing them for purposes of Vary matching, this is only made explicit in HTTPbis, and many (most?) intermediary cache implementations do not; i.e., they will store responses that Vary on the Accept-Encoding header separately for IE/Safari and FF.

This causes lower cache efficiency (due to wasted space) for intermediaries, as well as more end user-visible latency for people behind proxies (whether they be ISP, enterprise or accelerator/CDN caches). 

For example, if a Safari user visits site A and "primes" the cache with HTTP compressed content, an IE user behind the same cache will be able to get some amount of the content served locally. However, a FF user won't be able to; their first request will need to go to the origin server, because the responses have a different cache key.

It's absolutely true that this is not a Firefox bug, per se; it's fully conformant and the onus should be on the caches to do the right thing. However, there are a large number of deployed caches that won't be changed soon, and their users' experiences can be incrementally improved by a very small change in Firefox -- i.e., adding a single space character to the Accept-Encoding header, bringing it in line with Safari and IE.

Reproducible: Always

Comment 1

7 years ago
Created attachment 455218 [details] [diff] [review]
add a space

I think this will wind its way through our networking code
Attachment #455218 - Flags: review?


7 years ago
Attachment #455218 - Flags: review? → review?(jduell.mcbugs)


7 years ago
Assignee: nobody → sayrer
Blocks: 569993
Comment on attachment 455218 [details] [diff] [review]
add a space

Tested with wireshark.  It works.
Attachment #455218 - Flags: review?(jduell.mcbugs) → review+
Note that we used to have a space there, which went away in bug 181046.
Ever confirmed: true
blocking2.0: --- → ?

Are any of the other headers we removed spaces from in bug 181046 relevant for the caching issue?

Accept-Language: en-us,en;q=0.5
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Added a comment:

// NOTE: separate values with comma+space (", "): see bug 576033
Last Resolved: 7 years ago
Resolution: --- → FIXED

To test this fix, do you have any hints on how to tell in squid whether a resource has been cached once or more when hit by different browsers?

Comment 7

7 years ago
You can either hit it again with those browsers and observe whether it's a cache hit or miss in the logs, or you can turn on debugging.

If this is in the nightlies, I can test it...

The other headers probably deserve separate bugs; because their value space is more diverse, it's not so cut-and-dried.
It will be in tomorrow's nightlies.

Comment 9

7 years ago
[ Sorry for the delay; jetlag is not my friend. ]

Confirmed using Squid 2.7.HEAD.

Request from Safari to fill the cache:
  1278663149.685    446 TCP_MISS/200 3329 GET - DIRECT/ text/html

Subsequent request from FF 3.6.6 results in a cache miss:
  1278663186.525    889 TCP_MISS/200 3329 GET - DIRECT/ text/html

Request to a new URL from Safari:
  1278663111.706    466 TCP_MISS/200 4086 GET - DIRECT/ text/html

Subsequent request from Minefield with Gecko/20100708:
  1278663125.004      5 TCP_MEM_HIT/200 4094 GET - NONE/- text/html
... is a cache hit.

Both of these URLs are cacheable and support compression through negotiation (confirmed with


7 years ago
blocking2.0: ? → final+
You need to log in before you can comment on or make changes to this bug.