1008706 - Google.com PREF cookie keeps coming back even with network disabled and cookies disabled

Status: RESOLVED INVALID

(Toolkit :: Safe Browsing, defect)

Description

[A, T]

Attachment: Screenshot from 2014-05-11 15:11:39.png

Last 2 days I've noticed that a certain cookie that shouldn't even be there popping up in my cookie list, I've heard this comes from Safebrowsing, unfortunately it keeps respawning even with all of that disabled, on top off that, my "Accept cookies from sites" is turned off, so I should naturally assume that everything but my whitelisted cookies should not be there. 

Why is this allowed?

Comment 2

[(no longer active)]

Has anyone looked to see which network request this is coming from?

Comment 3

[(no longer active)]

(Should be easy to test by turning on some NSPR logging options...)

Comment 4

[maxhuber80]

I'm experiencing the exact same problem on Windows 8.1 with FF30. Reproducable by:

1) pull your network cable (go physically offline!)
2) Block ALL coockies
3) delete the remaining ones
4) in about:config clear out all safebrowsing stuff (including URLs)
5) filter for "google" in about:config and clear everything

(sum up of step 1 to 5: go physcially offline and block all "background connection" that FF could make to google)

6) look into your cookie window: fing google.com PREF cookie with unique ID.
7) delete it.
8) directly close and directly reopen only the cookie window
9) find the google cookie again with the same unique ID
10) worry about NSA

Comment 5

[:Gavin Sharp [email: [e-mail]]]

If you open the Browser Console (Tools->Web Developers->Browser Console, or Cmd/Ctrl+Shift+J) and flip to the network tab, it should be possible to figure out which network request is setting this pref (by inspecting each one and then seeing other details about it). Can someone affected do that and report back here?

Comment 6

[Matthew N. [:MattN]]

The list of plugins and extensions from about:plugins and about:support respectively would also be useful. Testing in safe mode would temporarily disable all extensions (but not plugins).

Comment 7

[Garrett Robinson [:grobinson]]

> If you open the Browser Console (Tools->Web Developers->Browser Console, or Cmd/Ctrl+Shift+J) and flip to the network tab, it should be possible to figure out which network request is setting this pref

In normal Firefox, the Safe Browsing requests have the PREF cookie set. Here's an example from the Browser Console's Net selection:

GET https://safebrowsing-cache.google.com/safebrowsing/rd/ChVnb29nLWJhZGJpbnVybC1zaGF2YXIQARjt4AEggOEBMgdtcAAA__8P [HTTP/1.1 200 OK 843ms]

So now there are two separate things to test:

1. Are Safe Browsing requests being sent, even when it is disabled (browser.safebrowsing.enabled)?
2. Are cookies being stored, even when they are disabled? (Uncheck "Accept Cookies from Sites" in the privacy prefs)

Comment 8

[Garrett Robinson [:grobinson]]

Side note: you can debug cookies by running a debug build of Firefox with the following environment variables set:

export NSPR_LOG_FILE=/tmp/cookie.log
export NSPR_LOG_MODULES=cookie:5

Comment 9

[Garrett Robinson [:grobinson]]

Attachment: cookies.log

Attempt to reproduce:

1. Create a new profile
2. Disable cookies
3. Disable safebrowsing
4. Check that there are no cookies stored - confirmed the list is empty.
5. Visit reddit.com while watching the Net logging in the Browser Console. No safe browsing requests are logged.
6. Visit facebook.com. *Safe browsing requests are logged!!*
7. Recheck the cookie list - there are still no cookies stored. I do not see the PREF cookie.

I've attached the NSPR log for cookie events. I see COOKIE NOT ACCEPTED for every request, including the Safe Browsing requests, which is what I would expect. However, I also see COOKIE NOT SENT for many requests, including Safe Browsing. If no cookie was accepted, I would not expect it to also not be sent (I would expect it not to be stored in the first place). However, I may be misunderstanding the language in the logging.

Upshot:
1. I was unable to reproduce the phantom PREF cookie (although I only visited two sites)
2. I saw logging for Safe Browsing requests, even when it was disabled. Filed Bug 1025358

Comment 10

[Garrett Robinson [:grobinson]]

> everything but my whitelisted cookies should not be there

This might be a dumb question, but - do you have any Google domains whitelisted?

Comment 11

[maxhuber80]

Maybe you did not read my comment fully:

Go OFFline physically (there can NOT be send any http request!). Delete the cookie. Find the cookie again, directly.

Browser console stays empty. The cookie gets generated without HTTP request!

I only have Adobe Acrobat plugin (latest version). 

My extensions (all latest):
Adblock Edge	2.1.2	true	{fe272bd1-5f76-4ea4-8501-a05d35d823fc}
Disconnect	3.14.0	true	[e-mail]
HTTPS-Everywhere	4.0development.17	true	[e-mail]
NoScript	2.6.8.28	true	{73a6fe31-595d-460b-a920-fcc0f8843232}

If you search the web for this problem, you'll also find some people figured out that Google Chrome does automagically generate this cookie, but Internet Explorer and Opera do not generate it. And other people have this cookie without extensions on a clean profile.

Comment 12

[maxhuber80]

Oh, just made another "great" discovery!

Let the cookie be created. Close Firefox. Open the cookies.sqlite with your favorite SQLite Database Browser. Do NOT find the cookie in there.

Try the same when Firefox is running: Open cookie window and see the PREF cookie with your eyes. In parallel open the cookie.sqlite file and find out that the PREF cookie is NOT in there!

Comment 13

[maxhuber80]

I did some further research:
It's NONE of these extensions (the onyl that I have): Adblock Edge, Disconnect, HTTPS-Everywhere, NoScript. I tried with them all disabled and even did a quick search in their source code for "google.com" and "PREF" without reasonable results. The unique google.com PREF cookie reappeared automagically itself:

- without HTTP request 
- with DISabled network card.
- without appearing in cookies.sqlite

It seems to be present only in RAM when Firefox is running. And since it has always the same unique content it must be generated somehow from parameters of the user's system.

The content of the cookie is:

ID=<firstUniqueID>:TM=<secondUniqueID>:LM=<againSecondUniqueID>:S=<thirdUniqueID>

You will find online news about leaks that NSA uses/spoofs/cracks exactly this google.com PREF cookie to track people.

Comment 14

[maxhuber80]

Sorry, for double posting: I was wrong: The cookie DOES appear in cookies.sqlite!

But it has appId value "-2", whilee ALL others have "0". so this is a special cookie somehow.

Comment 15

[maxhuber80]

I found a solution!

- close FF
- Open cookies.sqlite in a SQLite program.
- Search the PREF cookie and change the field "appId" from "-2" to "0".
- Write the changes to the database.
- open FF
- go to your cookie window
- delete the PREF cookie
- close FF
- open FF
- The cookie stays away! :)

So, somehow Firefox seems to know different kinds of cookies. And for some obscure reason the google.com PREF cookie has an appId of "-2" while all regular cookies have one of "0". And it seems FF has build in that cookies with "-2" can't be deleted.

Please fix this!

Comment 16

[[:mmc] Monica Chew (no longer reading bugmail)]

(In reply to maxhuber80 from comment #14)
> Sorry, for double posting: I was wrong: The cookie DOES appear in
> cookies.sqlite!
> 
> But it has appId value "-2", whilee ALL others have "0". so this is a
> special cookie somehow.

This is because safebrowsing uses its own cookie jar to avoid leaking the regular pref cookie in safebrowsing requests. This work was completed in bug 897516 as of FF 27.

The fact that you are seeing this cookie at all means that safebrowsing is still enabled, because only the safebrowsing code knows to set that appId. Please check that both browser.safebrowsing.enabled and browser.safebrowsing.malware.enabled are set to false. These are exposed in the UI under Prefs > Security > "Block reported attack sites" and "Block reported web forgeries."

Comment 17

[maxhuber80]

I checked several times, EVERYTHING is empty or "false" in about:config under "safebrowsing"! Still I could only get rid of the cookie by manually manipulating the cookies.sqlite.

This problem seemed to appear in one of the latest versions, because there exist old tutorials where disabling safebrowsing was enough to delete the PREF cookie, but all websites/blogs about this cookie in the last 6 or so months report that it is indestructible.

Comment 18

[[:mmc] Monica Chew (no longer reading bugmail)]

(In reply to maxhuber80 from comment #17)
> I checked several times, EVERYTHING is empty or "false" in about:config
> under "safebrowsing"! Still I could only get rid of the cookie by manually
> manipulating the cookies.sqlite.

Hi Max,

In comment 4 in your STR:

> 5) filter for "google" in about:config and clear everything

The prefs to disable safebrowsing do not contain the string "google." Can you please check one more time that both browser.safebrowsing.enabled and browser.safebrowsing.malware.enabled are false? I appreciate your patience.

Any cookies that actually appear in the cookie UI are sent or set by safebrowsing, because I don't think that the cookie manager knows about appIds other than 0. The same is true of private browsing cookies (the cookie manager doesn't know how to display those either during a private browser session).

Thanks,
Monica

Comment 19

[maxhuber80]

My settings look almost like the attached picture from the beginning (except with "" instead of "foo" for the URL's).

And as I said: The cookie stays away now after I changed the appId from -2 to 0 manually with a SQLite editing tool and deleted the cookie through the regulat FF cookie window afterwards.

Summary: FF showed the "-2" cookie, even though safebrowsing was fully disabled and google.com is fully blocked. Changing the cookie manually to "0" and deleting it solved the problem for me.

Comment 20

[Ed Morley [:emorley]]

(In reply to maxhuber80 from comment #19)
> My settings look almost like the attached picture from the beginning (except
> with "" instead of "foo" for the URL's).

To save everyone else having to look at the screenshot, both browser.safebrowsing.enabled and browser.safebrowsing.malware.enabled are set to false in it.

Comment 21

[Gian-Carlo Pascutto [:gcp]]

I've been trying to make sense of this bug and see which parts I can reproduce, and one thing that fell out is that on a normal, clean profile, we are blocking the sending of the cookie because safebrowsing-cache.google.com is considered a 3rd party until you've actually visited Google. That's probably harmless but it certainly can't have been intentional.

Comment 22

[Gian-Carlo Pascutto [:gcp]]

FWIW, I can't reproduce the original report at all.

1) Make a fresh profile.
2) Watch network/cookie traffic: no SafeBrowsing cookies are sent because of the 3rd party restriction.
3) Forcibly visit google.com (just doing a search doesn't work because I get a localized cookie)
4) See SafeBrowsing traffic use it's -2 cookie now because google.com is known.
5) Disable SafeBrowsing.
6) Close Firefox.
7) Inspect cookies.sqlite and delete all Google cookies.
8) Restart Firefox.
9) No SafeBrowsing traffic observed.
10) No SafeBrowsing cookie visible.

I think the very first report possibly failed to disable SafeBrowsing properly, and the second one is likely related to the cookie manager not knowing about the sandboxed cookie. To verify the latter, I did:

1) Enable SafeBrowsing.
2) Visit google.com.
3) Check that there is SafeBrowsing traffic after a while.
4) Open Settings->Privacy->Delete cookies
5) Delete the PREF cookie that I verified in the network traffic to correspond to SafeBrowsing.
6) See all PREF cookies disappear in cookie manager.
7) Reopen cookie manager and see that the cookie is back.

I did one final test:

1) Disable cookies in the privacy settings.
2) Enable SafeBrowsing.
3) Wait and check the network traffic for a SafeBrowsing update.

This did not generate a cookie (COOKIE NOT ACCEPTED/COOKIE NOT SENT) and cookies.sqlite stayed empty.

Based on this, I'm closing this bug as INVALID unless someone posts actual working STR showing there is a problem. I will open a new bug about the inability of the cookie manager to delete the SafeBrowsing cookie if it is already there.

Comment 23

[Gian-Carlo Pascutto [:gcp]]

Filed bug 1026538 for the issue with the cookie manager.

Comment 24

[Garrett Robinson [:grobinson]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)
> [...] on a normal, clean profile, we are blocking the sending of the cookie because
> safebrowsing-cache.google.com is considered a 3rd party until you've
> actually visited Google.

Note that this cookie behavior is only the default on Nightly and Aurora. Since this regularly causes confusion, it has been proposed that we get rid of this difference in the default prefs (bug 999170).