998370 - gethash completions broken in Beta

Status: VERIFIED FIXED

(Toolkit :: Safe Browsing, defect)

Comment 1

[[:mmc] Monica Chew (no longer reading bugmail)]

Patches to rollback:

https://hg.mozilla.org/releases/mozilla-beta/rev/9617ccbb0f82
https://hg.mozilla.org/releases/mozilla-beta/rev/363950e031c0
https://hg.mozilla.org/releases/mozilla-beta/rev/034a63535df0

Comment 2

[[:mmc] Monica Chew (no longer reading bugmail)]

One more:
https://hg.mozilla.org/releases/mozilla-beta/rev/10904f84c309

Comment 3

[[:mmc] Monica Chew (no longer reading bugmail)]

Attachment: Rollback bugs 997759, 989232, 985720, 985623 in beta (

Comment 4

[[:mmc] Monica Chew (no longer reading bugmail)]

Comment on attachment 8409026 [details] [diff] [review]
Rollback bugs 997759, 989232, 985720, 985623 in beta (

[Approval Request Comment]
Bug caused by (feature/regressing bug #): bugs 997759, 989232, 985720, 985623 collectively converted some preferences to comma-separated lists, which is breaking gethash completions.

User impact if declined: Phishing and malware protection broken in 29

Testing completed (on m-c, etc.): In progress

Risk to taking this patch (and alternatives if risky): It is risky, but the most reasonable thing we can do at this point. GCP found the bug but there's not time to really test it any more. This resets the safebrowsing state to FF 28.

String or IDL/UUID changes made by this patch: None.

Comment 5

[Gian-Carlo Pascutto [:gcp]]

mDisallowCompletionsTables is correctly constructed by splitting the multiple tables, but the other preferences aren't, even though they (can) also contain multiple lists:
http://hg.mozilla.org/mozilla-central/file/45ba19361b97/toolkit/components/url-classifier/nsUrlClassifierDBService.cpp#l1090

We can do a proper fix in Aurora and up.

Comment 6

[[:mmc] Monica Chew (no longer reading bugmail)]

Backouts applied cleanly in this order:

https://hg.mozilla.org/releases/mozilla-beta/rev/034a63535df0
https://hg.mozilla.org/releases/mozilla-beta/rev/9617ccbb0f82
https://hg.mozilla.org/releases/mozilla-beta/rev/10904f84c309
https://hg.mozilla.org/releases/mozilla-beta/rev/363950e031c0

Comment 7

[[:mmc] Monica Chew (no longer reading bugmail)]

Attachment: Rollback bugs 997759, 989232, 985720, 985623 in beta (

Comment 8

[[:mmc] Monica Chew (no longer reading bugmail)]

Comment on attachment 8409045 [details] [diff] [review]
Rollback bugs 997759, 989232, 985720, 985623 in beta (

[Approval Request Comment]
Comment 4.

I tested this patch manually by going to phishtank.com and checking that firefox caught some of the phishing URLs there. Also updates still work.

mchew@mchew-12604:~/mozilla-beta$ ls -al ~/.cache/mozilla/firefox/pepruq2g.safebrowsing-test/safebrowsing/
total 1772
drwxr-xr-x 2 mchew mchew   4096 Apr 18 10:52 .
drwx------ 6 mchew mchew   4096 Apr 18 10:52 ..
-rw-r--r-- 1 mchew mchew     12 Apr 18 10:52 goog-badbinurl-shavar.cache
-rw-r--r-- 1 mchew mchew 244552 Apr 18 10:52 goog-badbinurl-shavar.pset
-rw-r--r-- 1 mchew mchew 258669 Apr 18 10:52 goog-badbinurl-shavar.sbstore
-rw-r--r-- 1 mchew mchew  12332 Apr 18 10:52 goog-downloadwhite-digest256.cache
-rw-r--r-- 1 mchew mchew     16 Apr 18 10:52 goog-downloadwhite-digest256.pset
-rw-r--r-- 1 mchew mchew  14888 Apr 18 10:52 goog-downloadwhite-digest256.sbstore
-rw-r--r-- 1 mchew mchew     12 Apr 18 10:52 goog-malware-shavar.cache
-rw-r--r-- 1 mchew mchew 281784 Apr 18 10:52 goog-malware-shavar.pset
-rw-r--r-- 1 mchew mchew 379563 Apr 18 10:52 goog-malware-shavar.sbstore
-rw-r--r-- 1 mchew mchew     12 Apr 18 10:52 goog-phish-shavar.cache
-rw-r--r-- 1 mchew mchew 335920 Apr 18 10:52 goog-phish-shavar.pset
-rw-r--r-- 1 mchew mchew 218498 Apr 18 10:52 goog-phish-shavar.sbstore
-rw-r--r-- 1 mchew mchew     44 Apr 18 10:52 test-malware-simple.cache
-rw-r--r-- 1 mchew mchew     16 Apr 18 10:52 test-malware-simple.pset
-rw-r--r-- 1 mchew mchew    232 Apr 18 10:52 test-malware-simple.sbstore
-rw-r--r-- 1 mchew mchew     44 Apr 18 10:52 test-phish-simple.cache
-rw-r--r-- 1 mchew mchew     16 Apr 18 10:52 test-phish-simple.pset
-rw-r--r-- 1 mchew mchew    232 Apr 18 10:52 test-phish-simple.sbstore

Comment 9

[[:mmc] Monica Chew (no longer reading bugmail)]

Btw, the latest version of the patch includes the hotfix code to not request problematic tables from Google: bug 985627

Comment 10

[Gian-Carlo Pascutto [:gcp]]

Comment on attachment 8409045 [details] [diff] [review]
Rollback bugs 997759, 989232, 985720, 985623 in beta (

Review of attachment 8409045 [details] [diff] [review]:
-----------------------------------------------------------------

Verified by applying to mozilla-beta and diffing the resulting SafeBrowsing code against mozilla-release. This points to the only difference being HTTPS support and better OOM protection (this is OK and expected, those changes did not produce problems during beta). The Application Reputation code is gone as wanted.

As discussed with mmc, the download_allow_table and download_block_table preferences must be empty. They are not so in the Firefox 28 release (source), but we hotfixed that after the release.

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8409045 [details] [diff] [review]
Rollback bugs 997759, 989232, 985720, 985623 in beta (

Approved

Comment 12

[[:mmc] Monica Chew (no longer reading bugmail)]

remote:   https://hg.mozilla.org/releases/mozilla-beta/rev/4f4941d4cda9

Comment 13

[Catalin Varga [QA][:VarCat]]

I verified the bug using the following environment:
FF 29 RC
Build Id:20140421221237
OS:
Ubuntu 14.04 x32
Mac Os 10.9.2
Windows 7 x64

I verified the bug using the information provided on comment 8

On Ubuntu and Mac I didn't find the following files:

-rw-r--r-- 1 mchew mchew     12 Apr 18 10:52 goog-badbinurl-shavar.cache
-rw-r--r-- 1 mchew mchew 244552 Apr 18 10:52 goog-badbinurl-shavar.pset
-rw-r--r-- 1 mchew mchew 258669 Apr 18 10:52 goog-badbinurl-shavar.sbstore
-rw-r--r-- 1 mchew mchew  12332 Apr 18 10:52 goog-downloadwhite-digest256.cache
-rw-r--r-- 1 mchew mchew     16 Apr 18 10:52 goog-downloadwhite-digest256.pset
-rw-r--r-- 1 mchew mchew  14888 Apr 18 10:52 goog-downloadwhite-digest256.sbstore

If this is not expected behavior please provide additional information.

Comment 14

[Gian-Carlo Pascutto [:gcp]]

Not finding those tables is the expected result (that change was made and is implied in comment 9). Just checking: when you say Ubuntu and Mac, you mean that is what you tested? Or did you find them on Windows instead?

Comment 15

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

Since Catalin verified this on my Win 7 x64 machine, I went ahead and double checked this behavior on Windows. I see that the "goog-badbinurl-shavar" and "goog-downloadwhite-digest256" files display for previous versions (e.g. Firefox 29 Beta 9), but are no longer displayed for Firefox 29 RC.

Comment 16

[Gian-Carlo Pascutto [:gcp]]

Sounds good. Given that the other branches are being fixed in bug 998396, I'll close this?

Comment 17

[:Gavin Sharp [email: [e-mail]]]

(In reply to Gone til 4/28 [:mmc] (please use needinfo) from comment #4)
> Bug caused by (feature/regressing bug #): bugs 997759, 989232, 985720,
> 985623 collectively converted some preferences to comma-separated lists,
> which is breaking gethash completions.

Can you correctly update those bugs to be marked as status-firefox29/30: disabled?

Comment 18

[Gian-Carlo Pascutto [:gcp]]

I set them to disabled on firefox29, where we did the backout. They are fixed in firefox30.