936041 - Enable new malware download protection on Android

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Download Manager, defect)

Description

[Gian-Carlo Pascutto [:gcp]]

https://bugzilla.mozilla.org/show_bug.cgi?id=895476#c20

Firefox for Android can't currently use the malware download protection.

Comment 1

[Aaron Train [:aaronmt]]

Dependancy fixed, is this enabled?

Comment 2

[Gian-Carlo Pascutto [:gcp]]

I'm guessing it would if we set the prefs. I added the badbinurl table and tried a download, and saw:

I/Gecko   (13706): [13706] WARNING: NS_ENSURE_SUCCESS(Preferences::GetCString("browser.safebrowsing.appRepURL", &serviceUrl), nsresult::NS_ERROR_NOT_AVAILABLE) failed with result 0x8000FFFF: file /home/morbo/hg/mozilla-central/toolkit/components/downloads/ApplicationReputation.cpp, line 948

Which suggests that 1) I need one more pref 2) The right code is being activated.

Comment 3

[Gian-Carlo Pascutto [:gcp]]

Attachment: Patch 1. Enable badbin download blocks on Android.

How can we test this?

Comment 4

[[:mmc] Monica Chew (no longer reading bugmail)]

Comment on attachment 8422386 [details] [diff] [review]
Patch 1. Enable badbin download blocks on Android.

Review of attachment 8422386 [details] [diff] [review]:
-----------------------------------------------------------------

The prefs are correct. Presence of the badbinurl db would be a first step, as well as making sure the unittest runs on android: but it looks like it's already enabled.

http://mxr.mozilla.org/mozilla-central/source/toolkit/components/downloads/test/unit/test_app_rep.js

Comment 5

[:Margaret Leibovic]

Downloads.jsm landed in 37, so we should try finishing this bug.

However, we should be sure to look into how much memory this new database takes up.

Comment 6

[Gian-Carlo Pascutto [:gcp]]

>However, we should be sure to look into how much memory this new database takes up.

│       ├──0.68 MB (00.31%) ── goog-badbinurl-shavar
│       ├──0.00 MB (00.00%) ── goog-downloadwhite-digest256

Comment 7

[François Marier [:francois]]

BTW, I've asked Google if they could add a sample bad APK on http://testsafebrowsing.appspot.com to help with manual end-to-end testing.

Comment 8

[:Margaret Leibovic]

What's preventing us from shipping this? The memory increase?

Comment 9

[François Marier [:francois]]

That patch is missing these:

  pref("browser.safebrowsing.downloads.enabled", true);
  pref("browser.safebrowsing.downloads.remote.enabled", true);

Comment 10

[Gian-Carlo Pascutto [:gcp]]

We can fix that, but do we have a test URL nowadays?

Comment 11

[:Margaret Leibovic]

Barbara, this is another item we could track in Aha.

Comment 12

[François Marier [:francois]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #10)
> We can fix that, but do we have a test URL nowadays?

We still don't have a test .apk, but downloading the .exe should trigger the warning on Fennec too.

Comment 13

[Barbara Bermes [:barbara] - NI please!]

(In reply to :Margaret Leibovic from comment #11)
> Barbara, this is another item we could track in Aha.

Added. Is this for prioritization, and status would be in development?

Comment 14

[:Margaret Leibovic]

Can we work on landing this? It sounds like we just need to test it, is that right?

Comment 15

[Gian-Carlo Pascutto [:gcp]]

Attachment: Enable new malware download protection on Android

Francois, should remote lookups be enabled here?

I tested on safebrowsing.appspot.com. The first link gets downloaded (no extension),
the 2 .exes are blocked. But there's no notification whatsoever what happens, the downloads
just disappear from the notification bar.

Margaret, do you think we'd need more UX work in light of the above?

Comment 16

[François Marier [:francois]]

Comment on attachment 8705666 [details] [diff] [review]
Enable new malware download protection on Android

Review of attachment 8705666 [details] [diff] [review]:
-----------------------------------------------------------------

The patch looks good.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #15)
> I tested on testsafebrowsing.appspot.com. The first link gets downloaded (no
> extension),
> the 2 .exes are blocked. But there's no notification whatsoever what
> happens, the downloads
> just disappear from the notification bar.

So this is a bit weird because the first three links under "Download Warnings" should get blocked. The first one doesn't reach the remote lookup step though, it gets blocked because the domain is on the goog-badbin-shavar list. On Desktop we get:

-468416704[7f5de2d755c0]: Application reputation service started up
-468416704[7f5de2d755c0]: Starting application reputation check [query=7f5db6e58360]
-468416704[7f5de2d755c0]: Created pending lookup [this = 7f5db29ada00]
-468416704[7f5de2d755c0]: ApplicationReputation: Got 3 redirects
-468416704[7f5de2d755c0]: ApplicationReputation: Appending redirect http://download.safebrowsingtest.com/download/test
-468416704[7f5de2d755c0]: ApplicationReputation: Appending redirect http://download.safebrowsingtest.com/download/test
-468416704[7f5de2d755c0]: ApplicationReputation: Appending redirect http://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test
-468416704[7f5de2d755c0]: Created pending DB lookup [this = 7f5db6e62bc0]
-468416704[7f5de2d755c0]: Checking principal http://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test [this=7f5db6e62bc0]
-468416704[7f5de2d755c0]: Checking DB service for principal http://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test [this = 7f5db6e62bc0]
-468416704[7f5de2d755c0]: Didn't find principal http://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test on any list [this = 7f5db6e62bc0]
-468416704[7f5de2d755c0]: Created pending DB lookup [this = 7f5db6ecc400]
-468416704[7f5de2d755c0]: Checking principal http://download.safebrowsingtest.com/download/test [this=7f5db6ecc400]
-468416704[7f5de2d755c0]: Checking DB service for principal http://download.safebrowsingtest.com/download/test [this = 7f5db6ecc400]
-468416704[7f5de2d755c0]: Destroying pending DB lookup [this = 7f5db6e62bc0]
-468416704[7f5de2d755c0]: Found principal http://download.safebrowsingtest.com/download/test on blocklist [this = 7f5db6ecc400]
-468416704[7f5de2d755c0]: Application Reputation check failed, blocking bad binary in 145.203032 ms [this = 7f5db29ada00]

Could it be that the badbin list hadn't been downloaded yet when you tested this?

Comment 17

[:Margaret Leibovic]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #15)

> I tested on safebrowsing.appspot.com. The first link gets downloaded (no
> extension),
> the 2 .exes are blocked. But there's no notification whatsoever what
> happens, the downloads
> just disappear from the notification bar.
> 
> Margaret, do you think we'd need more UX work in light of the above?

Yeah, it sounds like we should file a follow-up bug to improve communication here. I feel like we could do something simple like a toast or a dialog that at least explains that we prevented a malicious download from happening.

What does desktop do? Do they show a notification to say that the download was blocked?

Comment 18

[:Margaret Leibovic]

Comment on attachment 8705666 [details] [diff] [review]
Enable new malware download protection on Android

Review of attachment 8705666 [details] [diff] [review]:
-----------------------------------------------------------------

I'm fine with us landing this, but let's file a follow-up bug for a notification.

Comment 19

[François Marier [:francois]]

Attachment: apprep-desktop-screenshot.png

(In reply to :Margaret Leibovic from comment #17)
> What does desktop do? Do they show a notification to say that the download
> was blocked?

This is what Desktop does. The popup menu (right-click) is how you can unblock a download.

Comment 20

[Gian-Carlo Pascutto [:gcp]]

(In reply to François Marier [:francois] from comment #16)
> Could it be that the badbin list hadn't been downloaded yet when you tested
> this?

No, I checked that this was present/up to date. I'll try again with full logging, I was asking because I wasn't sure the extensionless one was supposed to be blocked by our impl.

>What does desktop do? Do they show a notification to say that the download was blocked?

It's also visible in the download manager and can be unblocked there.

Comment 21

[Gian-Carlo Pascutto [:gcp]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #15)
> Francois, should remote lookups be enabled here?

To answer myself, comment 9 already points out that they should.

Comment 22

[Gian-Carlo Pascutto [:gcp]]

I/PRLog   ( 4091): 1972994320[75b2d100]: Application reputation service started up
I/PRLog   ( 4091): 1972994320[75b2d100]: Starting application reputation check [query=92125120]
I/PRLog   ( 4091): 1972994320[75b2d100]: Created pending lookup [this = 92b23800]
I/PRLog   ( 4091): 1972994320[75b2d100]: ApplicationReputation: Got no redirects [this=92b23800]
I/PRLog   ( 4091): 1972994320[75b2d100]: Created pending DB lookup [this = 8fdf99a0]
I/PRLog   ( 4091): 1972994320[75b2d100]: Checking principal https://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test [this=8fdf99a0]
I/PRLog   ( 4091): 1972994320[75b2d100]: Checking DB service for principal https://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test [this = 8fdf99a0]
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/download/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/download/download/test
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/
I/PRLog   ( 4091): -2055661704[92411e80]: Checking table goog-badbinurl-shavar
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/, hash 0ED4BB8600DFEC2B884F0DFDF88C3189B10B12801BC96429B5B71992417BC6A8 (86BBD40E)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 86BBD40E, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/, hash BB2825E3AA77C51BDD05132BB9590D5AD2E458B1CF47CABDE7B55D823A9B3F71 (E32528BB)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: E32528BB, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/, hash 19702C88242FBBA3AB54EFA12ACF14CB63D53275445547404E884E700F318FFA (882C7019)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 882C7019, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test, hash 9313762AE551A96FE372C3B48916A38B32CF202EA1708B5A6080BAE5E8771479 (2A761393)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 2A761393, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment a2528ba5-a-0468d689-s-sites.googlegroups.com/, hash BE5E9448EEF65A135602D0C824D7126DF0EF981DE5F0C5571C990683AEDA9CC1 (48945EBE)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 48945EBE, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/, hash 33555A007A448D3430D0EE6F31A2283AC8878BF241BD25AC671588E746AA6A91 (5A5533)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 5A5533, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/, hash 57A522E8E8166EFB1FACABFF24D7BDB2028616D8F623D596CAE9A14159EBA660 (E822A557)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: E822A557, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/download/, hash 4DAFF9BFD4C570FDB3FE2AAEF289C038950E0E4E579706D09E770777FA8751EE (BFF9AF4D)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: BFF9AF4D, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/a/safebrowsingtest.com/download/download/test, hash 6A047A9B799B873CA4B95DCE7898B6783C84A49A817A33095E61C2E9C974202B (9B7A046A)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: 9B7A046A, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Checking fragment googlegroups.com/, hash 73A005E0103392CF6FE0697A41895FE65C1613A2BAD7549D6788124C2724A90A (E005A073)
I/PRLog   ( 4091): -2055661704[92411e80]: Probe in goog-badbinurl-shavar: E005A073, found 0
I/PRLog   ( 4091): -2055661704[92411e80]: Found 0 results.
I/PRLog   ( 4091): -2055661704[92411e80]: Found 0 results.
I/PRLog   ( 4091): -2055661704[92411e80]: query took 10ms
I/PRLog   ( 4091): 1972994320[75b2d100]: Didn't find principal https://a2528ba5-a-0468d689-s-sites.googlegroups.com/a/safebrowsingtest.com/download/download/test on any list [this = 8fdf99a0]
I/PRLog   ( 4091): -2055661704[92411e80]: nsUrlClassifierDBServiceWorker::CacheMisses [92413580] 0
I/PRLog   ( 4091): 1972994320[75b2d100]: Suggested filename: test(4) [this = 92b23800]
I/PRLog   ( 4091): 1972994320[75b2d100]: Not eligible for remote lookups [this=92b23800]
I/PRLog   ( 4091): 1972994320[75b2d100]: Application Reputation check passed in 15.931634 ms [this = 92b23800]
I/PRLog   ( 4091): 1972994320[75b2d100]: Destroying pending DB lookup [this = 8fdf99a0]
I/PRLog   ( 4091): 1972994320[75b2d100]: Destroying pending lookup [this = 92b23800]



The blacklisted principal (http://download.safebrowsingtest.com/download/test) doesn't even show up on Android. Is this a problem with the redirect chain getting lost?

Comment 23

[François Marier [:francois]]

> The blacklisted principal (http://download.safebrowsingtest.com/download/test) doesn't even show up on Android. Is this a problem with the redirect chain getting lost?

It certainly looks like it. That http://download.safebrowsingtest.com/download/test URL is the very first one in the redirect chain.

I fear it means that redirects aren't working anywhere in Safe Browsing on Fennec :(

Comment 24

[Gian-Carlo Pascutto [:gcp]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e7350a1a51a59248722e02d66b583e20a3b50425
Bug 936041 - Enable new malware download protection on Android. r=francois f=margaret

Comment 25

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e7350a1a51a5

Comment 26

[Barbara Bermes [:barbara] - NI please!]

In the Aha card, it says Desktop will ship this in 47, and we are proposing 46, any issues with that?

Comment 27

[Gian-Carlo Pascutto [:gcp]]

We shipped basic download protection in Firefox 31 for Desktop.

Comment 28

[Liz Henry (:lizzard) (privacy/fingerprinting team)]

Is this something you might like to put into release notes? If so, can you nominate it and suggest wording? Thanks.

Comment 29

[Gian-Carlo Pascutto [:gcp]]

I don't think it can be nominated yet because bug 1241566 disabled it again.