1058420 - Add a MozMill test for Application Reputation

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, defect, P1)

Description

[Gian-Carlo Pascutto [:gcp]]

As per https://bugzilla.mozilla.org/show_bug.cgi?id=1057764#c46

Things to cover:

1) Wait for initial SafeBrowsing DB update
2) Test a download of a file that is whitelisted (I'd suggest a previous Firefox release)
3) Test a download of a file that need a remote lookup (A .zip file on a non-mozilla.org/com domain)
4) Test a download of a file that is ignored by extension (none of these: http://dxr.mozilla.org/mozilla-central/source/toolkit/components/downloads/ApplicationReputation.cpp#371)

Ideally:

5) Test a download of a file that is blacklisted (harder to come by, we'd need to host malware on some non-mozilla.org/com domain)

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Before we can start on this test, the patch on bug 908649 has to be landed. It will happen hopefully soon. But it should not stop us from starting to investigate what's necessary here regards of testcase data. Andreea, who could collect the necessary data, and make those available on mozqa.com?

Please keep in mind that this is a Windows only test. I'm setting the priority to P1 due to the major regression in Firefox 32.0 beta.

Comment 2

[Andreea Matei [:AndreeaMatei]]

Andrei will look into that and file a testcase-data bug.

Comment 3

[Andrei Eftimie]

Gian-Carlo, would you mind helping me out with some additional info?
I'm having a hard time finding relevant information for this stuff.

a) Where can I find the "whitelist" mentioned in 1)? Can we modify the list (ie. to add a resource / domain into it so we are sure it is whitelisted?

b) For 2) we would like to use mozqa.com. I think that _should_ trigger a remote lookup. But then again, since its hosted in SCL3 and our test machines are in the same network I'm not sure how the DNS resolving works and if this will be considered a "remote lookup". Do you have any pointers on how to test this?

c) For 5) is there a way for us a add a custom blacklist?

Comment 4

[Gian-Carlo Pascutto [:gcp]]

a) The list comes from Google and we cannot (directly) read or modify it. However AFAIK mozilla.org is whitelisted.
b) The lookups are by domain, IP or network is irrelevant.
c) Yes, but that would mean we test our custom blacklist, not the list the users use. Which means the test would be useless. We should not do that.

If you have a Firefox built with logging, then
export NSPR_LOG_MODULES=nsUrlClassifierDBService:5
should get you the relevant logs to see if a lookup is remote or not.

Monica knows this feature best, needinfoing her to see if she has anything to add.

Comment 5

[[:mmc] Monica Chew (no longer reading bugmail)]

gcp covered all of the points here. For 2) and 3) the binary must be unsigned. In all three cases the user-visible outcome will be that the download succeeds as normal.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Mozmill is dead, WONTFIX the remaining bugs.