Find a way to test that malware/phishing/unwanted sites are correctly blocked by Safe Browsing

NEW
Unassigned

Status

()

Toolkit
Safe Browsing
P3
normal
2 years ago
7 months ago

People

(Reporter: francois, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

As mentioned in bug 1240027, we don't have any guarantees as to how long it's going to take for the local lists to contain the entries required to make https://testsafebrowsing.appspot.com work correctly.

We need to find a way to test this reliably before each release.
(In reply to François Marier [:francois] from comment #0)
> As mentioned in bug 1240027, we don't have any guarantees as to how long
> it's going to take for the local lists to contain the entries required to
> make https://testsafebrowsing.appspot.com work correctly.
> 
> We need to find a way to test this reliably before each release.

Do you have an approximate upper bound, even if it's hours? See bug 1240027 comment #9.
(In reply to Tony Mechelynck [:tonymec] from comment #1)
> Do you have an approximate upper bound, even if it's hours? See bug 1240027
> comment #9.

We don't, that's the point of this bug I think.
(In reply to Gian-Carlo Pascutto [:gcp] from comment #2)
> (In reply to Tony Mechelynck [:tonymec] from comment #1)
> > Do you have an approximate upper bound, even if it's hours? See bug 1240027
> > comment #9.
> 
> We don't, that's the point of this bug I think.

Well, bug 1240027 comment #12 and 13 show that at least three of the tests ran after 14 hours idling. Of course this is probably a grossly exaggerated upper bound. For the other tests I detailed what I did in more detail in bug 1240027 comment #12 than in bug 1240027 comment #9 to let you evaluate if maybe I didn't get far enough. OTOH, on Linux where I am, MSDOS/Windows executables can be regarded as irrelevant. Maybe additional tests are needed for (Linux-type) ELF binaries and for (Mac-type) executables or .dmg Unified Builds.
(In reply to Tony Mechelynck [:tonymec] from comment #3)
> (In reply to Gian-Carlo Pascutto [:gcp] from comment #2)
> > (In reply to Tony Mechelynck [:tonymec] from comment #1)
> > > Do you have an approximate upper bound, even if it's hours? See bug 1240027
> > > comment #9.
> > 
> > We don't, that's the point of this bug I think.
> 
> Well, bug 1240027 comment #12 and 13 show that at least three of the tests
> ran after 14 hours idling. Of course this is probably a grossly exaggerated
> upper bound.

I did the same and left a new Firefox profile running for 24 hours. After that long, all of the Safe Browsing test URLs work.

So while it's probably a reasonably consistent upper bound, it's not very useful for the purpose of automated tests or manual smoketests.
> So while it's probably a reasonably consistent upper bound, it's not very
> useful for the purpose of automated tests or manual smoketests.

For automated tests, indeed not.
For manual smoketests, maybe a cautionary warning can be put wherever it is detailed which tests to do and how, telling the smoketesters that these safe-browsing tests should be run in a Firefox instance which has _already_ been running for "some high enough" number of hours before the test.
I asked Google about this and they said that this will be much easier when we use version 4 of the protocol because the test entries are always sent first.

For v2.2, they don't have a good answer.
Priority: -- → P5
Priority: P5 → P3
No longer blocks: 1167038
No longer blocks: 1149867
You need to log in before you can comment on or make changes to this bug.