1523705 - Raptor tests do not run with url-classifier tables downloaded

Status: RESOLVED INACTIVE

(Testing :: Raptor, defect, P3)

Description

[(no longer active)]

STR:

1. Run ./mach raptor-test -t any-test --debug-mode.
2. Go to about:url-classifier.

You will see that the provider list is empty:

Provider
Provider Last update time Next update time Back-off time Last update status Update
google N/A N/A N/A N/A
google4 N/A N/A N/A N/A
mozilla N/A Wed Dec 31 1969 19:00:00 GMT-0500 (Eastern Standard Time) Tue Jan 29 2019 13:59:13 GMT-0500 (Eastern Standard Time) N/A

This means that when these tests run, the parts of the browser that depend on the url-classifier data to be available (like enhanced tracking protection) will not function correctly.

Comment 1

[Andrea Marchesini [:baku]]

The reason why raptor tests don't have safe-browsing tables updated, is because browser.safebrowsing.provider.* prefs are all set to http://127.0.0.1/safebrowsing-dummy/something. privacy.trackingprotection.annotate_channels is disabled too. See this:

https://searchfox.org/mozilla-central/rev/78cd247b5d7a08832f87d786541d3e2204842e8e/testing/profiles/perf/user.js#29-34

Discussing this with Dimi, he says that, the reason why we disable safe-browsing in these tests, is because it can harm the performance, and it's hard to guarantee the correct downloading of the lists (network issues, etc) and also hard to know the timing the download is triggered. That may cause lots of nondeterministic problems.

Note, that in general, at the first loading, it's possible that the list has not been downloaded yet on a fresh profile.

Comment 2

[(no longer active)]

Is it possible to "prime the profile" by running Firefox first and run some code to trigger a download of the tables, wait for the download to finish and then start running the tests?

Comment 3

[Dimi Lee [:dimi]]

(In reply to :Ehsan Akhgari from comment #2)

Is it possible to "prime the profile" by running Firefox first and run some code to trigger a download of the tables, wait for the download to finish and then start running the tests?

Regarding "run some code to trigger a download of the tables, wait for the download to finish", to manually trigger SafeBrowsing download, there are two ways:

1. Using listmanager API[1]
2. Modify preference "browser.safebrowsing.provider.mozilla.nextupdatetime" to 1

and then listen to "safebrowsing-update-finished" or monitor the preference to wait for the download to finish.

Since the tables we care about here are our tables, another approach is to write a script that converts the list to SafeBrowsing file format and then preinstalls it. I am not sure which approach is easier(probably manually trigger the download), but if such preinstallable tables exist, it may be easier for the other tests that have the same requirement to integrate(assume preinstall is easy...)

[1] https://searchfox.org/mozilla-central/rev/dd7e27f4a805e4115d0dbee70e1220b23b23c567/toolkit/components/url-classifier/nsIUrlListManager.idl#80