1479898 - Consider increasing the application reputation remote lookup timeout

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, enhancement, P3)

Description

[François Marier [:francois]]

We currently see about 3% of beta users failing to reach the application reputation server: https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2018-07-26&keys=__none__!__none__!__none__&max_channel_version=beta%252F62&measure=APPLICATION_REPUTATION_REMOTE_LOOKUP_TIMEOUT&min_channel_version=nightly%252F51&processType=*&product=Firefox&sanitize=1&sort_keys=submissions&start_date=2018-06-25&table=1&trim=1&use_submission_date=0

We should consider increasing that timeout (currently 10 sec, see browser.safebrowsing.downloads.remote.timeout_ms) maybe by 50% (15 sec) to see if it helps at all.

Comment 1

[Dimi Lee [:dimi]]

Attachment: Bug 1479898 - P1. Only set false to APPLICATION_REPUTATION_REMOTE_LOOKUP_TIMEOUT when it is not timeout. r?francois

Right now no matter it is timeout or not, we will always add the counter
to the false case of APPLICATION_REPUTATION_REMOTE_LOOKUP_TIMEOUT.
We should only set the counter(false) when it is not timeout.

Comment 2

[Dimi Lee [:dimi]]

Attachment: Bug 1479898 - P2. Add telemetry to know the response time of download protection remote lookup. r?francois

Telemetry::APPLICATION_REPUTATION_REMOTE_LOOKUP_RESPONSE_TIME can be
used to give us an idea how could we adjust the timeout accordingly.

Comment 3

[Dimi Lee [:dimi]]

Attachment: Bug 1479898 - P3. Increase the application reputation remote lookup timeout to 15sec. r?francois

3% download protection remote lookup failures are from timeout.
Increase the timeout from 10sec to 15sec to see if this help.

Comment 4

[François Marier [:francois]]

Comment on attachment 9007219 [details]
Bug 1479898 - P3. Increase the application reputation remote lookup timeout to 15sec. r?francois

François Marier [:francois] has approved the revision.

Comment 5

[Dimi Lee [:dimi]]

Attachment: request-bug1479898.txt

Comment 6

[François Marier [:francois]]

Comment on attachment 9007216 [details]
Bug 1479898 - P1. Only set false to APPLICATION_REPUTATION_REMOTE_LOOKUP_TIMEOUT when it is not timeout. r?francois

François Marier [:francois] has approved the revision.

Comment 7

[François Marier [:francois]]

Comment on attachment 9007218 [details]
Bug 1479898 - P2. Add telemetry to know the response time of download protection remote lookup. r?francois

François Marier [:francois] has approved the revision.

Comment 8

[François Marier [:francois]]

Comment on attachment 9009944 [details]
request-bug1479898.txt

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Histograms.json.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, Dimi Lee.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1.

5) Is the data collection request for default-on or default-off?

Default ON, all channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5d39eab1a17c
P1. Only set false to APPLICATION_REPUTATION_REMOTE_LOOKUP_TIMEOUT when it is not timeout. r=francois
https://hg.mozilla.org/integration/autoland/rev/4d2c3be4d725
P2. Add telemetry to know the response time of download protection remote lookup. r=francois
https://hg.mozilla.org/integration/autoland/rev/12f48815d40f
P3. Increase the application reputation remote lookup timeout to 15sec. r=francois

Comment 10

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/5d39eab1a17c
https://hg.mozilla.org/mozilla-central/rev/4d2c3be4d725
https://hg.mozilla.org/mozilla-central/rev/12f48815d40f