1553842 - the URL classifier should not attempt an update on every Gecko startup - especially on Android

Status: RESOLVED DUPLICATE of bug 1531354

(Toolkit :: Safe Browsing, defect, P2)

Description

[Randell Jesup [:jesup] (needinfo me)]

From geckoview_example profiles like https://perfht.ml/2HRuy2Q it's clear that the URL classifier is doing updates on every Gecko startup, shortly after start and often right in the middle (or before) the initial page requested by an intent loads.

On android in particular (and to a lesser extent on desktop) Gecko may be started frequently and often with an initial URL to load. On android this is especially true, as the OS may kill of the Gecko process aggressively.

There are several options, which I don't have a strong opinion on:

• deferring updates until very idle (not just requestIdleCallback())
• not updating if the last update was within N minutes/hours (and wait for the next "regular" check, using the time of the last check as a guide when to schedule a check)

Comment 1

[Chris Peterson [:cpeterson]]

We probably need a centralized (Gecko or Android) job scheduler to handle the various startup or periodic update checks.

Comment 2

[Dimi Lee [:dimi]]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #0)

There are several options, which I don't have a strong opinion on:

• deferring updates until very idle (not just requestIdleCallback())

I think this is something we can do, but I also think we should include a hard limit to specify how long Safe Browsing can wait when the periodic update timer is fired. Otherwise, a busy CPU may cause Safe Browsing database never being updated(An attacker may make use of this).

The reason that keeps Safe Browsing database up to date is important is because the phishing lists are changing very frequently, not updating means Firefox users are vulnerable to phishing attacks.

• not updating if the last update was within N minutes/hours (and wait for the next "regular" check, using the time of the last check as a guide when to schedule a check)

Safe Browsing doesn't attempt an update on startup. Right now, we have two scheduled update for v2 tables(tracking protection, flash block...etc) and v4 tables(phishing protection).
For V2 updates, it is triggered every 60mins. For V4 updates, it is triggered every 30 mins(These two timers don't have to sync)

It looks like Safe Browsing triggers an update every startup because the scheduled timer is up. We can use about:url-classifier to see the next scheduled update.
But if you are seeing SafeBrowsing attempts an update on every startup, this should be a bug.

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

New profile, on the second run within a minute or so of killing the first run, which I allowed to load a URL and sit for a few minutes, so timers should not have expired. URL classifier seems busy for ~170ms immediately on startup, opening file and writing files.

https://perfht.ml/2W08BZn

Comment 4

[Dimi Lee [:dimi]]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #3)

New profile, on the second run within a minute or so of killing the first run, which I allowed to load a URL and sit for a few minutes, so timers should not have expired. URL classifier seems busy for ~170ms immediately on startup, opening file and writing files.

https://perfht.ml/2W08BZn

I think I know the root cause, it is because we wrote test entries on startup - Bug 779008 Comment 7
I am already working on that and I plan to fix it before 69.

Comment 5

[Dimi Lee [:dimi]]

This is fixed in Bug 1531354 by loading test entries directly in lookup cache instead of using an update to store test entries.