1432274 - A Bad 'Safebrowsing' Causes High CPU Usage & Slow Startup Times

Status: NEW

(Toolkit :: Safe Browsing, defect, P3)

Description

[therube]

A Bad 'Safebrowsing' Causes High CPU Usage & Slow Startup Times

STR:

Create a new Profile
Open FF
Quit
Reopen FF
Quit

Replace existing Profile/safebrowsing/ directory contents with that contained in safebrowsing_FF_52-BAD.zip

Start FF
 
 

Note that the Profile is slow, 20+ seconds on a relatively slow computer, to load the "home page" (regardless of what that page may or may not be) & during that time CPU using is high, ~100% of 1 core.

Observed in FF 52.6 & SeaMonkey 2.49.1, WinXP.
(Unable to test FF 57 at present. Though the mozillazine thread [see, Additional, below] seems to indicate that FF 57 is also affected.)
 
 
No idea what caused Safebrowsing to go "bad", but it has gone bad on at least two of my Profiles.  No idea what expected dates of the files in the safebrowsing directory should be, if files from 01-15-2018 (or earlier) would be considered outdated?
 
 
Starting FF in Safe Mode bypasses the issue.
Setting the home page to a web page, say, www.google.com, makes it easier to "see" the issue.
You'd also want to monitor CPU usage & duration during startup.

Additional information, https://forums.informaction.com/viewtopic.php?f=7&t=24461.

Comment 1

[therube]

Attachment: safebrowsing_FF_52-BAD-reduced-testcase.zip

Replace existing /safebrowsing/ directory contents with that in this ZIP.
(Orig ZIP was too large to attach, but I think this reduced testcase should suffice.  If not, full ZIP can be gotten from 'Additional' link.)

Comment 2

[therube]

FF 57 & 58 have a different /safebrowsing/ format so don't even know if my (reduced, bugzilla) testcase applies.
Nonetheless, simply by coping those two files into the /safebrowsing/ directory, FF 57 & 58 are likewise affected.

(Win7 x64, FF x64)

Comment 3

[François Marier [:francois]]

I can reproduce this but only on ESR 52. The reason is that this phishing database file is using the old V2 format and Firefox 57+ uses the new V4 format.

Comment 4

[François Marier [:francois]]

Actually, nevermind, I was wrong about being able to reproduce this.

Does the problem go away after the phishing list gets updated or does it persist forever once the profile gets into this bad state?

Comment 5

[therube]

> Does the problem go away after the phishing list gets updated or does it persist forever once the profile gets into this bad state?

Judging by file dates (also available through the above link, full zips), I'm going to say that on the Profiles in question, they are not updating, have stopped updating.

One Profile - file dates of Nov/Dec 2017, exceptions being the "test" files, dated current (at the time).
Other Profile - file dates of 12/13 or 1/15, again exceptions being the "test" files, dated current (1-22-2018).

I don't know how often the files are expected to update, whether week old (1/15, at the time) is stale?
Nor do I know how to attempt to force an update?

In all cases, safebrowsing is enabled, prefs are at their defaults.
 
 
I'll will note, that my Profile are not in places like %appdata%, but where I specify, so in that regard, /cache/ & /safebrowsing/ are in my Profile Folder itself - not that I would expect any difference in that regard. (Likewise, there is nothing external, like antivirus, that would be interfering in any way.)

Comment 6

[François Marier [:francois]]

(In reply to therube from comment #5)
> I don't know how often the files are expected to update, whether week old
> (1/15, at the time) is stale?

The goog-* lists are expected to be updated several times a day. So yes, you appear to have stale lists.

> Nor do I know how to attempt to force an update?

You can set the following in about:config:

  browser.safebrowsing.provider.google.nextupdatetime = 1

and then restart the browser.

In recent versions of Firefox, there's a handy about:url-classifier debug page, but 52 doesn't have it.

Comment 7

[therube]

(oh, just going to ramble...)

it appears that once a /safebrowsing/ goes bad, it will not update

having changed browser.safebrowsing.provider.mozilla.nextupdatetime to '1' (or reset its value), it may or may not update its value to current - depending - though thinking that in most instances it will, though once bad, you don't actually get updates

not all non-updating /safebrowsing/ issues result in slow browser startup, high CPU usage
in those cases, one could perpetually be using stale data

one likely would even be aware anything was amiss particularly, unless something odd happened, like observing the file dates & saying, "hey, that doesn't look right"

browser.safebrowsing.debug (thanks frg), starts with : needsUpdate:, guessing has to do with nextupdatetime's '1' setting, then it reports :no updates needed or already initialized:
no updates need is obviously wrong, if already initialized is correct, nothing ever actually completes, so you're never updated

other program, actions may interfere with /safebrowsing/
(i'm actually kind of surprised, well, perhaps, that antivirus does not affect things?)

open a C: prompt in the /safebrowsing/ directory is enough to "lock" (if you will) the directory
you may end up with things like /safebrowsing-backup/, /safebrowsing-to_delete/ - which may or may not persist over time
likewise a directory monitoring program (*Nirsoft's FolderChangesView set to monitor /safebrowsing/) will do similar

obviously none of that is expected or typical, nor am i necessarily saying anything like that has any bearing on /safebrowsing/ going "bad", but situations like that can happen

typical test procedure: new Profile, insert bad /safebrowsing/ data set, start browser, set nextupdatetime to 1, wait for "update" (can verify by viewing nextupdatetime becoming current), reset nextupdatetime to 1, quit, restart, reset, quit, restart, reset... throw in the unexpected, like opening the C: prompt...
 
 
* Nir:
> I checked the issue and I found out that when the monitored folder itself is deleted,
> it's impossible to create it again until stopping the folder monitoring... It's a
> limitation of folder monitoring in Windows operating system, it's not a specific problem
> in  FolderChangesView.
>
> Maybe in future versions I'll try to detect when the monitored folder is deleted and
> stop the monitoring automatically to allow the program to create the folder again.