1368936 - Profile on SMB Share - Download Google Safebrowsing database goes minutes

Status: UNCONFIRMED

(Toolkit :: Safe Browsing, defect, P5)

Description

[itd]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20170518000419

Steps to reproduce:

- Move the Firefox profile to a Windows SMB Share
- Start Firefox
- browse to a website


Actual results:

The tab is in status "Connecting" for up to 3 minutes. Then the website shows up. 

After the first website is loaded the browsing experience is as expected. But during the day it happen again. I guess it reloads then the Google Safebrowsing Database.


Expected results:

The website should show up within seconds.

Please note:

FireFox Profile local and "Block dangerous and deceptive content" activated = expected behavior

FireFox Profile on SMB Share and "Block dangerous and deceptive content" activated = described behavior

FireFox Profile on SMB Share and "Block dangerous and deceptive content" deactivated = expected behavior

Comment 1

[Benjamin Smedberg]

I expect the safe browsing database should be in the local profile (AppData/Local...) instead of the roaming profile (AppData/Roaming...). Does your roaming setup has only AppData/Roaming on SMB, although that's recommended/standard.

Does the browser freeze during this time, or is it just stuck in "connecting"? And are you able to follow these instructions while you see the error to collect a performance profile? You probably need to run Firefox nightly instead of release to get a good profile: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler

Comment 2

[itd]

Attachment: Tracking 20170106 FireFox 52.1.2 ESR

Comment 3

[itd]

Dear Benjamin

> I expect the safe browsing database should be in the local profile (AppData/Local...)

All our users have the safebrowing folder within the normal FireFox Profile.

> Does your roaming setup has only AppData/Roaming on SMB, although that's recommended/standard.

We don't redirect the AppData\Roaming. We are using the FireFox profiles.ini and change there the path to the SMB Networklocation. For example:

"Path=U:\FFTBProfiles\Staff\User\FireFox"

> Does the browser freeze during this time, or is it just stuck in "connecting"?

You can interact with the firefox. For example open a new tab and try to browse to another site. But then the new tab is also stuck in the state "connecting".

I uploaded a profile with FireFox 52.1.2 ESR as our user are using this version. I will generate also Profiles for the "normal" FireFox and the nightly build. But as far as i can see the nightly build has the newer gooogle safebrowsing version.

Best regards

Comment 4

[itd]

Attachment: Tracking 20170106 FireFox 53.0.3

Comment 5

[itd]

Attachment: Tracking 20170106 Nightly 55.0.0.6360

Comment 6

[itd]

I correct myself. Some times Firefox is stuck.

Comment 7

[Benjamin Smedberg]

(In reply to itd from comment #3)
> Dear Benjamin
> 
> > I expect the safe browsing database should be in the local profile (AppData/Local...)
> 
> All our users have the safebrowing folder within the normal FireFox Profile.
> 
> > Does your roaming setup has only AppData/Roaming on SMB, although that's recommended/standard.
> 
> We don't redirect the AppData\Roaming. We are using the FireFox profiles.ini
> and change there the path to the SMB Networklocation. For example:
> 
> "Path=U:\FFTBProfiles\Staff\User\FireFox"

This is the source of your problem. By setting an absolute path, the local and remote profile directories are pointing to the same location. This means that data like the network disk cache, the safe browsing data, and other similar data is being saved to a network drive instead of kept on the local machine, and that is absolutely going to affect performance.

Right now there isn't a way in profiles.ini to explicitly specify the local directory; we only use the relative-path setup to do that. I'd be willing to mentor/review a patch that allows for an optional explicit LocalPath property in profiles.ini.

The code in question would have to go around here: http://searchfox.org/mozilla-central/rev/1a0d9545b9805f50a70de703a3c04fc0d22e3839/toolkit/profile/nsToolkitProfileService.cpp#491

Comment 8

[itd]

Dear Benjamin

I see. Redirecting AppData\Roaming is not an option for us because we don't want to mix Windows 7, Windows 10, Windows Server 2008 R2 and Windows Server 2016 AppData's together only for the case that FireFox (and Thunderbird) is everywhere the same :/

We already save the cache files locally with the variable "browser.cache.disk.parent_directory". Is there something similar for the other folders?

Best regards

Comment 9

[François Marier [:francois]]

(In reply to itd from comment #0)
> FireFox Profile on SMB Share and "Block dangerous and deceptive content"
> activated = described behavior

Are you able to test your setup with Firefox Nightly 56?

We have changed the way that Safe Browsing updates are done (they are now in a separate thread) and so I'm curious to see if that has improved the performance in your setup.

Comment 10

[itd]

Dear François

I tested FireFox Nightly 56 and i have the impression that it is running smoother. 

I can browse and see that in the background the safebrowsing data are under way.

But i don't know how reliable this test was as i have now also a folder called "safebrowsing-failedupdate" within my profile.

Best regards

Comment 11

[Henry Chang [:hchang]]

(In reply to itd from comment #10)
> Dear François
> 
> I tested FireFox Nightly 56 and i have the impression that it is running
> smoother. 
> 
> I can browse and see that in the background the safebrowsing data are under
> way.
> 
> But i don't know how reliable this test was as i have now also a folder
> called "safebrowsing-failedupdate" within my profile.
> 
> Best regards

That means the update is failed :( 

Could you help check if this failure happens permanently?

Comment 12

[François Marier [:francois]]

(In reply to itd from comment #10)
> But i don't know how reliable this test was as i have now also a folder
> called "safebrowsing-failedupdate" within my profile.

Like Henry said, the updates are not working for you.

Given your slow setup, it's possible you need to play with the following settings in "about:config" and increase the timeouts (which are in milliseconds):

  urlclassifier.update.response_timeout_ms
  urlclassifier.update.timeout_ms

If you find values that work, let us know and we may be able to change the defaults so that it works out of the box.

Comment 13

[itd]

Dear François

Sorry for the late reply. Tested with Nightly 56.0a1 (2017-06-22).

Just for fun i set the following values:

user_pref("urlclassifier.update.response_timeout_ms", 150000);
user_pref("urlclassifier.update.timeout_ms", 600000);

I still have immediately a safebrowsing-failedupdate folder.

Current folders with files:
SAFEBROWSING
│   test-block-simple.pset
│   test-block-simple.sbstore
│   test-malware-simple.pset
│   test-malware-simple.sbstore
│   test-phish-simple.pset
│   test-phish-simple.sbstore
│   test-track-simple.pset
│   test-track-simple.sbstore
│   test-trackwhite-simple.pset
│   test-trackwhite-simple.sbstore
│   test-unwanted-simple.pset
│   test-unwanted-simple.sbstore
│
└───google4
        goog-badbinurl-proto.metadata
        goog-badbinurl-proto.pset
        goog-downloadwhite-proto.metadata
        goog-downloadwhite-proto.pset
        goog-malware-proto.metadata
        goog-malware-proto.pset
        goog-phish-proto.metadata
        goog-phish-proto.pset
        goog-unwanted-proto.metadata
        goog-unwanted-proto.pset

--------------------------------------------------------------
SAFEBROWSING-FAILEDUPDATE
│   tableupdates.bin
│   test-block-simple.pset
│   test-block-simple.sbstore
│   test-malware-simple.pset
│   test-malware-simple.sbstore
│   test-phish-simple.pset
│   test-phish-simple.sbstore
│   test-track-simple.pset
│   test-track-simple.sbstore
│   test-trackwhite-simple.pset
│   test-trackwhite-simple.sbstore
│   test-unwanted-simple.pset
│   test-unwanted-simple.sbstore
│
└───google4
        goog-badbinurl-proto.metadata
        goog-badbinurl-proto.pset
        goog-downloadwhite-proto.metadata
        goog-downloadwhite-proto.pset
        goog-malware-proto.metadata
        goog-malware-proto.pset
        goog-phish-proto.metadata
        goog-phish-proto.pset
        goog-unwanted-proto.metadata
        goog-unwanted-proto.pset
--------------------------------------------

Best regards