Open Bug 1835223 Opened 2 years ago Updated 1 years ago

Update search (happens multiple times a day) leads to freeze of the browser

Categories

(Core :: Networking, defect, P2)

Firefox 113
defect

Tracking

()

UNCONFIRMED

People

(Reporter: b.loeffler, Unassigned)

References

Details

(Whiteboard: [necko-triaged])

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/113.0

Steps to reproduce:

Firefox installation wit the default settings.
The problem seems to occur randomly multiple times a day without a specific period.

Actual results:

When firefox searches for updates, then the browser is not usable, its "freezes". New websites cant be opened and currently opened websites cant be loaded again. It is like this for a couple of minutes 0-3min. So we all just wait, and wait and wait....
Navigation to the firefox settings is possible.
This is fine, if it just occurs once a day or once a week.
BUT: This happens multiple times a day, sometimes even multiple time an hour.
There seems not to be a systematic behind it. I already checked about:config and there the update related settings, but the timings which are specified seems to be find for a daily check. So the configuration is fine I think (also I did a fresh reinstallation and did never change anything in about:config).

Expected results:

Check of updates according to about:config
Possibly no freeze of the whole browser while firefox searches for updates. If updates are installed, then of course its fine, that firefox is not usable. But while the search happens, firefox should be usable.

The Bugbug bot thinks this bug should belong to the 'Toolkit::Application Update' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Application Update
Product: Firefox → Toolkit

Hello, thank you for filing a bug.

Could you run the profiler before checking for an update and then attach the resulting profile to this bug please?

Flags: needinfo?(b.loeffler)
Flags: needinfo?(b.loeffler)

I uploaded the profiler file here in the ticket and here is also the permalink: https://share.firefox.dev/42g9cU3

And another profiler file, where the issue also occurred: https://share.firefox.dev/43gKkgl

Hmm, I'm not really seeing any update code in here. I also don't have a ton of experience reading profiles. I'm going to send this over to the Performance team and see if they would be kind enough to help diagnose what's going on here.

Component: Application Update → Performance
Product: Toolkit → Core

This bug was moved into the Performance component.

:b.loeffler, could you make sure the following information is on this bug?

  • âś… For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
  • For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
  • Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm it is recent.

Thank you.

Flags: needinfo?(b.loeffler)
Attached file memory-report.json.gz —
Flags: needinfo?(b.loeffler)
Attached file RawData.txt —

I see you have "Sophos Intercept X" installed.
Sophos seems have caused performance issues to some of its users. Could you perhaps try to disable it?
(when disable, I'd use still Windows Defender, or whatever the builtin anti-virus tool is called).

If that doesn't help, could you create a new performance profile, but enable first all the threads in the profiler's settings
"Bypass selections above and record all registered threads".

Flags: needinfo?(b.loeffler)

Hello,
I can't disable Sophos, because its our company anti virus....
And it worked the past 7 years so I don't know if this is the problem.

I attached a new profiler log here: https://share.firefox.dev/461hl1V

Flags: needinfo?(b.loeffler)

Hmm, something seems to be blocking the network connections. I still wonder if it is the anti-virus software.
At least historically they've affected rather negatively to network speed.

I understand you don't want to share a profile publicly, but if you can send me one with a bit more information, that might reveal something.
The profile doesn't show high cpu usage except once the network responses come in and the content processes can show the pages.

Moving this to Core: Networking, since that is what the issue seems to be about, something blocking that network connections don't get through.

Component: Performance → Networking

Hello, what additional information do you need in the profiler? Can you tell me which settings I need to activate?

There's definitely something happening there that's holding off network requests for 40s.
From what I can tell there's also a PAC script involved?

A full profiler log would probably be helpful.
Also some logs, either regular or captured via the profiler. See this link here:
https://firefox-source-docs.mozilla.org/networking/http/logging.html

Thanks!

Flags: needinfo?(b.loeffler)

Hello,
so what is a full profiler log? Can you please tell me this? I would be really happy to be able to give you a full log, but I dont know what you mean by "full". So please tell me, what profiler settings you need.

What is a "PAC script"? I'm not sure if I can tell you if this is involved.

Sorry for the inconvenience, but I'm not a profiler or logger expert... Just a regular embedded software developer, so please be patient with some "dump" questions.

Flags: needinfo?(b.loeffler)

PAC is proxy autoconfig. That means either Firefox is set to autodetect proxy settings, or your OS settings are set to do that. That means for every network request, a script is run in order to determine if the request should go through the proxy or not. If the script is malfunctioning for some reason, that could block all of your network requests.

By full profiler log we mean a log without the URLs anonymized - it would make it a bunch easier to figure out what requests are failing and why.
Regardless, if you are able to capture such logs/profile, you might want to send them to necko@mozilla.com as they might contain sensitive info.

When trying to reproduce it might help if you disable all extensions before, so we make sure the problem isn't extension specific.
https://support.mozilla.org/en-US/kb/troubleshoot-extensions-themes-to-fix-problems#w_start-firefox-in-troubleshoot-mode

Flags: needinfo?(b.loeffler)

Hello,
I provided a profiler report in my onedrive. (Since its to big for a mail attachment)
The extensions were not disabled in the profiler log. But if I capture one, I will provide it also to you.
Here is the onedrive link, you should have access to it with the mail account necko@mozilla.com:
https://envea-my.sharepoint.com/:f:/g/personal/b_loeffler_envea_global/EiIgnI6pXLRPgIHFraYb-w8BJJUyqzbsNoGQRd3pWcR53w?email=necko%40mozilla.com&e=YhI2qv

Please tell me, if you can access the file and when I can delete it again.

Flags: needinfo?(b.loeffler)

I've added the profile to our temporary internal storage, you can pull this down.
I took a quick look and it looks like a hang in the socket thread.
Some http logs might be helpful here, again you'll probably want to send to necko@mozilla.com.

Thanks!

Flags: needinfo?(b.loeffler)

I send a log to the mail address.

Flags: needinfo?(b.loeffler)

Hi,

Thank you for sharing the log file.
I checked the HTTPHeaderLive.txt log file that you shared. Unfortunately, this is not the log file we are looking for.
This file contains mainly the header information.

Could you kindly share the http logs based on the instructions mentioned here:
https://firefox-source-docs.mozilla.org/networking/http/logging.html

Please let us know if you have any problems in capturing the logs

Flags: needinfo?(b.loeffler)

Hello, so now i'm confused. Which part of the howto side do I need to do for you? I already provided a log file according to the chapter "Using about:logging"
So what do I need to do? Please, instead of just pasting a link tell me which chapter you want me to do. Do you need the part of the chapter "Logging HTTP activity by manually setting environment variables" or something else?
Please be more clear. I'm really happy to help you out but I need a precise information what to to of the big "howto".

Flags: needinfo?(b.loeffler)

Ok, I just saw, that its not exactly the same as the profiler. Sorry.
So you want me to log a logfile via "about:logging" I think. I will provide this. If this is not the part you need, please inform me.

The severity field is not set for this bug.
:jesup, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(rjesup)

Hello,
I could catch a logging of the update search occurring. I uploaded it directly in the profiler: https://share.firefox.dev/3rieTUU

(In reply to b.loeffler from comment #24)

Hello,
I could catch a logging of the update search occurring. I uploaded it directly in the profiler: https://share.firefox.dev/3rieTUU

I saw this line from the above profiler.

2023-07-04 12:17:20.698 UTC - [Parent Process 10468 Socket Thread] D/nsSocketTransport     ...returned after 81958 milliseconds

It seems the first poll was blocked for a long time for some reason. As comment #12 said, I assume this is caused by an antivirus software.
Is it possible to disable the antivirus software and see if you can reproduce this?

Flags: needinfo?(b.loeffler)

Hello,
mh, I think this is a problem, since the bug occurs not all the time I would have to disable the antivirus for example the whole day while browsing. My IT-Department will not like this (also I'm myself are not able/allowed do deactivate the antivirus software).

Generally it is no problem for me, if the update check freezes the browser. But only if this happens once a day. My big problem though is, that this check can occur multiple time a day or even multiple times an hour.
Why is that? Shouldn't firefox only check once a day? It is configurable and I already tried to change the about:config but firefox keeps checking very often for updates.
So maybe we can do something about that? If There is only one check a day i do not care if the brower freezes for half a minute. So is this an option?

Flags: needinfo?(b.loeffler)

(In reply to b.loeffler from comment #26)

Hello,
mh, I think this is a problem, since the bug occurs not all the time I would have to disable the antivirus for example the whole day while browsing. My IT-Department will not like this (also I'm myself are not able/allowed do deactivate the antivirus software).

Can you confirm that this is caused by the antivirus software?

Generally it is no problem for me, if the update check freezes the browser. But only if this happens once a day. My big problem though is, that this check can occur multiple time a day or even multiple times an hour.
Why is that? Shouldn't firefox only check once a day? It is configurable and I already tried to change the about:config but firefox keeps checking very often for updates.
So maybe we can do something about that? If There is only one check a day i do not care if the brower freezes for half a minute. So is this an option?

Not sure what's the recommended way for this.
Robin, do you know if there is a way to make Firefox check updates less frequently?

Flags: needinfo?(bytesized)

NI for the question about the antivirus software.

Flags: needinfo?(b.loeffler)

No I can not confirm that this is caused by the antivirus. As I mentioned, I cant disable the antivirus for security reasons. My IT department will NOT allow this. So no, I cant confirm this.

Flags: needinfo?(b.loeffler)

One Idea:
If the anitivirus software would be the problem. What would be the workaround?
Maybe, rather than deactivating the antivirus software I could just test the workaround and see if the bug disappears. This way I can check it but do not need to disable the antivirus software.

(In reply to b.loeffler from comment #30)

One Idea:
If the anitivirus software would be the problem. What would be the workaround?
Maybe, rather than deactivating the antivirus software I could just test the workaround and see if the bug disappears. This way I can check it but do not need to disable the antivirus software.

Maybe use a VPN or setup a proxy for Firefox?
Or try to block the origin of Firefox update (aus5.mozilla.org) locally by adding 127.0.0.1 aus5.mozilla.org to your /etc/hosts file, but I don't recommend this way.

Which antivirus are you using? (By the way, thanks for all the work helping track this down!)

Severity: -- → S3
Flags: needinfo?(rjesup) → needinfo?(b.loeffler)
Priority: -- → P2

(In reply to Kershaw Chang [:kershaw] from comment #27)

Robin, do you know if there is a way to make Firefox check updates less frequently?

I'm not excited about addressing this issue this way, but the pref that controls update check frequency is app.update.interval. Changing that pref doesn't change the update timer immediately though. The implementation is a little complicated, but I think that changing it will take effect after the next update check.

The value of app.update.interval is measured in seconds, and it is capped at 24 hours (86400 seconds). Attempting to set it higher than that will result in the value of 24 hours being used.

Flags: needinfo?(bytesized)

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #32)

Which antivirus are you using? (By the way, thanks for all the work helping track this down!)
@Randell Jesup we use "Sophos". And your welcome, I'm happy if I can help.

(In reply to Robin Steuber (they/them) [:bytesized] from comment #33)

The value of app.update.interval is measured in seconds, and it is capped at 24 hours (86400 seconds). Attempting to set it higher than that will result in the value of 24 hours being used.
I set the value now to 86400 seconds. Fingers crossed that the check will only happen once a day.

I don't know if this helps, but here are some other people reporting this problem (I also left a comment there):
https://connect.mozilla.org/t5/discussions/stop-firefox-constantly-checking-for-updates/td-p/29616

Flags: needinfo?(b.loeffler)

Please see comment #31 amd #comment#33.
Does using VPN or extend app.update.interval helps?
Thanks.

Flags: needinfo?(b.loeffler)

One Update regarding this issue:

  • I did change the app.update.interval value 86400 seconds. But unfortunately a update search is still done multiple times a day... So the issue persists :(

  • One thing I noticed: If I open an other browser (e.G. the Edge) while the update search is active, then upon opening the other brower the update search stops right away.

  • Regarding comment #31:

(In reply to b.loeffler from comment #30)
Maybe use a VPN or setup a proxy for Firefox?
Or try to block the origin of Firefox update (aus5.mozilla.org) locally by adding 127.0.0.1 aus5.mozilla.org to your /etc/hosts file, but I don't recommend this way.
I'm not an IT expert and I'm not the admin of our company system, so a proxy (don't know how I would do this) or a VPN (I do not want to use a VPN the whole time) is not an option.
Regarding the blocking: this solution is maybe not an option for me, since you say you DON'T recommend to block the origin locally in the /etc/hosts file.

Did you have a look into the other reported issues from other users yet?
https://connect.mozilla.org/t5/discussions/stop-firefox-constantly-checking-for-updates/td-p/29616

Flags: needinfo?(b.loeffler)

I think it is safe for us to assume that the browser freeze is caused by the anti-virus software. As already pointed out there are few more users experiencing this issue.
As this issue is directly related to the update search, the problem could be mitigated by reducing the frequency of the update search by Firefox.
As the reporter has already tried to reduce the frequency of search there seems to be some problem in that as well.
I would like to address that problem first.
@Robin: Do you know why the pref is not working for the reporter? Are there more prefs which the reporter could use to reduce the frequency of search?

Flags: needinfo?(bytesized)

(In reply to Sunil Mayya from comment #37)

@Robin: Do you know why the pref is not working for the reporter?

No. If they are willing to help debug this issue, a separate bug should be filed where that can happen.

Are there more prefs which the reporter could use to reduce the frequency of search?

I'm afraid not.

Flags: needinfo?(bytesized)

@robin - could it be related to the offline/online behavior around updates? See toolkit/mozapps/update/UpdateService.sys.mjs : _registerOnlineObserver()

Flags: needinfo?(bytesized)

I don't think so. I believe that function is only invoked after an update check has already begun. It's meant for resuming an update, not initiating one.

Flags: needinfo?(bytesized)

I'm going to see if we have contacts with Sophos.

Flags: needinfo?(rjesup)
Whiteboard: [necko-triaged]

Chloe - could you take a look at this report? The user is required to use Sophos, and it appears this is triggering a very long delay when Firefox update checks happen. Thanks!

Flags: needinfo?(rjesup) → needinfo?(chloe.bell)

Hi all,

From the discussion so far it looks like Sophos could be a factor here but as you cannot test without the AV installed on that machine then we can't say for sure.

I am running Firefox on my machine with Sophos AV and have not seen any issues so there may be some further configuration of the environment that I do not have in order to reproduce this.

In order to investigate further we would need more detailed logs and information on your environment. The best way to progress this would be to raise a ticket with Sophos support (you may need to do this with your IT department).

Thanks,

Chloe

Flags: needinfo?(chloe.bell)

Please see comment 43. Thanks!

Flags: needinfo?(b.loeffler)

So what can I do to help you more? Or dos the last commend mean that you cant help any more or cant fix the problem?
Or can't you help me with the problem and I need to contact Sophos?
I would then need some instruction or information I can tell them what exactly the problem is.

The problem is getting worse on my side. Today Firefox constantly checked for updates... there was not even half an hour without problems. The configuration to let firefox only check in a specific period one a day for example does not work.

Please at least help me to fix this. Could you just clone this ticket with all the information here for a new ticket to fix the update-period problem?

Sorry for all this work, but I too get tired of this problem :(
I practically cant work any more with Firefox since a lot of our daily work happens in cloud based systems...

Flags: needinfo?(b.loeffler)

NeedInfo to Chloe for comment 45.

@b.loeffler I think she needs you to open a ticket with Sophos (which may mean asking your IT department to do so). I.e. work through their tech support team (probably linking her in with them).

Flags: needinfo?(chloe.bell)

Hi,

In order to investigate the impact Sophos may be having here then you need to open a ticket with Sophos support (via your IT department). Then we can gather the relevant information in order to understand the issue better. I did try to reproduce it but can't so we need to understand your environment better and this is best done by raising a support ticket with Sophos.

Thanks

Flags: needinfo?(chloe.bell)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: