504990 - Firefox is not GREEN (excessive useless polling)

Status: RESOLVED DUPLICATE of bug 458592

(Firefox :: General, defect)

Description

[Robert Bradbury]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12pre) Gecko/2009071319 Firefox/3.0.12pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12pre) Gecko/2009071319 Firefox/3.0.12pre

Even when Firefox is largely idle, it continues to make excessive system calls (gettimeofday, read, poll, writev).  It makes them at a rate which will continually consume 13-20% of the CPU time on a machine preventing the machine from entering into and maintaining low power consumption states.

Linux now has "ondemand" CPU frequency capabilities which allow it to automatically drop CPU clock rates and or CPU voltages into power-saving modes and ramp them back up "ondemand" (without user intervention).  If tuned properly this enables the maximum power conservation on the machine.  With more than a billion PC's in the world and the Firefox market share increasing a significant concern for Firefox developers should be to make it as "GREEN" as possible (making this a distinguishing feature relative other browsers).

Reproducible: Always

Steps to Reproduce:
1. Startup a firefox session (in this situation restart an old, large session).
2. Wait until startup is complete.
3. Monitor CPU consumption with ps and/or monitor system calls with strace.
Actual Results:  
Firefox continues to eat CPU, a minimum of 10-20% of the available CPU even when it is doing nothing.  System call monitoring reveals a continual stream of poll() / gettimeofday() / read() calls which appear to do nothing.

Expected Results:  
When Firefox is "idle" it should *really* be idle.  If none of the firefox windows have been touched in the last 5, 10, 60 minutes then it should lengthen any timeouts to take into account that the process really is "idle".  Few users are going to close their browser every time they go to the bathroom or lunch or a meeting (esp. given the time Firefox takes to resume large sessions).  Firefox should adjust its behavior to account for large "non-use" periods and resume "normal" functioning when the user actually needs it.

These tests were run on a Firefox version 3.0.13pre (recent CVS) with Javascript disabled and *no* extensions.  So it isn't a function of either abusive javacripts or NoScript/Adblock, etc.  The CPU use problem gets worse when Javascript is enabled and/or "common" extensions are active.

Session, strace and CPU use documentation will be attached.

Sidenote: I have noticed excessive updates to .sqlite files when nothing appears to be going on in Firefox, this may be part of the problem here which may need to be checked.

Comment 1

[Robert Bradbury]

Attachment: Large session (90 windows / 986 tabs) causing polling problem.

Though this is a large session, I do not think the session size is the primary source of the problem.  Because there are no extensions and no javascript for the session the only files being accessed (polled) should be those required for a "basic" firefox session (prefs.js, .sqlite files, CACHE files, X server, etc.)

Comment 2

[Robert Bradbury]

Attachment: Strace of useless system calls

This is a partial strace of system calls being executed by Firefox on previously attached session.  It shows the repetitive nature of the calls.  While this is only 32 calls, a 10 second strace of the process makes *27,000* such calls.  That can hardly be called placing a minimal load on the CPU.

Comment 3

[Damian Shaw [Quan]]

You might want to take a look at bug 428565 and it's dependencies.

Comment 4

[Robert Bradbury]

Attachment: Documentation of CPU usage

Example of documentation of CPU usage by the process.  The lock file date is taken as the Firefox start time, the current ps on the process indicates that over a 26+ hour period it consumed roughly 3 and a half hours of CPU time.  Almost all of this time the windows were minimized and the session was "idle".  Session "resume" time took ~20 real time minutes, during which time ~50-60% of the CPU was in use, so one could subtract off 10-15 minutes of CPU time from the total CPU time as being due to resume requirements.

One can assume that this is not a "Linux" only problem unless the polling process under Windows or other OS is fundamentally different.  The only difference may be that Windows may lack "ondemand" scheduling which will throttle back the CPU frequency (Pentium IV Prescott's can set the CPU frequency over approx. an order of magnitude range).  Laptop CPUs can significantly scale back the voltage.  These reduce the CPU heat production (lengthing chip lifetime hopefully) and decrease CO2 emissions associated with electricity production & consumption.  A reduction from 150W to 100W (desktop machine) isn't much.  But when multiplied by hundreds of millions of machines that are "on" but doing nothing a significant fraction of the wall clock time the wasteful power consumption is significant!

And yes, my machine is on 24/7 because it hosts a low use web server and Firefox is left running for days to weeks (largely because of the extensive delays restarting the large Firefox sessions I usually run).  The machine could easily throttle back the clock (and reduce energy consumption) for significant parts of the day.

Comment 5

[Jo Hermans]

You might set browser.sessionstore.enabled to false (in about:config), that will make sure that the sessionstore.js file isn't stored every 10 seconds. But then you loose the ability to restore to restore your windows & tabs when you crash.

Comment 6

[Jo Hermans]

bug 477850 will help a bit here

Comment 7

[Wayne Mery (:wsmwk)]

I may be missing something, but I don't see how this is substantially different from bug 458592.  Dupe?

Comment 8

[Tyler Downer [He/Him]]

Reporter, please retest with Firefox 3.6.12 or later in a fresh profile (http://support.mozilla.com/kb/Managing+profiles). Also update your plugins (flash, adobe reader, java, quicktime, silverlight, etc.) Go to the developer's website and download the latest version from there. If you no longer see this issue, please close this bug as RESOLVED, WORKSFORME. If you do see the bug, please post a comment.

Comment 10

[Robert Bradbury]

As noted in Bug #458592 the bug still exists in Firefox 3.6.12.

This bug (Firefox is not GREEN) is not strictly the same as Bug #458592 involving excess CPU consumption.

Break it down int 3 problems:
1) Excessive CPU consumption and/or polling and/or gettimeofday calls by FLASH plugin.  Solution: put flash in a separate process which can be Stopped (kill -s STOP FLASH-PID-#) [which the firefox process *could* do when all the "on top" window(s) do not have running flash components(!)[1] -- or the user could do with external scripts when it is noticed that FLASH is consuming CPU wastefully].

2) Excessive CPU consumption because Firefox has too many threads which require attention (perhaps due to JS, perhaps due to thread swapping, perhaps due to file handle mismanagement -- really why is the firefox session cited in Bug #458592 (Attachment #491828 [details]) polling *14* file descriptors?!?).

3) Excessive CPU consumption because Firefox fails to take into account and adjust its state to reflect an idle windows or even complete sessions (user leaves browser running overnight, user switches to a different virtual terminal, user goes to lunch, user goes for a cup of coffee, etc.).

Problem #3 is the case of the browser being non-GREEN (i.e. this bug).  Making the browser GREEN so it detects states where it should enter a minimal CPU use state (hibernate/suspend) is different from those where the architecture is flawed (#1) or poorly programmed (#2).

So I would request that this bug be reopened and confirmed ( as not being a duplicate of Bug #458592 as very different solutions are required.

The way to confirm this bug is simply to leave a moderately complex (20-40 tabs - no javascript, no flash) open overnight (i.e. idle) or unused for several days and note the accumulation of CPU time.

1. Note that if the only flash I am using is gmail, [all other flash blocked by Flashblock or Noscript] then when I log out of gmail firefox should force FLASH into a "stopped" state or completely unload the plugin.