919485 - modest CPU usage while checking add-ons behind not configured proxy

Status: RESOLVED DUPLICATE of bug 683651

(MailNews Core :: Networking, defect)

Description

[Dmitry Veltishchev]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0 (Beta/Release)
Build ID: 20130803215302

Steps to reproduce:

I have internet connection via proxy server, but our mail server is available without proxy. So I have 'No proxy' option in Preferences - Advanced - Network & Disk space - Connection settings.  Mail processing works fine, but when I open Tools - Add-ons menu  and try to search for addons or extensions, try to check for updates for installed extension, Thunderbird falls into infinite search process and it never ends.

strace shows repeating socket operations: 
....
read(4, 0x7fffa82a1690, 16)             = -1 EAGAIN (Resource temporarily unavailable)
futex(0x7fc0a8173c8c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fc0a8173c88, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\313\36\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\363\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\310\36\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\364\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\305\36\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\365\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\373 \0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\366\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0n \0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\367\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\0I\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\370\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\302\36\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\371\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
writev(6, [{"\24\0\6\0\277\36\0\0042\1\0\0\6\0\0\0\0\0\0\0\4\0\0\0", 24}, {NULL, 0}, {"", 0}], 3) = 24
poll([{fd=6, events=POLLIN}], 1, -1)    = 1 ([{fd=6, revents=POLLIN}])
recvfrom(6, "\1\0\372\235\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 4096, 0, NULL, NULL) = 32
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(6, 0x7fc0bbae1074, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x7fc0a8173c8c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fc0a8173c88, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
poll([{fd=6, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=6, revents=POLLOUT}])
...

And CPU is loaded by 15-20 % on rather fast i5 multi-core machine with 8Gig of RAM.

I spent a while before I discovered 'the root of all evil'. I configured proxy in TB settings, and specified exception for my corporate mail server. And - hoorray - all became fine! The main problem is this lack of connectivity is NOT detected in any way. I suppose Thunderbird should INFORM user about connection failure (for, e.g. 10-15 seconds) to force him/her checking settings, etc, but not just eat CPU and in fact do nothing.





Actual results:

After pressing 'Get Add-ons' button on Addons manager, Thunderbird display 'Loading...' message and starts endless connection attempts, eating some processor time.


Expected results:

Thunderbird should detect lack of internet connectivity, stop trying and inform user about this issue.

Comment 1

[David von Oheimb]

I can confirm this bug for Thunderbird 24.3.0.
When using Thunderbird behind a firewall that requires an HTTP proxy and the proxy is not properly configured or not responding, there is a high CPU load until the connection times out. Obviously there is some busy loop waiting for the connection.

Comment 2

[Wayne Mery (:wsmwk)]

see this in version 31?
http://www.mozilla.org/en-US/thunderbird/all-beta.html

Comment 3

[Dmitry Veltishchev]

I've tried thunderbird 31.0 beta3 (64-bit version) on Ubuntu 14.04.
Looks like it is OK: CPU consumption is much less than before, but still about 10-20 percent.

Note that after rather long "hanging" (1 minute or even more) TB does not report any errors about connection timeout. I see only panel "What are Add-ons?" without any notice about connection problems.

I have no idea why poll() timeout could not be increased to some seconds, not milliseconds. And user should be noticed about connection problems after... hmmm... 10 seconds, I guess - not more, with button "retry again", for example. Long awaiting just raises thoughts like "aarrgh, why it's so stupid and slow!"

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to David von Oheimb from comment #1)
> I can confirm this bug for Thunderbird 24.3.0.
> When using Thunderbird behind a firewall that requires an HTTP proxy and the
> proxy is not properly configured or not responding, there is a high CPU load
> until the connection times out. Obviously there is some busy loop waiting
> for the connection.

did you see this in version 17?

Comment 5

[David von Oheimb]

Wayne, it is straightforward to reproduce this issue.
Simply put any garbage IP address and port number in the HTTP proxy settings,
then for instance search for add-ons via Tools->Add-Ons->"Search all add-ons".

On current TB 24.6.0 on my mac, idle CPU load is 5%, which in itself is already too much.
After pressing the search button, the CPU load doubles to continuous 10% on a quad-core. 
This is not acceptable, in particular for machines running on battery.

Similar busy loops occur in other contexts as well. Also with Firefox.

Comment 6

[David von Oheimb]

Note that this bug was reported already in 2007 - see bug 378525.
Yet that bug was - for wrong reasons - closed by Wayne in 2009.

Comment 7

[David von Oheimb]

A further bug that should have been re-opened is TB bug 683651.

See also ancient FF bug 482642, which got wrongly closed as well.

Comment 8

[Dmitry Veltishchev]

(In reply to Wayne Mery (:wsmwk) from comment #4)
> (In reply to David von Oheimb from comment #1)
> > I can confirm this bug for Thunderbird 24.3.0.
> > When using Thunderbird behind a firewall that requires an HTTP proxy and the
> > proxy is not properly configured or not responding, there is a high CPU load
> > until the connection times out. Obviously there is some busy loop waiting
> > for the connection.
> 
> did you see this in version 17?

I have not used TB v17 via proxy, so I don't know how it worked before.

Comment 9

[David von Oheimb]

As I wrote before, it is straightforward (for anyone on any version!) to reproduce this bug:
Simply put any garbage IP address and port number in the HTTP proxy settings,
then for instance search for add-ons via Tools->Add-Ons.

This phenomenon is still present in TB 31.2.0, and likewise in FF 33.1.
I wonder why this 7+ years old bug is still marked as UNCONFIRMED.

Moreover, those ugly battery-wasting bugs due to busy loop(s) on connect should hot be too hard to correct. My guess is that all of them can be traced down to a single careless chunk of low-level code.

Comment 10

[Dmitry Veltishchev]

(In reply to David von Oheimb from comment #9)
> This phenomenon is still present in TB 31.2.0, and likewise in FF 33.1.
> I wonder why this 7+ years old bug is still marked as UNCONFIRMED.

Maybe just add more votes for this issue... and pray that They will see it? :)

Comment 11

[David von Oheimb]

Thanks vdm for marking this high priority, which should help a lot getting attention. 

I am pretty sure that the issue is fundamental one, since it is observable also for Firefox (and likely any other Mozilla product; BTW I just opened a new report for FF: bug 1107251) for all sorts of HTTP connections. Likely it is even independent of proxy use.
Thus solving it will be a huge gain for many (if not most) users.

Comment 12

[Wayne Mery (:wsmwk)]

(priority field is for deveopers to set)

data and focus are required, not so much priority settings and lots of comments. The reason I asked about version 17 in comment 4 is because there was a regression in that time frame, and if someone had investigated then we'd have more data.  In addition, the older bug reports my be related, or not related depending on whether you had tested them and proved them to be reproducible with the versions on which they were reported.

I tested a daily build on windows with poorly configured proxy settings. I see modest CPU usage (10-20%) which for every use case (start page, addons, imap, news) eventually timed out and CPU returned to near zero.  This level of CPU usage matches what is stated in comment 0, at least as how I read it.

If you have a use case which is far worse it may be a different issue, but please provide a profile. 
https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Thunderbird_Performance_Problem_with_G

Also, if you provide a profile in bug 1107251 we can see how that plays out.

Comment 13

[Wayne Mery (:wsmwk)]

Attachment: TB bug 919485 proxy settings.png

settings I used

Comment 14

[David von Oheimb]

I can confirm the cpu usage figures (15-20%) given by Wayne and in the original report on my Macbook Air on the latest TB and FF release versions. (When FF tries loading several pages simultaneously, the figures increase only marginally.)

Unfortunately I cannot afford the time to install the trunk version and use profiling tools, and I do not have TB version 17 at hand to test with that one. Yet any developer interested could do for himself easily.

Again, why is this bug still marked UNCONFIRMED?

Comment 15

[David von Oheimb]

While awaiting response for the question in my last two posts above,
here is some more info regarding platforms.

On Windows I was not able to reproduce the issue by checking for add-on updates and searching for add-ons (as I reported for Mac OS), but when viewing an email that tries loading external contents via http(s).
Also here, missing or flaky network connections cause needlessly high CPU load.

Comment 17

[David von Oheimb]

Here are two profiles witnessing the effect when loading pictures from HTML emails on Windows via a flaky connection (simulated by misconfiguring the http(s) proxy settings):
http://people.mozilla.org/~bgirard/cleopatra/?1457532425822#report=4337b4bf7dd34b88aa7b8283378cfa4d12c8e4e5
http://people.mozilla.org/~bgirard/cleopatra/?1457532688572#report=e1acfd17e9b13ca4c28ce527f6647839cc4b0b4f

Comment 18

[David von Oheimb]

This bug should not anymore marked as UNCONFIRMED.

The issue appears the same as in bug 1107251 (for FF) and could be related to bug 683651 (for TB).

Comment 19

[Wayne Mery (:wsmwk)]

(In reply to David von Oheimb from comment #17)
> Here are two profiles witnessing the effect when loading pictures from HTML
> emails on Windows via a flaky connection (simulated by misconfiguring the
> http(s) proxy settings):
> http://people.mozilla.org/~bgirard/cleopatra/
> ?1457532425822#report=4337b4bf7dd34b88aa7b8283378cfa4d12c8e4e5
> http://people.mozilla.org/~bgirard/cleopatra/
> ?1457532688572#report=e1acfd17e9b13ca4c28ce527f6647839cc4b0b4f

These are 
https://cleopatra.io/#report=4337b4bf7dd34b88aa7b8283378cfa4d12c8e4e5
https://cleopatra.io/#report=e1acfd17e9b13ca4c28ce527f6647839cc4b0b4f