403566 - Digg.com never stops loading

Status: VERIFIED FIXED

(Tech Evangelism Graveyard :: English US, defect)

Description

[thekurst]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b2pre) Gecko/2007111004 Minefield/3.0b2pre
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b2pre) Gecko/2007111004 Minefield/3.0b2pre

When going to Digg.com even after all content has loaded, the spinning activity indicator never stops and the status bar indicates that data is still being transferred. This does not happen in Camino or Safari

Reproducible: Always

Steps to Reproduce:
1. load http://www.digg.com/news/technology
2. watch page indicate it's loading
3. page never stops loading
Actual Results:  
Page activity indicator constantly spins. 

Expected Results:  
Web page should load all content and finish.

Comment 1

[Justin Dolske [:Dolske]]

Yeah, I've noticed this too recently.

If I let it load for a couple of minutes (to allow other connections to idle out) and then check "netstat -a", I see:

Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
[...]
tcp4       0      0  h-179.office.moz.58877 ns-vip3.hitbox.c.http  ESTABLISHED

A Wireshark sniff of my traffic shows that if I click "Stop" at this point, the connection gets terminated. Looking at the traffic for that stream I see:

GET /HG?hc=&[.......long query string........] HTTP/1.1
Host: y.digg.com
Keep-Alive: 300
Connection: keep-alive

HTTP/1.0 200 OK
Server: Hitbox Gateway 9.3.2
nnCoection: close
Content-Type: image/gif
Content-Length: 43

[...43 bytes of GIF data...]

Note that the server reply is HTTP/1.1, and has a mangled "nnCoection: close" header. I'd guess this box had some recent update applied, and has a byte-ordering issue when generating headers. With HTTP/1.1, persistent connections are the default, and since the server didn't request closing the connection (due to the mangled header) it gets stuck.

Comment 2

[Justin Dolske [:Dolske]]

Hmm, well, there's another problem here.

The above would explain why Digg doesn't finish loading quickly. Except that if you wait ~5 minutes for the connection to time out, the throbber keeps going. :-/ I verified with netstat and lsof that Firefox no longer has the connection established, so somewhere we're failing to catch the close/timeout and stop the throbber.

Comment 3

[Ria Klaassen (not reading all bugmail)]

Regression range for this is http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&date=explicit&mindate=2006-04-06+08%3A00&maxdate=2006-04-06+14%3A00
2006-04-06 09 works and 2006-04-06 13 fails. I rechecked it a couple of times and this must be it.

Comment 4

[Justin Dolske [:Dolske]]

Hmm... Nothing obvious in that range, but I do see:

"Bump version on trunk to 3.0a1. b=330528 r=mconnor"

Could be that the Digg problem is only triggered by Minefield's useragent string.

Comment 5

[Brian Polidoro]

It looks like it. When I changed to the FF 2.0.0.9 UA string the loading finished.  

I found the problem stopped when I added *ads1.msn.com* as an adblock plus filter.  

Comment 6

[Dave Miller [:justdave]]

FYI, the mangled Connection header is on purpose, it's a common trick used by caching proxy servers to obfuscate the header it gets back from the back end system so that it won't be valid when sent to the client.  It's cheaper to swap bytes in the header name to make it invalid than it is to remove it when you're trying to move lots of data as quick as possible.

Comment 8

[Alexander Smith]

I'm having the same problem in 3.0b3, both before and after adding *ads1.msn.com* to my Adblock filter list. User agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3) Gecko/2008020514 Firefox/3.0b3

Add-ons loaded:
PDF Download
DownThemAll!
DownloadHelper
Adblock Plus

All add-ons are current.

Comment 10

[chris hofmann]

anyone have any contacts at digg?

Comment 11

[Manish Singh]

Attachment: Small testcase that reproduces the problem

This is a small testcase that trips up the same thing that the http://ads1.msn.com/library/dap.js script does. It opens up the contentDocument on an iframe, and never closes it. If I've read the code correctly, it's because opening an HTML document creates a wyciwyg channel, which is added to the load group of the page (hence the throbber). Since it's never closed, and it's a channel not actually backed by a network request, it spins forever.

The code in the dap.js script actually has a setTimeout that will call .close() on the iframe document, but for whatever reason, the script explicitly disables that for Firefox 3. Yes, it actually looks at the UA, detects Firefox, figures out the major version, and decides to only enable the logic to make digg finish loading if it's Firefox 1.5 or 2.0.

It seems sort of bad how trivial it is to break the document load status logic, but I'm not really sure how to fix it.

Comment 12

[Youri Volkov]

Hi as a temporary fix, because I was getting aggravated I did:

0.0.0.0 y.digg.com in /etc/hosts

So far I haven't seen any loss in functionality on the website

Comment 13

[Chris Lawson (gone)]

Manish, can you file a new bug on that specific behaviour, please?

This one is more about digg explicitly sniffing for user-agent names, although what you've described certainly sounds like something that ought to be looked into.

cl

Comment 14

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #11)
> This is a small testcase that trips up the same thing that the
> http://ads1.msn.com/library/dap.js script does. It opens up the contentDocument
> on an iframe, and never closes it. If I've read the code correctly, it's
> because opening an HTML document creates a wyciwyg channel, which is added to
> the load group of the page (hence the throbber). Since it's never closed, and
> it's a channel not actually backed by a network request, it spins forever.

That's just bug 81980, right?

Comment 15

[Manish Singh]

Hm, indeed it is. So Microsoft arbitrarily decided that bug 81980 is fixed in Firefox 3, nevermind the reality.

We (Flock) have contacts at Digg, and have told them what's up.

Comment 16

[Joanne Killow]

this bug also prevents some of the functionality. e.g. the "share" icon (either underneath a story's description on the main page, or on the comments page for a particular story) doesn't work as long as the spinning activity indicator is still spinning.  The share function will only work after you tell Firefox to stop loading the page.

Comment 17

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

This seems to have been fixed by digg!

Comment 18

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED using:

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9pre) Gecko/2008040404 Minefield/3.0pre

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9pre) Gecko/2008040404 Minefield/3.0pre

-and-

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9pre) Gecko/2008040404 Minefield/3.0pre

Comment 19

[Dão Gottwald [:dao]]

(In reply to comment #17)
> This seems to have been fixed by digg!

By removing the msn ads, right? So this is still an issue, even if not on digg.com. See bug 427771.

Comment 20

[Jim]

If this bug is meant to be specific to digg.com not working: then this bug is resolved, but we should open a new one (and transfer Manish's test case there).

If this bug is not meant to be specific to digg.com, then this bug should be reopened.  It could then be set to "NOTAPROBLEM" if Firefox's never-stop-throbbing behavior is considered correct for never-closed document writes, or set to RESOLVED once Firefox's code is changed.

Comment 21

[Jesse Ruderman]

That's bug 81980, which is controversial.  But note that "fixing" that bug would only stop the throbber without fixing issues like the one in comment 16 that result from the onload not firing.

Comment 22

[Manish Singh]

(In reply to comment #20)
> If this bug is meant to be specific to digg.com not working: then this bug is
> resolved, but we should open a new one (and transfer Manish's test case there).

Bug 81980 already has a test case.

(In reply to comment #19)
> By removing the msn ads, right? So this is still an issue, even if not on
> digg.com. See bug 427771.

Well, to be accurate, they didn't remove the msn ad code, but put in code in digg hosted JS that closes the offending documents. So they just worked around the problem.

Comment 23

[Daniel Gutson]

Hi,
  I'm not sure it's about the same issue, but I experience something similar with gmail, where after loading everything, FF never ends while saying "Transferring data from chatenabled.mail.google.com...".

I'm using FF3b5

Still Tech Evangelism?

  Daniel.