Closed Bug 627729 Opened 13 years ago Closed 13 years ago

Selectively use the old HTML parser for Hotmail to work around repeated reloading

Categories

(Core :: DOM: HTML Parser, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Tracking Status
blocking2.0 --- final+

People

(Reporter: josh.tumath+bugzilla, Assigned: hsivonen)

References

Details

(Whiteboard: [4b10][hardblocker][has patch][backout patch in bug 636692])

Attachments

(1 file, 2 obsolete files)

User-Agent:       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b10pre) Gecko/20110121 Firefox/4.0b10pre
Build Identifier: 

Since the 2010-01-19 trunk build, I have been having problems in Hotmail.

The problem begins after clicking on an email to preview it or going into a different folder. When I do this, the page continually reloads.

This causes many problems, including:-
- When I scroll down the list of emails, it will keep going to the top again.
- It is difficult to preview a different email, because it might go back to previewing the email that I had originally selected.

Reproducible: Always
Version: unspecified → Trunk
Not that I got this in the console:
"Error: Permission denied for <http://view.atdmt.com> to call method Location.toString"
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows 7 → All
Hardware: x86_64 → All
Hmm, doing the regression test, then this issue appears, there is warning about an empty string passed to getElementById(). The error is new but the wrong behavior seems old.
This is a regression from Bug 622480, I think :(
If that is the case, I'll back out bug 622480 immediately.
Er, no, was making a mistake in testing. Something else has caused the regression.
I think the error is an issue but the continuous reloading is another. For what I've been able to see from mozregression, the issue described in comment 0 has been introduced here:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=83c887dff0da&tochange=d6bb0f9e9519
I was trying to find the regression for the reload issue, which is something new.
When it's in the continuous-reloading mode, can you get me a callstack to nsDocShell::InternalLoad ?
The range in comment 6 has the html5 parser pref flip in it.  Does turning that off make the problem go away?
(In reply to comment #9)
> The range in comment 6 has the html5 parser pref flip in it.  Does turning that
> off make the problem go away?

According to Olli on IRC, it is. I'm going to double-check that by reverting to this rev on a box.
(In reply to comment #8)
> When it's in the continuous-reloading mode, can you get me a callstack to
> nsDocShell::InternalLoad ?

#0  nsDocShell::InternalLoad (this=0x7fffd085af60, aURI=0x7fffd085a130, aReferrer=0x7fffd0029960, aOwner=0x7fffd03f4430, aFlags=0, aWindowTarget=0x7fffd0316d20, aTypeHint=0x0, aPostData=0x0, aHeadersData=0x0, aLoadType=1, aSHEntry=0x0, 
    aFirstParty=0, aDocShell=0x0, aRequest=0x0) at nsDocShell.cpp:7572
#1  0x00007fffe61aab06 in nsDocShell::LoadURI (this=0x7fffd085af60, aURI=0x7fffd085a130, aLoadInfo=0x7fffd07853e0, aLoadFlags=0, aFirstParty=0) at nsDocShell.cpp:1381
#2  0x00007fffe6ff2a70 in nsFrameLoader::ReallyStartLoadingInternal (this=0x7fffd03a2070) at nsFrameLoader.cpp:271
#3  0x00007fffe6ff260c in nsFrameLoader::ReallyStartLoading (this=0x7fffd03a2070) at nsFrameLoader.cpp:233
#4  0x00007fffe6fcfcad in nsDocument::MaybeInitializeFinalizeFrameLoaders (this=0x7fffd027f360) at nsDocument.cpp:5376
#5  0x00007fffe6fca2dd in nsDocument::EndUpdate (this=0x7fffd027f360, aUpdateType=1) at nsDocument.cpp:3930
#6  0x00007fffe7194810 in nsHTMLDocument::EndUpdate (this=0x7fffd027f360, aUpdateType=1) at nsHTMLDocument.cpp:3019
#7  0x00007fffe6d40b6d in mozAutoDocUpdate::~mozAutoDocUpdate (this=0x7fffffff6100, __in_chrg=<value optimized out>) at ./../../content/base/src/mozAutoDocUpdate.h:66
#8  0x00007fffe70edc0c in nsGenericHTMLElement::SetInnerHTML (this=0x7fffdc86b720, aInnerHTML=...) at nsGenericHTMLElement.cpp:699
#9  0x00007fffe70f5661 in nsGenericHTMLElementTearoff::SetInnerHTML (this=0x7fffd0395c20, aInnerHTML=...) at nsGenericHTMLElement.cpp:186
#10 0x00007fffee6f8f6f in nsIDOMNSHTMLElement_SetInnerHTML (cx=0x22ac190, obj=0x7fffd5879980, id=140736979998660, vp=0x7fffffff6550) at dom_quickstubs.cpp:17596
#11 0x00007ffff759796e in JSScopeProperty::set (this=0x7fffdcab5958, cx=0x22ac190, obj=0x7fffd5879980, vp=0x7fffffff6550) at jsscope.h:1025
#12 0x00007ffff7591cd1 in js_SetPropertyHelper (cx=0x22ac190, obj=0x7fffd5879980, id=140736979998660, defineHow=1, vp=0x7fffffff6550) at jsobj.cpp:5082
#13 0x00007ffff755f86f in js_Interpret (cx=0x22ac190) at jsops.cpp:1822
#14 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x245a1b0, flags=0) at jsinterp.cpp:842
#15 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=1, vp=0x245a140) at jsfun.cpp:2069
#16 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#17 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x245a010, flags=0) at jsinterp.cpp:842
#18 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=1, vp=0x2459fa0) at jsfun.cpp:2069
#19 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#20 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x2459cd8, flags=0) at jsinterp.cpp:842
#21 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=1, vp=0x2459c68) at jsfun.cpp:2069
#22 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#23 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x2459c30, flags=0) at jsinterp.cpp:842
#24 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=1, vp=0x2459bc0) at jsfun.cpp:2069
#25 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#26 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x2459ba8, flags=0) at jsinterp.cpp:842
#27 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=1, vp=0x2459b38) at jsfun.cpp:2069
#28 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#29 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=0, vp=0x2459b28, flags=0) at jsinterp.cpp:842
#30 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=0, vp=0x2459ab8) at jsfun.cpp:2069
#31 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#32 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=3, vp=0x24598b0, flags=0) at jsinterp.cpp:842
#33 0x00007ffff74faab4 in array_extra (cx=0x22ac190, mode=FOREACH, argc=3, vp=0x2459868) at jsarray.cpp:3149
#34 0x00007ffff74facb4 in array_forEach (cx=0x22ac190, argc=1, vp=0x2459868) at jsarray.cpp:3205
#35 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#36 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=3, vp=0x2459760, flags=0) at jsinterp.cpp:842
#37 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=3, vp=0x24596f0) at jsfun.cpp:2069
#38 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#39 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=4, vp=0x24595b0, flags=0) at jsinterp.cpp:842
#40 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=4, vp=0x2459540) at jsfun.cpp:2069
#41 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#42 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=4, vp=0x24594f0, flags=0) at jsinterp.cpp:842
#43 0x00007ffff753b799 in js_fun_apply (cx=0x22ac190, argc=4, vp=0x2459480) at jsfun.cpp:2069
#44 0x00007ffff7561d2c in js_Interpret (cx=0x22ac190) at jsops.cpp:2210
#45 0x00007ffff75754bd in js_Invoke (cx=0x22ac190, argc=1, vp=0x2459358, flags=0) at jsinterp.cpp:842
#46 0x00007fffee6788cd in nsXPCWrappedJSClass::CallMethod (this=0xdd8600, wrapper=0x2aec890, methodIndex=3, info=0x9f8890, nativeParams=0x7fffffffcb70) at xpcwrappedjsclass.cpp:1696
#47 0x00007fffee66ec93 in nsXPCWrappedJS::CallMethod (this=0x2aec890, methodIndex=3, info=0x9f8890, params=0x7fffffffcb70) at xpcwrappedjs.cpp:570
#48 0x00007ffff69214a9 in PrepareAndDispatch (self=0x2aeb620, methodIndex=3, args=0x7fffffffcd10, gpregs=0x7fffffffcc90, fpregs=0x7fffffffccc0) at xptcstubs_x86_64_linux.cpp:153
#49 0x00007ffff692153d in SharedStub () at xptcstubs_x86_64_linux.cpp:159
#50 0x00007fffe70e5926 in nsDOMEventListenerWrapper::HandleEvent (this=0x2aec910, aEvent=0x21cb090) at nsDOMEventTargetHelper.cpp:65
#51 0x00007fffe70a26cf in nsEventListenerManager::HandleEventSubType(struct {...} *, nsIDOMEventListener *, nsIDOMEvent *, nsPIDOMEventTarget *, PRUint32, nsCxPusher *) (this=0x2af16e0, aListenerStruct=0x2af1728, aListener=0x2aec910, 
    aDOMEvent=0x21cb090, aCurrentTarget=0x2aeb2e0, aPhaseFlags=6, aPusher=0x7fffffffd210) at nsEventListenerManager.cpp:1085
#52 0x00007fffe70a2b8e in nsEventListenerManager::HandleEventInternal (this=0x2af16e0, aPresContext=0x0, aEvent=0x2424620, aDOMEvent=0x7fffffffd1e0, aCurrentTarget=0x2aeb2e0, aFlags=6, aEventStatus=0x7fffffffd1e8, aPusher=0x7fffffffd210)
    at nsEventListenerManager.cpp:1181
#53 0x00007fffe70d2257 in nsEventListenerManager::HandleEvent (this=0x2af16e0, aPresContext=0x0, aEvent=0x2424620, aDOMEvent=0x7fffffffd1e0, aCurrentTarget=0x2aeb2e0, aFlags=6, aEventStatus=0x7fffffffd1e8, aPusher=0x7fffffffd210)
    at nsEventListenerManager.h:144
#54 0x00007fffe70d2787 in nsEventTargetChainItem::HandleEvent (this=0x2008530, aVisitor=..., aFlags=6, aMayHaveNewListenerManagers=0, aPusher=0x7fffffffd210) at nsEventDispatcher.cpp:212
#55 0x00007fffe70d0510 in nsEventTargetChainItem::HandleEventTargetChain (this=0x2008530, aVisitor=..., aFlags=6, aCallback=0x0, aMayHaveNewListenerManagers=0, aPusher=0x7fffffffd210) at nsEventDispatcher.cpp:341
#56 0x00007fffe70d11ed in nsEventDispatcher::Dispatch (aTarget=0x2aeb2e0, aPresContext=0x0, aEvent=0x2424620, aDOMEvent=0x21cb090, aEventStatus=0x0, aCallback=0x0, aTargets=0x0) at nsEventDispatcher.cpp:628
#57 0x00007fffe70d15c2 in nsEventDispatcher::DispatchDOMEvent (aTarget=0x2aeb2e0, aEvent=0x0, aDOMEvent=0x21cb090, aPresContext=0x0, aEventStatus=0x0) at nsEventDispatcher.cpp:691
#58 0x00007fffe70e658d in nsDOMEventTargetHelper::DispatchDOMEvent (this=0x2aeb2e0, aEvent=0x0, aDOMEvent=0x21cb090, aPresContext=0x0, aEventStatus=0x0) at nsDOMEventTargetHelper.cpp:229
#59 0x00007fffe7070985 in nsXMLHttpRequest::ChangeState (this=0x2aeb2e0, aState=16, aBroadcast=1) at nsXMLHttpRequest.cpp:3019
#60 0x00007fffe706c7a4 in nsXMLHttpRequest::RequestCompleted (this=0x2aeb2e0) at nsXMLHttpRequest.cpp:2190
#61 0x00007fffe706c626 in nsXMLHttpRequest::OnStopRequest (this=0x2aeb2e0, request=0x2aec270, ctxt=0x0, status=0) at nsXMLHttpRequest.cpp:2153
#62 0x00007fffe6f9b0b4 in nsCrossSiteListenerProxy::OnStopRequest (this=0x2b08e80, aRequest=0x2aec270, aContext=0x0, aStatusCode=0) at nsCrossSiteListenerProxy.cpp:340
#63 0x00007fffecc421ee in nsHTTPCompressConv::OnStopRequest (this=0x2b28a20, request=0x2aec270, aContext=0x0, aStatus=0) at nsHTTPCompressConv.cpp:127
#64 0x00007fffeccdd408 in nsHttpChannel::OnStopRequest (this=0x2aec220, request=0x2b0aa50, ctxt=0x0, status=0) at nsHttpChannel.cpp:5305
#65 0x00007fffecbe3508 in nsInputStreamPump::OnStateStop (this=0x2b0aa50) at nsInputStreamPump.cpp:578
#66 0x00007fffecbe2d9e in nsInputStreamPump::OnInputStreamReady (this=0x2b0aa50, stream=0x2b0a808) at nsInputStreamPump.cpp:403
#67 0x00007ffff68cf7e3 in nsInputStreamReadyEvent::Run (this=0x2b0ab30) at nsStreamUtils.cpp:112
#68 0x00007ffff69010c3 in nsThread::ProcessNextEvent (this=0x643730, mayWait=1, result=0x7fffffffd77c) at nsThread.cpp:527
#69 0x00007ffff6883395 in NS_ProcessNextEvent_P (thread=0x643730, mayWait=1) at nsThreadUtils.cpp:250
#70 0x00007fffe84a9d15 in nsBaseAppShell::Run (this=0xe87280) at nsBaseAppShell.cpp:178
#71 0x00007fffe3d052f1 in nsAppStartup::Run (this=0xf1c730) at nsAppStartup.cpp:184
#72 0x00007ffff7990fa2 in XRE_main (argc=3, argv=0x7fffffffe078, aAppData=0x607680) at nsAppRunner.cpp:3564
#73 0x000000000040136f in main (argc=4, argv=0x7fffffffe078) at nsBrowserApp.cpp:158
I realize it would have been nicer to have that in a attachment...
That's reloading a subframe, right?  Is the relevant iframe inside the innerHTML text being set?  I guess the other question is what kicks off the load.

And yes, attachments for stacks, please.
I've double-checked, it's a regression from HTML5 Parser.
I guess we should think of blocking given that using Hotmail is really annoying with this bug.
blocking2.0: --- → ?
Raul, has something changed recently in Hotmail?
Component: DOM → HTML: Parser
QA Contact: general → parser
Can you be more specific on what changes you are interested in? We have frequent updates to the site and will vary based on your account.
FWIW, the reload bug wasn't happening before I updated my version of Minefied. I just updated to 2011-01-21 and what I'm seeing is the ads refreshing once after a message refresh.
UA spoofing helps.
This problem fixed if I set general.useragent.override to "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/10.0.607.0 Safari/534.14".

Tested on
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b10pre) Gecko/20110121 Firefox/4.0b10pre ID:20110121030329
Ah, is this more of hotmail's "assume random things about how different UAs handle scripts and have two different codepaths even though the IE/Webkit one works for all browsers" stuff?
This is a top top complaint on SUMO (and input) about Firefox 4.  Complaints started coming in on the 19th (really blew up on the 20th) which doesn't correlate at all with any Firefox 4 beta pushes.
Raul, is there a change around the 19th that could produce the described issue?
Whiteboard: [4b10]
pinging the hotmail team and making them aware.
...and if I'd read more, I'd see that Raul is on the bug already. Raul, feel free to ping me direct, I sent email, and see if we can get to the bottom of things.
FWIW, I, too, had a look at the pushlog from 18th to 20th (inclusive) and I didn't see any changes to the content side of Gecko that could have plausible caused this.
This bug is not recent regression of Minefield.

Now, this happens from the following old  build.
http://hg.mozilla.org/mozilla-central/rev/3a7920df7580
Mozilla/5.0 (Windows; U; Windows NT 6.1; WOW64; en-US; rv:1.9.3a5pre) Gecko/20100503 Minefield/3.7a5pre ID:20100503105056

I guess this problem is caused by MS's changes( around 01-18-2011).
Yes, this is very likely caused by a recent change in Hotmail that is not compatible with the HTML5 parser.
Tons of reports coming in on this issue.  This is a major problem and hard blocks betaN.
blocking2.0: ? → betaN+
Whiteboard: [4b10] → [4b10][hardblocker]
Who owns this?
Damon, per comment 25 it sounds like it's a bug on Hotmail's end and we need to contact them....
I think this needs two owners, for now. One assuming hotmail can fix it on their end (kev?) and a second on our end in case we have to work around this with a change to our code -- assuming that's possible.
On our end that should likely be Henri....
They have been contacted (Raul is on it from MS, and has commented above). Who
can I put him in contact with to assist with tracking it down? From what I see,
we think that something changed on Hotmail around the 20th, but it's unclear
from there side if it's something in the beta that changed. Who can I put in
touch with Raul direct to help track (Raul has said they've been in touch with Henri as well)?
Kev, see comment 25.  This problem is happening in builds going back to May.  It certainly didn't use to happen before, with those same builds.  

Henri is probably the best contact...
I guess, I'm the contact.
Assignee: nobody → hsivonen
I get a slightly different regression range from comment 25. The build from 20100504040059 is when I start seeing this problem on XP, around the time the html5 parser was turned on by default. Previous builds don't refresh every second when viewing mail.
(In reply to comment #35)
> I get a slightly different regression range from comment 25. The build from
> 20100504040059 is when I start seeing this problem on XP, around the time the
> html5 parser was turned on by default. Previous builds don't refresh every
> second when viewing mail.
That is hourly build which is html5  parser was turned on by default.
Whiteboard: [4b10][hardblocker] → [4b10][hardblocker][see comment 26]
Microsoft has a fix.

A Gecko-side fix wouldn't be appropriate, because Firefox 4 is behaving per spec here (and the spec makes sense).

I'm leaving this open until I get a confirmation that the fix has been pushed to production.
Status: NEW → ASSIGNED
Whiteboard: [4b10][hardblocker][see comment 26] → [4b10][hardblocker][Microsoft has a fix; see comment 38]
(In reply to comment #38)
> Microsoft has a fix.
> 
> A Gecko-side fix wouldn't be appropriate, because Firefox 4 is behaving per
> spec here (and the spec makes sense).

This is somewhat of a slippery slope, isn't it (i.e. relying on a third party to correct the issue and maintain the fix on a go forward basis)?  I have no idea how much work a Gecko-side fix would be, but might it not be an idea to have one should this issue re-appear?
(In reply to comment #40)
> This is somewhat of a slippery slope, isn't it (i.e. relying on a third party
> to correct the issue and maintain the fix on a go forward basis)?

Being able to browse the Web with Firefox means relying on third parties to maintain compatible code all the time.

See comment 26. This wasn't a Gecko regression around Jan 18th. It was a change to Hotmail code around Jan 18th, so it's pretty reasonable to apply the fix to Hotmail code.

I realize that it's inconvenient for beta users to wait for the fix to get deployed, but I'd like to ask for a bit of patience here. Unfortunately, beta testing comes with a higher risk of incidents like this than using "final" browser releases.

> I have no
> idea how much work a Gecko-side fix would be, but might it not be an idea to
> have one should this issue re-appear?

The probability of problems like this getting introduced to site code gets much lower once Firefox 4 final has been shipped and site developers are more likely to test Firefox 4 as part of their routine processes.
(In reply to comment #41)
> I realize that it's inconvenient for beta users to wait for the fix to get
> deployed, but I'd like to ask for a bit of patience here. Unfortunately, beta
> testing comes with a higher risk of incidents like this than using "final"
> browser releases.

Fair enough, my apologies if it came across as if I was pushing for an immediate fix.  I read the comment chain and saw that the issue was introduced by a Microsoft change.  I was thinking that long term, if it was a trivial fix, it may be worth including it given the chance that the issue may occur again, if not on Hotmail, perhaps on some other site.

> 
> > I have no
> > idea how much work a Gecko-side fix would be, but might it not be an idea to
> > have one should this issue re-appear?
> 
> The probability of problems like this getting introduced to site code gets much
> lower once Firefox 4 final has been shipped and site developers are more likely
> to test Firefox 4 as part of their routine processes.
If it were a trivial fix, we would have done it.  It's not (e.g. fixing it without breaking other sites requires radical changes of some sort to the way the HTML parser works... and no one knows what sort of changes would be needed to avoid the site breakage).
Confirming on the latest Minefield build (1/27/11 Windows) under a Windows Live account (login.live.com) and going to my e-mail.  It looks like a communications issue between the webpage and the advertisement server, as the right-hand side ad keeps changing when trying to read an e-mail.  With the page constantly reloading, other tabs' refresh / scrolling times are slowed.

For all of you Hotmail / Windows Live users.....
I recommend downloading the Windows Live Mail software program to your computer until a solution is found to this bug.  The software is much faster loading your e-mail, because all messages already downloaded load from your hard drive.  You can always re-sync again with (login.live.com) when this bug is resolved.  The download link is:

http://explore.live.com/windows-live-mail?os=other
The ads don't even display at all under Internet Explorer 9 Beta (but the page isn't in a refresh loop), so something is wrong on Microsoft's end.
(In reply to comment #38)
> A Gecko-side fix wouldn't be appropriate, because Firefox 4 is behaving per
> spec here (and the spec makes sense).

What was the (non-)issue? Are there already relevant tests in the tree for it?
(In reply to comment #49)
> (In reply to comment #38)
> > A Gecko-side fix wouldn't be appropriate, because Firefox 4 is behaving per
> > spec here (and the spec makes sense).
> 
> What was the (non-)issue? 

I'm not sure what level of detail the Hotmail team is comfortable with sharing here.

> Are there already relevant tests in the tree for it?

Not sure.

- -

damons, is it OK to move this bug report to Tech Evangelism (which would cancel the blocking +, since evang bugs can't block), since this is now basically being handled as an evang bug that's waiting for an already-developed fix to get deployed?
I think there's value in having this listed as a blocker so we keep in on the radar in case MS doesn't have the fix deployed when we're getting very close we can re-connect with them and ask harder.
Whiteboard: [4b10][hardblocker][Microsoft has a fix; see comment 38] → [4b10][hardblocker][evangelism][Microsoft has patch for hotmail; see comment 38]
I agree that blocker is good.

I am only a beta tester (not code contributer), and have had to discontinue beta use until this gets resolved.
that's odd.. i have not experienced this problem on any of the betas.
(In reply to comment #54)
> that's odd.. i have not experienced this problem on any of the betas.

For me, the problem has appeared within the last month or so. No issues with any of the betas and Hotmail prior to that time.
I should have been more specific.  Sorry.

Yes, Hotmail began to fail about two weeks ago (around 20Jan2011).

The seriousness of this issue to beta testers is easily accessed by doing the following:

1) Visit page: http://input.mozilla.com/en-US/beta/search?product=firefox&version=4.0b10&date_start=&date_end=&sentiment=sad

2) Search for hotmail.

3) Look at the MANY, MANY, MANY unhappy beta testers.  Literally, and always, just in the past 2 hours.

4) No matter when you run this search, you will get MANY unhappy people, any time of the day, just minutes before.  It is impressive that there are so many beta testers.  Really cool.
i dont even have this problem on pre beta 12.
(In reply to comment #57)
> i dont even have this problem on pre beta 12.

This issue comes from a change in Hotmail around the 20th of January. Maybe that change hasn't been deployed to all users. According to comment 16, it's possible.
My simple, but annoying, workaround is to just hit the escape key to kill the refreshes.  I can then at least read my mail.
(In reply to comment #59)
> My simple, but annoying, workaround is to just hit the escape key to kill the
> refreshes.  I can then at least read my mail.

A simpler workaround would be to disable the html5 parser: go to "about:config" and change to false 'html5.enable'. If you do that, don't forget to re-enable it after!
i personally think this shouldnt block any of the beta releases or release candidates.. only block the final release.
(In reply to comment #61)
> i personally think this shouldnt block any of the beta releases or release
> candidates.. only block the final release.

i say this because the fix might not be out for a while because they only roll out changes to a few users at a time. if this delays firefox 4 for months than :\
@mattburles - we in Hotmail have a fix on the fast track before FF4.0 RC
I would still be interested to hear why this doesn't repro for you. Would you mind privately sending me the URL you land when going to Hotmail?
(raul@live.com)

(In reply to comment #62)
> (In reply to comment #61)
> > i personally think this shouldnt block any of the beta releases or release
> > candidates.. only block the final release.
> 
> i say this because the fix might not be out for a while because they only roll
> out changes to a few users at a time. if this delays firefox 4 for months than
> :\
okay sorry for the dup guys, juts found the thread here after posting this:
https://bugzilla.mozilla.org/show_bug.cgi?id=631483
Just screw Microsoft and be done with it. Redirect any complaints to them and forget about it. They'll have to fix it eventually.
yeah they are rather slow at rolling out updates so i could see it taking months for this to be fixed.. cant this just be put as a known issue on the release notes so people are away of it and it will be fixed when microsoft decides to.
(In reply to comment #67)
> yeah they are rather slow at rolling out updates so i could see it taking
> months for this to be fixed.. cant this just be put as a known issue on the
> release notes so people are away of it and it will be fixed when microsoft
> decides to.

I don't think this is necessary.  See comment 63: "we in Hotmail have a fix on the fast track before FF4.0 RC".
folks, try and keep comments on point, instead of using the forum to bash. the Hotmail team has said they have a fix that will be pushed before RC. they're also asking for assistance in finding places where it isn't reproducible. if you can help, awesome. if not, rant elsewhere, please.
is that all the information they gave was that it will be before release candidate.. no estimated date or time frame?
@mattburles

Correct, we've lined up a fix and are committed to deploying before FF 4.0 RC timeframe.

(In reply to comment #70)
> is that all the information they gave was that it will be before release
> candidate.. no estimated date or time frame?
(In reply to comment #71)
> Correct, we've lined up a fix and are committed to deploying before FF 4.0 RC
> timeframe.

Is there an ETA on FF 4.0 RC?
(In reply to comment #73)
> (In reply to comment #71)
> > Correct, we've lined up a fix and are committed to deploying before FF 4.0 RC
> > timeframe.
> 
> Is there an ETA on FF 4.0 RC?

I cannot find the site, but heard Mozilla is aiming to release the RC by the end of February.  If you want to know how many bugs remain before release, just go to:

https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking2%3Abeta%2Cfinal%20sw%3Ahard


There are currently 47 bugs to be fixed before the RC will be released and even though the listing will eventually say "Zarro boogs found" (zero bugs = Bugzilla word play = even though zero bugs currently remain there could be some that haven't been found yet) it takes a couple more days for all the language builds to be ready and posted on the official Mozilla download pages.
(In reply to comment #73)
> Is there an ETA on FF 4.0 RC?
Bugzilla is not the appropriate forum for unrelated discussions of release schedules. Please take this kind of discussion to the newsgroup. 

The only people that should be discussing release timing in a bug are the engineers/qa/product people who need to know specifics in order to determine how the particular bug fits into the release.
(In reply to comment #75)
> (In reply to comment #73)
> > Is there an ETA on FF 4.0 RC?
> Bugzilla is not the appropriate forum for unrelated discussions of release
> schedules. Please take this kind of discussion to the newsgroup. 

Fair enough, but where the bug fix from Microsoft is apparently tied to 4.0 RC, I was attempting to get an idea of the time frame for the bug fix.
In meantime "IE tab 2" addon works with hotmail with just very minor glitches.

Till Hotmail hot-team fixes the bug sometime in far far future...
i dont get why this is blocking beta N shouldnt it be final+? i say this because it would be blocking beta 12 and isnt the fix going to be after beta 12 meaning before release candidate?
Just some feedback...while disabling html5 and using the escape key both work, my alternative is adding *.live.com/* to the sites filter on IE Tab Plus. Not a bad idea if you already use the add-on or use other Live features like Mesh which require IE anyways.
Adjusting the status whiteboard so that this doesn't show up in softblocker queries!
Whiteboard: [4b10][hardblocker][evangelism][Microsoft has patch for hotmail; see comment 38] → [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38]
Moving to final.
blocking2.0: betaN+ → final+
Is this fix live yet?
I do not believe so.
what is Microsoft waiting for?
OK, when can we expect this to go live?
People, PLEASE stop spamming this bug report and asking "when will this land"!

I keep getting e-mail updates on this bug thinking it has been resolved, only to find out that people can't read all the comments before posting.

Comment 63 from Raul D at Microsoft says the patch will be applied for the RELEASE CANDIDATE, not for Beta 12 or a possible Beta 13, so STOP asking!


I don't work for Mozilla, but wish I had authority to change the above "Whiteboard" section to state "Please read comment 63 before responding".....
<3 Chris B. maybe you want to bookmark the bug and check it periodically instead?  I checked to see if there was a live title that would give the status of the bug, but it looks like no.

Alternately you can change your email preferences so you'll only be notified if the status of the bug changes - https://bugzilla.mozilla.org/userprefs.cgi?tab=email
I don't comment on platform stuff much, but it seems like this should not be a [hardblocker] for the following reasons:

- The bug is exclusively caused by client-side code making overreaching, incorrect assumptions about the UA.
- We have notified MS about the bug and their patch seems imminent.
- Why are we slapping [hardblocker] status on such a specific, client-caused bug with "too big to fail" as the reason? Will there not inevitably be many large sites that throw errors due to errant UA sniffing? (feature detection FTW)

Just playing devil's advocate against the [hardblocker] tag here ;)
Why is "hardblocker" necessary?

I have an important hotmail email address. I cannot use hotmail with this bug. I have two viable options:
A) Stop using hotmail (even if it's very important for me as I said).
B) Switch to Chrome.

Guess which one I chose...

This bug must be fixed before 100000 persons do the same. Hotmail, whether you like it or not, is a major product used by LOTS of people.

Sad but true
(In reply to comment #93)
> - Why are we slapping [hardblocker] status on such a specific, client-caused
> bug with "too big to fail" as the reason? Will there not inevitably be many
> large sites that throw errors due to errant UA sniffing? (feature detection
> FTW)
> 
> Just playing devil's advocate against the [hardblocker] tag here ;)

I'm not sure devil's advocate discussions fit with bugzilla etiquette, but...

I would hope there aren't "many large sites" (for large values of large) that don't work in Firefox 4. If there are, they should probably also be hardblockers too.

I'm sure shipping a browser which only works on correct sites would be an admirable goal and I imagine sure it would make a lot of the development easier. But you won't find many people using it...
@alexandre, please understand, this is *not* a Firefox bug, and that is the reason why I think [hardblocker] status should be removed, because it makes people think it is unless they read through 94 comments. (I do agree with you that it is *important* and we should keep after it!)

The fact is, the bug is cause by the code on Hotmail's page, code which we don't control, and thus have no direct ability to fix. The code pattern that causes the bug, UA sniffing, is performed by many sites at their own folly. Switching to a new browser would just be a temporary escape from single piece of bad code. If Chrome updates with new features or a new version in the future, this could very well happen to you again.

We have to send a clear message here: Firefox is *not* breaking Hotmail, Hotmail is breaking *itself*.

While I will submit to the judgement of folks with more wisdom than I (Beltzner, Johnathan, etc) as to the [hardblocker] status of this bug, it certainly is going to give people the impression that we are the at-fault party.

OK, I did my best to "lobby for or against hardblockers". Here's hoping it at least provoked some thoughtful consideration :)
Daniel - I think the whiteboard status takes care of your concerns "Microsoft has patch..."  Otherwise bugzilla needs to be used for the productivity of the developers, and not cater towards those who won't actually read. The justification for making this a hardblocker makes it much more likely that hotmail will work in Fx4 and that's what matters.

Another compromise might be to change the name of the bug "Make sure microsoft pushes hotmail fix before final" but we should leave that to those who are actually working on the bug.
Lucy has a hellofa good idea IMO: +1 for retitling this bug "Make sure Microsoft fixes Hotmail reload bug before Fx 4 final"
(In reply to comment #98)
> Lucy has a hellofa good idea IMO: +1 for retitling this bug "Make sure
> Microsoft fixes Hotmail reload bug before Fx 4 final"

Will do.
Keywords: regression
Summary: Hotmail web page continually reloads every second → Ensure that Microsoft fixes Hotmail reload bug before Firefox 4 final
Whiteboard: [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38] → [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38 and comment 71]
Is there a way we can hardcode a workaround in our browser for this, since we can't get a guaranteed fix out of Microsoft by the time we ship?
(In reply to comment #100)
> Is there a way we can hardcode a workaround in our browser for this, since we
> can't get a guaranteed fix out of Microsoft by the time we ship?

Since this bug involves the constant loading of the advertisements, why don't you temporarily make a patch that disables the right-side ads, then when Microsoft has the official patch you can back out the advertisement-disable patch and apply the official fix from Microsoft.  I don't think the advertisers will notice a slight decline in revenue for a couple weeks, as Firefox isn't the only browser people use.
The solution that has gained traction on the SUMO forums is disabling the HTML5 parser via about:config.  An adblock-type solution seems to have only moderate success.
(In reply to comment #102)
> The solution that has gained traction on the SUMO forums is disabling the HTML5
> parser via about:config.  An adblock-type solution seems to have only moderate
> success.

Okay - but if the HTML5 parser is disabled through a patch, will that apply to ALL sites that use HTML Version 5.....OR.....can (login.live.com) be forced to load using HTML 4 and other sites can still load using HTML Version 5, if the site supports it?

I don't think it is a good idea to turn off support for the new HTML Version 5 for ALL sites, if only ONE is causing problems.
Can people please stop spamming the bug?  It's really getting in the way of actually doing something about it....

Henri, turning off the HTML5 parser for the affected hostname(s) only was sounding like the only palatable solution to us here.  How feasible is that?
(In reply to comment #104)
> Can people please stop spamming the bug?  It's really getting in the way of
> actually doing something about it....
> 
> Henri, turning off the HTML5 parser for the affected hostname(s) only was
> sounding like the only palatable solution to us here.  How feasible is that?

Boris,

I wasn't spamming the bug.  I was offering a solution.  What's wrong with that???
(In reply to comment #102)
> The solution that has gained traction on the SUMO forums is disabling the HTML5
> parser via about:config. 

This may be slightly tangential to this bug, but what happens to all the people who were advised via SUMO to disable the html5 parser in about:config ; once Hotmail fixes the problem? 

Six months along the line, people are going to be using the same profiles with Firefox 4/5+ and be wondering why certain sites that rely on html5 parser specific features don't work. Unless the html5 parser pref becomes depreciated or later versions of Firefox reset it to true for all users, surely there are going to be longer reaching implications for those who were using Fx 4 betas and advised by SUMO to change the pref?
(In reply to comment #106)
We're not officially telling users to disable the parser. In fact, I've deliberately avoided giving that advice where I find it, instead opting to tell users to try to convince Microsoft to hurry the fix.  However as with all forums and such help sites, the steps that ACTUALLY WORK are the ones that will get all of the +1s and rise to the top.

So, yes, those users get a little screwed unless we can proactively re-enable html5 for them in a later release, but there are really no other reasonable workarounds.  Telling people to install IETab and figure it out isn't really viable -- even beta users don't want to go through that kind of hassle when the next post down says "change a pref" and has a 100% success rate.
In the past extensions that flip the preferences have been preferred since they can be blocklisted/incompatible with future releases. Perhaps it should be easier to make these extensions/have them available for SUMO's use.
(In reply to comment #104)
> Henri, turning off the HTML5 parser for the affected hostname(s) only was
> sounding like the only palatable solution to us here.  How feasible is that?

We cache the value of the html5.enable pref in nsHtml5Module::sEnabled.  We use this value to pick up alternate behavior when the HTML5 parser is enabled in a bunch of places: <http://mxr.mozilla.org/mozilla-central/search?string=nsHtml5Module%3A%3AsEnabled>  But it seems to me that in some of those places, such as in NS_NewHTMLIsIndexElement <http://mxr.mozilla.org/mozilla-central/source/content/html/content/src/nsHTMLSharedElement.cpp#154>, we do not have enough information about the source domain for the document, so after a quick look, special casing opting out of the HTML5 parser for some domains doesn't seem trivial...
I'd prefer not to special-case the parser choice for live.com. It would add attack surface and set a confusing precedent.
(In reply to comment #104)
> Henri, turning off the HTML5 parser for the affected hostname(s) only was
> sounding like the only palatable solution to us here.

There's another one: Using a less Firefox-ish and more WebKitish UA string for Hotmail.

> How feasible is that?

If my understanding of the problem is correct, we'd only need to pick the old parser for the document.open()ed case when the caller of open() is Hotmail. I supposed that would be possible given the nsHTMLDocument knows its URL.

However, I think faking the UA string in order to make Hotmail not run the offending code in the first place would be a less invasive hack. Also, it wouldn't add attack surface that Jesse mentions.

I'll get to investigating the feasibility of both options right away.

(In reply to comment #111)
> I'd prefer not to special-case the parser choice for live.com. It would add
> attack surface and set a confusing precedent.

Gecko has managed over a decade without site-specific hacks, so I think adding one now sets a bad precedent, since then sites can just go "well, add one of those hacks that you have anyway".
I figured that using the old parser is less invasive than varying the UA string. This patch forces the old parser for document.open()ed docs if the hostname ends with ".mail.live.com".

I didn't force the old parser for network streams or innerHTML, because forcing the old parser for those would be unnecessary.

In order to help Hotmail engineers test the behavior in the absence of this hack, I added a pref called html5.hotmailworkaround (which defaults to true).

I'd still very much like Hotmail to deploy their patch, because the old parser is going away after Firefox 4.
Attachment #515018 - Attachment is obsolete: true
Attachment #515025 - Flags: review?(bzbarsky)
Whiteboard: [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38 and comment 71] → [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38 and comment 71][has patch][needs review]
(In reply to comment #107)
> So, yes, those users get a little screwed unless we can proactively re-enable
> html5 for them in a later release, but there are really no other reasonable
> workarounds.  Telling people to install IETab and figure it out isn't really
> viable -- even beta users don't want to go through that kind of hassle when the
> next post down says "change a pref" and has a 100% success rate.

Filed as bug 636689 and nominated as a blocker.

Resummarizing this bug to reflect the new direction.

Spinning the evangelism tracking to bug 636690.

The removal of the site-specific hack is bug 636692.
Summary: Ensure that Microsoft fixes Hotmail reload bug before Firefox 4 final → Selectively use the old HTML parser for Hotmail to work around repeated reloading
Whiteboard: [4b10][hardblocker][evangelism][Micros‌oft has patch for hotmail; see comment 38 and comment 71][has patch][needs review] → [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs review]
Henri, I think even you'll agree this is an ugly hack. Maybe a blacklist for the HTML5 parser that starts off with Hotmail in it would be a better route. It would also be easy to then add another buggy site to it if needed.
(In reply to comment #116)
> It
> would also be easy to then add another buggy site to it if needed.

That's a very good reason not to do it.
(In reply to comment #117)
> (In reply to comment #116)
> > It would also be easy to then add another buggy site to it if needed.
> 
> That's a very good reason not to do it.

I can't disagree with that, but it's better than explicitly hardcoding for a bug in Hotmail or some other site if we end up in a similar situation.
Comment on attachment 515025 [details] [diff] [review]
Force the old parser in document.open() if hostname ends with ".mail.live.com", add pref to disable this hack

>+      PRInt32 index = host.RFind(".mail.live.com");
>+      if (index >= 0 && (PRUint32)index + 14 == host.Length()) {

How about:

  if (StringEndsWith(host, NS_LITERAL_CSTRING(".mail.live.com"))) {

instead?

r=me with that.  I really hope we don't have to use this.  :(
Attachment #515025 - Flags: review?(bzbarsky) → review+
Dave, even if we have to land the patch above (which is a big if), we plan to back it out the moment hotmail fixes this.  So no, infrastructure that we plan to keep around for managing things like this is not better...
(In reply to comment #120)
> Dave, even if we have to land the patch above (which is a big if)

Yes, we hope to not land this patch. If we do have to land it, we will back it out rather quickly.

The hotmail.com issue seems to be far and away the worst and most frequent negative feedback we get on our betas. Are we sure this patch fixes the bug, btw?
Do we know why Microsoft haven't pushed the fix yet? If we were to assume bad things, they could be choosing to hold it back for as long as possible to cause unhappy beta users and further complications for us. Unless they do the fix the issue before our final release, we will hear negative comments about whichever solution we choose. I don't want to believe that any company (even if it is Microsoft) would use their "power" in this way, though.
Don't attribute to malice what is more easily explained by just having a slow release cycle. They said it'd be ready for the 4.0 RC, which isn't here yet.
Yes but it is imminent.  This is the only BIG thing holding it up.  SO now we are in a chicken and egg thing.  I think we need to make sure it is clear to them that we are waiting on them at this point and are not going to be making changes that will break whatever fix they come up with as long as it works in a reasonable manner.
Yes, but they haven't replied here yet. :-\

I was wondering, would it be horrible if we did an extension for Hotmail users? We're told there's a fix on their end, it's just a matter of when it will go live. Perhaps we should just make an add-on that turns off the html5 parser and disable it when the fix is live?  

Obviously the drawback would be the users who don't bother looking for a fix and switch browsers, but maybe we could get the live/hotmail team to email their customers explaining that it's a problem on their end and link to the extension?
Well, rather than going through hoops to kludge this, my understanding (which could be wrong) is that spoofing the user-agent to say we are safari i9s sufficient to fix this.  If we are going for a kludge fix and that works then that is what I think should be done.  if the domain is hotmail.com, spoof the useragent.  This would seem to be safer than all of these other suggestions.
Although my personal opinion is that we should just release as is and let the EU take Microsoft back to court over this.
Whiteboard: [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs review] → [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs decision on whether to land]
Folks, please hold off on making any rash hacks or workarounds for Hotmail regarding this issue. Since it was brought up to us by our partners at Mozilla, we've been working diligently to investigate the bug and syncing up on progress. As I've already communicated through this medium we have a fix tested and ready to deploy shortly, and have been working on the proper release vehicle for it. Yes, we are running later than we'd hoped, but as promised will still have a fix before FF4.0 RC. 

As is the case with any major web service, this is unfortunately not the only issue we're working on at Hotmail. It would be foolish for us to neglect Firefox as the major platform it is for our shared customers, but there is also the reality and pitfalls inherent with a beta product and potential issues ensuing from changes necessary to move any technology product forward. 

The most reasonable workaround IMHO is for customers to temporarily disable the HTML5 capabilities if they haven't yet done so, but I will leave it to Mozilla folks to make the call on what the final recommendation should be.

I appreciate your patience and passion for getting the right fix. We are doing all we can to get this out as quickly as we can, hopefully without needing further action from Mozilla or our customers.

Thanks,

Raúl
(In reply to comment #128)
> Folks, please hold off on making any rash hacks or workarounds for Hotmail
> regarding this issue.

Agreed. Additionally the above comments implying some sort of conspiracy are fairly out of line. Please see https://bugzilla.mozilla.org/page.cgi?id=etiquette.html, specifically #1.

> Yes, we are running later than we'd hoped, but as promised will
> still have a fix before FF4.0 RC.

This is great to hear, though I'm not sure we ever heard you committing to releasing the fix before the RC (correct me if I am wrong). As a release manager this is of course nice, though with the RC build fairly imminent I wonder if it will be released after the RC build is created but before availability. If we didn't take a workaround in that case it would mean a leap of faith on our part and potentially tie the RC or final FF4 ship date to your rollout date...which is certainly less than ideal.

> As is the case with any major web service, this is unfortunately not the only
> issue we're working on at Hotmail. It would be foolish for us to neglect
> Firefox as the major platform it is for our shared customers, but there is also
> the reality and pitfalls inherent with a beta product and potential issues
> ensuing from changes necessary to move any technology product forward.

As we are keenly aware. Note though, that schedules unexpectedly slip so we need a contingency plan for FF4 as well...hence the bug to turn off the HTML5 parser if all other options are exhausted. We'll have to weigh the impact of the workaround vs the perceived uncertainty of the rollout date when we hit the point of no return.
Attachment 515305 [details] [diff] is qimportable and ready to go if the release drivers opt to go with this contingency plan. If a "go" decision comes when I'm asleep on my time zone, feel free to land after the release driver decision without waiting for me to wake up.

If someone other than me performs the landing(s) upon decision:
If by the time of the "go / no go" decision bug 636689 has gotten approval, please also land either one of the patches there. There's a patch that qimports on top of this one and another than qimports cleanly when this patch hasn't been applied.

(In reply to comment #119)
> How about:
> 
>   if (StringEndsWith(host, NS_LITERAL_CSTRING(".mail.live.com"))) {
> 
> instead?
> 
> r=me with that.

Uploaded a patch with that fixed. (I wish we had a more centralized and searchable location for all our string API stuff. I searched for "EndsWith" and failed to find that one.)

(In reply to comment #121)
> Are we sure this patch fixes the bug, btw?

The patch here successfully papers over the problem for me. I'll spin tryserver builds for all platforms to make it easy for others to verify.

(Furthermore, from what's been communicated to me about the nature of the Hotmail-side fix, my educated but untested belief is that this workaround doesn't break the real fix.)

(In reply to comment #125)
> I was wondering, would it be horrible if we did an extension for Hotmail users?

I think that's more complicated than landing a patch for Firefox 4.0 and backing it out for e.g. 4.0.1.

(In reply to comment #126)
> Well, rather than going through hoops to kludge this, my understanding (which
> could be wrong) is that spoofing the user-agent to say we are safari i9s
> sufficient to fix this.  If we are going for a kludge fix and that works then
> that is what I think should be done.  if the domain is hotmail.com, spoof the
> useragent.  This would seem to be safer than all of these other suggestions.

Actually, the patch attached here is safer. We don't know if spoofing a WebKit UA string breaks something else on Hotmail that hasn't been noticed yet.

(In reply to comment #128)
> Folks, please hold off on making any rash hacks or workarounds for Hotmail
> regarding this issue.

I hope we don't have to land the patch here, but I've prepared it so there's a contingency option ready to go.

> The most reasonable workaround IMHO is for customers to temporarily disable the
> HTML5 capabilities if they haven't yet done so, but I will leave it to Mozilla
> folks to make the call on what the final recommendation should be.

I think that's not a reasonable release-time workaround, because disabling the HTML5 parser not only disables a number of HTML parsing correctness issues (over 90 bugs reported against the old parser plus others that weren't tallied) but also disables the SVG-in-text/html feature. The SVG-in-text/html feature is a notable new feature of Firefox 4 and it would be bad if users failed to see the connection between following Hotmail-compat pref-flipping advice one day and an advertised new feature failing (and the advertising looking completely bogus) another day.

Also, even with beta users, we already have the problem of it being hard to get users to follow-up later and flip the pref back to a normal configuration. (Hence bug 636689.)

That said, I hope we don't need to put this contingency patch in the product.
I think we should go ahead and land this patch on trunk to get some testing feedback. Dropping an essentially untested patch in right before spinning an RC seems like a terrible idea. It seems lower risk to me if we back out the patch once Microsoft delivers on its promise. As a side-benefit trunk users get access to hotmail again.
Tryserver builds:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/hsivonen@iki.fi-1c6731c815d2/

It would be helpful if people who are experiencing this problem could test with a build from that directory to check if the workaround works for them.
Works fine for me with this build.
tried the trysever build and works for me too :)
"An unbalanced tree was written using document.write() causing data from the network to be reparsed."

Should I be getting this?
(In reply to comment #136)
> "An unbalanced tree was written using document.write() causing data from the
> network to be reparsed."
> 
> Should I be getting this?

Most likely, yes.

That's a warning that's emitted by the HTML5 parser (which is still used for network-originating content on Hotmail even with the workaround) when the page uses document.write() in a way that's not optimal for off-the-main-thread parsing performance. When that happens, the parser recovers but takes a bit more time to parse the page. The purpose of the warning is to help Web authors make pages that are nicer for off-the-main-thread parsing performance.
I also getting "Empty string passed to getElementById()" in the web console.

Does the workaround ignore this declaration?
(In reply to comment #138)
> I also getting "Empty string passed to getElementById()" in the web console.
> 
> Does the workaround ignore this declaration?

The workaround shouldn't affect that one either way.
Well, despite the comments against conspiracy theorists here, my opinion, which is i based on experience in trying to get this type of issue on the server side fixed, that checking in any kind of workaround will result in Microsoft NEVER fixing this issue, because they will decide we fixed in on our end.  IF we do check this into trunk, then we need to make sure it is backed out of any RC builds.  Otherwise Microsoft will just NEVER fix the real issue.
Microsoft are lazy when it comes to patches. Comment #140 hits the nail on the head; if the issue is still present they will fix it but if the issue is resolved one way or another they'll think nothing needs to be done.

I'd say make it **** MS so they'll be forced to fix it.
(In reply to comment #139)

Why would an empty string be passed to getElementById()? Sloppy coding? (Wouldn't be surprised.)
Thanks to everyone who tested the tryserver builds. I think we can conclude that the effectiveness of the workaround has been verified.

(In reply to comment #140)
> Well, despite the comments against conspiracy theorists here, my opinion, which
> is i based on experience in trying to get this type of issue on the server side
> fixed, that checking in any kind of workaround will result in Microsoft NEVER
> fixing this issue, because they will decide we fixed in on our end.

Past data about Hotmail suggests they follow through with promised fixes.

In January 2009, Chrome deployed a browser-side workaround for *.mail.live.com because were able to deploy a new build of the browser faster than Microsoft deployed a fix on Hotmail: http://src.chromium.org/viewvc/chrome/trunk/src/webkit/glue/webkit_glue.cc?revision=8764&view=markup&pathrev=8764

Microsoft deployed the Hotmail-side fix and Chrome was able to remove the workaround for Hotmail in September 2009: http://src.chromium.org/viewvc/chrome?view=rev&revision=25901

(Chrome is now using the infrastructure created for Hotmail to work around a similar issue with Yahoo! Mail: http://src.chromium.org/viewvc/chrome/trunk/src/webkit/glue/webkit_glue.cc?revision=32070&view=markup )
Nobody doubts that MS will follow through on their promise to have a fix in place before the RC. I just don't think we should take a scheduling dependency on it. Lets land the fix and make sure we are ready to go for the RC, and if as we assume MS obsoletes the need for the patch before we spin the RC, we pull it back. Agreed?
i think this should land so hotmail works for users during the RC period :)
Discussed in the triage call today - we should land the workaround now and leave this bug open so that we track the fix on hotmail's side, and the need for backout.
I agree this should be landed so people using the Firefox 4 RC don't have this problem. I'm sure many people are reliant on Hotmail and wouldn't expect this constant reload issue that doesn't happen in other browsers.

Will this become a meta bug since it will track the status of this issue?
Workaround patch landed, let's hope we can back this out before release. Leaving bug open.

http://hg.mozilla.org/mozilla-central/rev/47e21ce142a6
Whiteboard: [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs decision on whether to land] → [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs decision on whether to back out]
(In reply to comment #148)
> Workaround patch landed, let's hope we can back this out before release.
> Leaving bug open.
> 
> http://hg.mozilla.org/mozilla-central/rev/47e21ce142a6

I will try this just one more time for the hard of hearing.  The way web server admins who wait for the release candidate do things sork, is that they wait for a release candidate to:

1.  verify that the problem still exists

2.  Verify that the fix sitll works

Then they land it.

Having release candidates that include a workaround for the issue will ensure that condition 1 will never be met, so the problem will very likely NOT be fixed on the website.
I am not trying to say that having a fix in hand just in case they don;t fix this is a bad thing, and testing it on trunk is a good thing.  This needs to somehow not be included in release candidate builds because doing so will be counter productive to trying to get Microsoft to fix this on their end.  I have been through this before in my real job.
Bill, relax. MS has confirmed they intend to fix it. This workaround going in
the tree is merely so we get coverage on an alternate plan. When we go to build
the RC we have both options tested and can pick the proper way forward without
worrying about lack of testing.
But the issue is, as I thought I already made perfectly clear, is that they want to do a test with a release candidate to ensure the fix really works with what we intend to release, having this workaround in a release candidate makes it impossible for them to do that test.  I don't doubt they intend to fix it, but my point is they have said they want to fix this at release candidate time and not at beta time so they don't have to do multiple fixes.  If it is not broke at release candidate time, exactly how are they expected to be able to do this test?
(In reply to comment #152)
> exactly how are they expected to be able to do this test?

By setting html5.hotmailworkaround to false, as explained in comment 114:

(In reply to comment #114)
> In order to help Hotmail engineers test the behavior in the absence of this
> hack, I added a pref called html5.hotmailworkaround (which defaults to true).
Well, you see, that is the issue with bugs with this many comments.  It is too easy to miss the important ones. ;-)
We [Hotmail] have tested our fix with the current FF4.0b bits. I believe Henri tested his workaround bits with our current release. We will continue to test as new versions of the beta are published and as we continue to release on our side. 

I would expect at that point both codebases are less of a moving target for a while until Mozilla RTMs at least.
(In reply to comment #148)
> Workaround patch landed,

Thanks.

> let's hope we can back this out before release.

The patch no longer backs out without rejects/conflicts due to bug 636689 landing (thanks for landing that one, too), so I attached a reverse patch that qimports cleanly to bug 636692.

(In reply to comment #155)
> We [Hotmail] have tested our fix with the current FF4.0b bits. I believe Henri
> tested his workaround bits with our current release.

I tested with the current Hotmail release. Based on what was communicated to me about the pending Hotmail fix, I believe this workaround doesn't conflict with it, but I had no way to test in practice.

> We will continue to test
> as new versions of the beta are published and as we continue to release on our
> side. 

Thanks. As mentioned earlier, in builds that have the workaround, you can test the behavior as if the workaround didn't exist by setting html5.hotmailworkaround to false in about:config.
Whiteboard: [4b10][hardblocker][Please read comments 38, 71, 104, 114 and 115][has patch][needs decision on whether to back out] → [4b10][hardblocker][has patch][needs decision on whether to back out][backout patch in bug 636692]
Per earlier suggestions, I'm marking this bug fixed, and marking bug 636692 a final hard blocker.

Raul, I've cc'd you over there, feel free to CC whomever else needed from the Hotmail end, and thanks for the frequent updates on progress. We'll probably need to make the go/no-go decision this week.
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Good to see that Microsoft is moving to fix the Hotmail bug...

However, I have to wonder if this is a class issue in terms of fault tollerance that can be learned from this episode.  To wit:

(1) It is easy to point fingers, but when the end-user sees a FF session (apparently) fail, they will invariably blame FF, not Hotmail.  

(2) Fortunately, perhaps, the issue relates to Hotmail and we know who they are.  Microsoft can't hide!  And, those who have followed this thread will know that Raúl @ Microsoft has been quite understanding and cooperative.

(3) The problem really is with the millions of other websites where we really don't know the site webmasters and/or how to contact them.

So, it is in Mozilla's best interest to explore ways to prevent this class of failure occuring on ANY site, not just Hotmail.  

In short, is there some generalised failure class that can be extrapolated from this incident that can be implemented as a site UN-specific fix in order to improve fault tolerance to provide added "hardening" of FF and to ensure "gracefull degradation" if all else fails?
--
Good to see that Microsoft is moving to fix the Hotmail bug...

However, I have to wonder if this is a class issue in terms of fault tollerance that can be learned from this episode.  To wit:

(1) It is easy to point fingers, but when the end-user sees a FF session (apparently) fail, they will invariably blame FF, not Hotmail.  

(2) Fortunately, perhaps, the issue relates to Hotmail and we know who they are.  Microsoft can't hide!  And, those who have followed this thread will know that Raúl @ Microsoft has been quite understanding and cooperative.

(3) The problem really is with the millions of other websites where we really don't know the site webmasters and/or how to contact them.

So, it is in Mozilla's best interest to explore ways to prevent this class of failure occuring on ANY site, not just Hotmail.  

In short, is there some generalised failure class that can be extrapolated from this incident that can be implemented as a site UN-specific fix in order to improve fault tolerance to provide added "hardening" of FF and to ensure "gracefull degradation" if all else fails?
--
(In reply to comment #160 / #161)
The concept of "graceful degradation" as you put it is one of those old ideas that people used to think was good, and some still do, but is *horrible* in practice. Why? Because website designers write something then test it in browsers. If it works, the end. If the browser puts up with every broken miswritten little thing then it will appear "correct" and never get fixed. If, on the other hand, the browser doesn't tolerate things that are completely wrong and fails outright, then it'll actually get fixed. In short: failing on errors is the only way to get the errors actually fixed. Sweeping them under the rug "feels" like it makes things better, but those errors were considered errors because they were actual problems.

Case in point: This one. The old HTML parser is going away post-Firefox-4.0 at some point. If errors like this were just "gracefully degraded" automatically like the hack did for Hotmail, then any other site also with this problem or one similar won't notice it until Firefox 5, at which point it's too late and the hack isn't even doable anymore as-is.
I agree with Dave, this is the slippery slope that initially compelled me to cite my reservation for even blocking on Hotmail's exception throwing code.

Good to see that someone chimed in already, Dave Garrett, right on buddy!

We cannon and should not take into account a given developer or site's desire to bork themselves - the reality is, if they really want to put error riddled code on a URL we can't stop them.

Hopefully this kind of exception to the rule is never seen again.

Good to see MS fixed it, thank you Raul D @ Microsoft!
(In reply to comment #162)
> (In reply to comment #160 / #161)
> The concept of "graceful degradation" as you put it is one of those old ideas
> that people used to think was good, and some still do, but is *horrible* in
> practice. Why? Because website designers write something then test it in
> browsers. If it works, the end. If the browser puts up with every broken
> miswritten little thing then it will appear "correct" and never get fixed. If,
> on the other hand, the browser doesn't tolerate things that are completely
> wrong and fails outright, then it'll actually get fixed. 
> ... snip snip ...

The concept of "gracefull degradation" has been around for a long-time, but that does not make it "horrible".

Firefox is not the only game in town, and if FF crashes and fails when other browsers persevere, then the user is going to blame FF, not the website.

Terrible websites are a fact-of-life just as potholes in the road are a fact of life.  A good browser should be fault-tolerant, just as you would want your car to survive the odd unexpected pothole.

To address your concern, I encourage hardening of FF ... but with the caveat that faults are recorded ... both for on-line reporting to Mozilla (and, perhaps, offending website), but also with a pop-up warning/alert (or similar) to advise the user that the site is defective.

If FF is going to continue to grow market share it needs to out-perform other browsers and that includes (perhaps unfortunately) being website fault tolerant.  In thi case, perception is as important as technical precision.

FF should not be a website test application ... W3C already provides free tools to do that ... but it should be able to render some of the **** websites at least as well as the competitors.

By all means report, notify etc faulty websites, but at least make sure that FF is not the browser that crashes when other do not...

Peter Wilkins
--
(In reply to comment #164)
> (In reply to comment #162)
> > (In reply to comment #160 / #161)
> > The concept of "graceful degradation" as you put it is one of those old ideas
> > that people used to think was good, and some still do, but is *horrible* in
> > practice. Why? Because website designers write something then test it in
> > browsers. If it works, the end. If the browser puts up with every broken
> > miswritten little thing then it will appear "correct" and never get fixed. If,
> > on the other hand, the browser doesn't tolerate things that are completely
> > wrong and fails outright, then it'll actually get fixed. 
> > ... snip snip ...
> 
> The concept of "gracefull degradation" has been around for a long-time, but
> that does not make it "horrible".
> 
> Firefox is not the only game in town, and if FF crashes and fails when other
> browsers persevere, then the user is going to blame FF, not the website.
> 
> Terrible websites are a fact-of-life just as potholes in the road are a fact of
> life.  A good browser should be fault-tolerant, just as you would want your car
> to survive the odd unexpected pothole.
> 
> To address your concern, I encourage hardening of FF ... but with the caveat
> that faults are recorded ... both for on-line reporting to Mozilla (and,
> perhaps, offending website), but also with a pop-up warning/alert (or similar)
> to advise the user that the site is defective.
> 
> If FF is going to continue to grow market share it needs to out-perform other
> browsers and that includes (perhaps unfortunately) being website fault
> tolerant.  In thi case, perception is as important as technical precision.
> 
> FF should not be a website test application ... W3C already provides free tools
> to do that ... but it should be able to render some of the **** websites at
> least as well as the competitors.
> 
> By all means report, notify etc faulty websites, but at least make sure that FF
> is not the browser that crashes when other do not...
> 
> Peter Wilkins
> --

Please, understand that if we were to follow your advice, we would end up with a slow and bloated engine as fix after fix gets added because we are supposed to be tolerant against errors. Not to mention, that it was exactly this kind of thinking that has brought a lot of the issues we see on the web today, and has forced many non-IE browsers to reverse engineer IE-specific features. The web would end up as a broken mess where no one can rely on the standards, as these "fixes" you suggest aren't standardized.
(In reply to comment #164)
> The concept of "gracefull degradation" has been around for a long-time, but
> that does not make it "horrible".

The concept by itself is not always horrible. Often it's just problematic. When it becomes horrible is when it's applied to the Web where things are a mess. Prior HTML allowed for a lot of flexibility in this regard, which ended up with lots of subtle bugs as well as glaring ones that nobody would notice for a while. With HTML5 and into the future the rules have been tightened to resist problems. It'd be even better if XHTML and other more strict standards were adopted more widely. The less tolerant of bugs you get the more likely you are to fix them before they become a problem, as a general rule.

With Hotmail here, honestly, it's not a matter of doing anything that bad, well, except for the standard crap of serving up different sites based on user agent strings. It's just that HTML5 is new and everyone will have bugs with new stuff at first. Whilst there was much discussion here, Microsoft did in fact fix their problem before it became an issue in a stable Firefox version. (yes, some may have wished sooner, but they did fix it; if you don't like how long it took, don't use beta software or don't use Hotmail) If we were only trying to work around these problems instead, they may not have even _known_ they needed to.

> Firefox is not the only game in town, and if FF crashes and fails when other
> browsers persevere, then the user is going to blame FF, not the website.

Please don't confuse crashes with website problems. A crash is always bad and always the fault of what crashes. If Firefox crashes, file a Firefox bug with a crash report ID or stacktrace. If Flash crashes, report it to Adobe. If your OS crashes report it to your OS vendor, etc.

> Terrible websites are a fact-of-life just as potholes in the road are a fact of
> life.  A good browser should be fault-tolerant, just as you would want your car
> to survive the odd unexpected pothole.

Potholes are made by weather in an imperfect surface; websites are made by people in a non-physical environment where there are well defined rules to follow. These are two completely different universes. I find that the most frequent way people misunderstand something related to computers in general is when they try to apply rules of the physical world to something that does not exist in it with its rules. Potholes happen by nature; bugs happen by mistake. You may not be able to avoid potholes being created, but you can avoid writing a buggy website. The only way anyone can fix anything is if they know it needs fixing. A pothole you can see even if your car can handle it; a bug, if "paved over" with "graceful degradation" to continue the metaphor, you simply won't see to know it needs fixing.
(In reply to comment #165)
> (In reply to comment #164)
> > (In reply to comment #162)
> > > (In reply to comment #160 / #161)
> > > The concept of "graceful degradation" as you put it ...
> > > ... snip snip ...
> > 
> > ... snip snip ... 
> > 
> > By all means report, notify etc faulty websites, but at least make sure 
> > that FF is not the browser that crashes when other do not...
> > ... snip snip ... 
> 
> Please, understand that if we were to follow your advice, we would end up with
> a slow and bloated engine as fix after fix gets added because we are supposed
> to be tolerant against errors. Not to mention, that it was exactly this kind 
> of thinking that has brought a lot of the issues we see on the web today, and 
> has forced many non-IE browsers to reverse engineer IE-specific features. The 
> web would end up as a broken mess where no one can rely on the standards, as 
> these "fixes" you suggest aren't standardized.

For the record, I support a lean and mean browser, which is (after all) why I
am here.  Also, for the record, I am a strong proponent of web-standards
compliant web-sites.

My over-riding concern is that we should ensure that FF is not the only one to
trip-up on mal-formed websites.  We need to be not only technically superior,
but we also need to be mindful of the end-user experience.  (Note, the end-user
has absolutely no control over the websites they visit and probably little
interest on whether or not they follow web-standards.)

The acid test is if FF fails to render mangled HTML, yet other browswers can,
then we need to be mindful of the fact that the users will invariably blame FF,
rather than the website.

Bloat is the common enemy, I think we all agree on that.  Again, I am *NOT*
promoting any "fixes" ... just that we don't end-up in failure modes where FF
is blamed, when it is the website that is at fault.

Peter Wilkins
--
(In reply to comment #166)
> (In reply to comment #164)
  >>> snip snip <<<
> > Firefox is not the only game in town, and if FF crashes and fails when other
> > browsers persevere, then the user is going to blame FF, not the website.
> 
> Please don't confuse crashes with website problems. A crash is always bad and
> always the fault of what crashes. If Firefox crashes, file a Firefox bug with
> a crash report ID or stacktrace. If Flash crashes, report it to Adobe. If 
> your OS crashes report it to your OS vendor, etc.

Point taken.  
I didn't mean to type "crash" ... I meant failure to render as expected.

> > Terrible websites are a fact-of-life just as potholes in the road are a 
> > fact of life.  A good browser should be fault-tolerant, just as you would 
> > want your car to survive the odd unexpected pothole.
> 
> Potholes are made by weather in an imperfect surface; websites are made by
> people in a non-physical environment where there are well defined rules to
> follow. These are two completely different universes. 

Agreed.

I think we are on the same page here.

They key point is that we don't get blamed for website errors.  The end-user
does not really care whose fault website problems are.  If FF fails to render
when others appear to work, then FF will likely be blamed.
--
Sadly, blame gets thrown around all the time. All that can be done is try to fix the bugs we actually have control over, and accept that we can't fix the infinite reaches of the Internet and just create the best environment for those bugs to be found and fixed by those who make them.

Regardless, this debate has some merit but is now way off-topic. We don't need to discuss this anymore here and email a hundred or so people with this. ;)

With respect to this bug, we're done now, which is good. The hack was backed out in bug 636692 and the TE bug tracking the MS fix is bug 636690, also now resolved. The HTML5 parser enabling pref has also been changed to reset it for everyone who may have had to disable it for this or any other reason (bug 636689).
Whiteboard: [4b10][hardblocker][has patch][needs decision on whether to back out][backout patch in bug 636692] → [4b10][hardblocker][has patch][backout patch in bug 636692]
Peter, your concern is noted, but there's very little we can do to "not break" on a site that sniffs the UA string and sends different code to different browsers and has a bug only in code they send to Firefox.  Now can we please stop adding noise to this bug?  QA will have to read every single one of these comments to verify the fix....
You need to log in before you can comment on or make changes to this bug.