844461 - charset=iso-8859-1 parameter causes blank white page flash

Status: RESOLVED FIXED

(Core :: DOM: HTML Parser, defect)

Description

[geniusjjf]

Every page with <meta http-equiv="content-type" content="text/html; charset=iso-8859-1"> invariably rendered with the empty white flash page before rendering the actual page. Only noticeable on dark background pages.

However, every page with <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> rendered perfectly and instantly with absolutely NO white page flash.

I tested all this on my own websites and it worked every time for any size page. As long as it contained charset=utf-8 all is well. But if you code your page with a "charset=iso-8859-1 parameter, FF19 renders smoothly with NO white page prior to rendering. So it appears that the v19 core is choking on the "iso-8859-1" charset parameter and spitting out a white page before it renders the page

Comment 2

[geniusjjf]

"renders smoothly with NO white page prior to rendering" sorry, I meant to say that:

But if you code your page with a "charset=iso-8859-1 parameter, FF19 will flash an empty white page prior to rendering the actual page.

Comment 3

[Loic]

Provide a minimal testcase, please (attach it to this bug).

Comment 4

[Matthias Versen [:Matti]]

Is it possible that your server sends a utf-8 http header ?
That could cause a rendering due to a changed charset but my current knowledge is that any http header charset overrides a charset in the content.

Comment 5

[geniusjjf]

(In reply to Loic from comment #3)
> Provide a minimal testcase, please (attach it to this bug).

I don't do this for a living, my friend. It's pretty straight forward:
edit two html pages, one with each charset parameter described above, upload and test. Sorry, that's the best I can do.

Comment 6

[geniusjjf]

(In reply to Matthias Versen [:Matti] from comment #4)
> Is it possible that your server sends a utf-8 http header ?
> That could cause a rendering due to a changed charset but my current
> knowledge is that any http header charset overrides a charset in the content.

Well anything is possible. The simple fact is this: for previous FF versions, I did not experience the white-page flash on simple html pages that were loaded within a given website UNTIL FF 19 was released. Upon upgrading to said release, the browser exhibited the stated behavior.

So my question is: why would we want to introduce the mind-numbing variables of servers into this report? -- all other things being EXACTLY the same. If this was a server issue, why did all the previous versions of FF not exhibit the same behavior? It is a simple test. It worked for me every time, i.e edit two html pages, each with the stated charset parameters, upload them, clear FF19 cache and load each page within the same domain. Of course, you can't use pages with a WHITE background. It has to be a dark BG so you can see the rendering bug.

When you are loading simple html pages within the same website, there is no reason for the rendering engine to hesitate, flash a blank white page, before loading the actual page. That may be normal behavior when connecting to a new domain/server, but not within the same site.

Comment 7

[Henri Sivonen (:hsivonen)]

geniusjjf, if you navigate to about:config and search for intl.charset.detector, what's the value of the setting?

Comment 8

[Henri Sivonen (:hsivonen)]

Also, are you using a Japanese, Russian, Ukranian or Traditional Chinese localization of Firefox?

Comment 9

[Henri Sivonen (:hsivonen)]

How many bytes from the start of your HTML file until the > character of your meta tag?

In about:config, what's the value of the intl.charset.default setting?

Comment 10

[geniusjjf]

(In reply to Henri Sivonen (:hsivonen) from comment #7)
> geniusjjf, if you navigate to about:config and search for
> intl.charset.detector, what's the value of the setting?

Hello Henri, the "status" is "default" and the "value" is blank -- there is no stated value for this parameter.

Comment 11

[geniusjjf]

(In reply to Henri Sivonen (:hsivonen) from comment #8)
> Also, are you using a Japanese, Russian, Ukranian or Traditional Chinese
> localization of Firefox?

No. It is a standard English localization.

Comment 12

[geniusjjf]

(In reply to Henri Sivonen (:hsivonen) from comment #9)
> How many bytes from the start of your HTML file until the > character of
> your meta tag?
> 
> In about:config, what's the value of the intl.charset.default setting?

Not sure. It is on the 6th line, one line after the <head/>. I did try rearranging the metatags, but the result were the same. The pages with the iso charset, as described, produced the flashing white page regardless its physical position.

The setting is: intl.charset.default;ISO-8859-1.
I did not know of such a setting until now. So it apparently was set automatically during install and/or upgrades, which only asks your language preference prior to loading.

Comment 13

[Henri Sivonen (:hsivonen)]

Thanks. So your settings are at the defaults for an English build. Good. I can't think of a reason for a flash in that case. I'd need to see a test case to investigate.

Comment 14

[Masatoshi Kimura [:emk]]

The value of intl.charset.default must be a canonical name. (that is, no alias resolutions will be performed.) If the page contains a charset parameter indicating ISO-8859-1, it will be interpreted as windows-1252 since Firefox 19 because of bug 801402. ISO-8859-1 will be considered as a different encoding from windows-1252 even if the decoder is actually the same.
Henri, does this explain the flash?

Comment 15

[Henri Sivonen (:hsivonen)]

(In reply to Masatoshi Kimura [:emk] from comment #14)
> The value of intl.charset.default must be a canonical name. (that is, no
> alias resolutions will be performed.) If the page contains a charset
> parameter indicating ISO-8859-1, it will be interpreted as windows-1252
> since Firefox 19 because of bug 801402. ISO-8859-1 will be considered as a
> different encoding from windows-1252 even if the decoder is actually the
> same.
> Henri, does this explain the flash?

If the > character of the meta is not within the first 1024 bytes, yes. So we'll need to make the values read from prefs go through the alias mapping.

Comment 16

[Masatoshi Kimura [:emk]]

(In reply to Henri Sivonen (:hsivonen) from comment #15)
> we'll need to make the values read from prefs go through the alias mapping.

Or just fix bug 844776 :)

Comment 17

[Henri Sivonen (:hsivonen)]

Depends on test case changes from bug 844776.

(In reply to Masatoshi Kimura [:emk] from comment #16)
> (In reply to Henri Sivonen (:hsivonen) from comment #15)
> > we'll need to make the values read from prefs go through the alias mapping.
> 
> Or just fix bug 844776 :)

We need to take counter measures here in case the fix there doesn't propagate quickly to all localizations.

Comment 18

[Henri Sivonen (:hsivonen)]

Attachment: Don't trust the pref without alias resolution

Comment 19

[Axel Hecht [:Pike]]

Also, bug 844776 is just changing the default value per locale. that doesn't mean that profiles out there don't have legacy values of charset

Comment 20

[Henri Sivonen (:hsivonen)]

Attachment: Don't trust the pref without alias resolution, v2

Moving the test changes from bug 844776 here in order to have a single upliftable patch.

Comment 21

[Henri Sivonen (:hsivonen)]

Thank you. Landed:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cd0fa5851425

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/cd0fa5851425

Comment 23

[Henri Sivonen (:hsivonen)]

Comment on attachment 719458 [details] [diff] [review]
Don't trust the pref without alias resolution, v2

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Bug 801402.

User impact if declined: Users of Western European, North, Central and South American and Thai locales will see some pages using the most common legacy encoding of their locale reload in mid-load. Previously this happened only when viewing these pages with a mismatched localization (e.g. viewing Western European content with Korean-localized Firefox). The pages that are affected are those whose internal encoding declaration isn't completely within the first 1024 bytes of the file. Pages that declare the encoding within the first 1024 bytes, declare the encoding on the HTTP layer or don't declare the encoding are unaffected.

Testing completed (on m-c, etc.): Landed on m-c and didn't bounce. Patch rebased to beta was green on try.

Risk to taking this patch (and alternatives if risky): Low risk. The alternatives are riding the trains or editing the localization files for a large number of locales (not going to happen in practice).

String or UUID changes made by this patch: None.

Comment 24

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 719458 [details] [diff] [review]
Don't trust the pref without alias resolution, v2

Thanks for the thorough testing prior to nomination, we'll take this now for beta 3 and that gives time to verify prior to ship.

Comment 25

[Henri Sivonen (:hsivonen)]

Thanks. Landed.
https://hg.mozilla.org/releases/mozilla-aurora/rev/dab60b543514
https://hg.mozilla.org/releases/mozilla-beta/rev/c68679b48ed5

Comment 26

[Henri Sivonen (:hsivonen)]

(In reply to Lukas Blakk [:lsblakk] from comment #24)
> Thanks for the thorough testing prior to nomination, we'll take this now for
> beta 3 and that gives time to verify prior to ship.

I now realize beta 3 build 1 was tagged before his landed. :-(

Comment 27

[Henri Sivonen (:hsivonen)]

And landed a fix for faulty rebasing that only went red on Windows (the green try run was Linux64-only).
https://hg.mozilla.org/releases/mozilla-beta/rev/32e6fabc2667

Comment 28

[u279076]

Ioana, please make sure this gets verified as part of your next verification cycle.

Comment 29

[Ioana (away)]

Mozilla/5.0 (Windows NT 5.1; rv:20.0) Gecko/20100101 Firefox/20.0 (20130307075451)

I tried to reproduce this issue with the following testcase:
<!DOCTYPE html>
<html>
<head>
  <title>Bug 844461</title>
--paragraph with ~1200 lines of text
  <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
</head>
<body style="background:blue">
--paragraph with more than 1000 lines of text
</body>
</html>

I don't get any flashes while loading the page initially, but I do get some flashes when reloading it multiple times. The difference between Firefox 19.0 and 20b4 is that I get much more flashes in 19.0 than in beta.

I respected the hints in the above comments when creating the testcase but, since the issue reproduces intermittently on both builds, I suppose there might be something wrong there. Can anyone let me know what I should modify/add/remove?

Comment 30

[Henri Sivonen (:hsivonen)]

(In reply to Ioana Budnar [QA] from comment #29)
>   <title>Bug 844461</title>
> --paragraph with ~1200 lines of text
>   <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">

That should put the meta beyond the first 1024 bytes, so it should trigger the condition that matters here.

> Can anyone let me know what I should
> modify/add/remove?

I can't. I fixed this by reasoning what was wrong--not by trying to reproduce the reported problem itself.

Comment 31

[u279076]

(In reply to Henri Sivonen (:hsivonen) from comment #30)
> I can't. I fixed this by reasoning what was wrong--not by trying to
> reproduce the reported problem itself.

Is there any better advice you can give us for testing this is fixed? Is the in-testsuite coverage sufficient to call this verified?

Comment 32

[Henri Sivonen (:hsivonen)]

(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #31)
> (In reply to Henri Sivonen (:hsivonen) from comment #30)
> > I can't. I fixed this by reasoning what was wrong--not by trying to
> > reproduce the reported problem itself.
> 
> Is there any better advice you can give us for testing this is fixed?

Not really. Sorry. Overwhelming the main thread with other work to make things go slowly might help.

> Is the
> in-testsuite coverage sufficient to call this verified?

The test suite coverage is sufficient for testing that the patch does what it tries to do in terms of pref handling, but the test suite doesn't attempt to test the reported symptom.

Comment 33

[u279076]

Ioana, can you try retesting this by trying to simulate a high load condition? Failing that I think we'll just have to trust this is fixed and keep an eye out for post-release feedback mentioning this symptom.

Comment 34

[Ioana (away)]

(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #33)
> Ioana, can you try retesting this by trying to simulate a high load
> condition? Failing that I think we'll just have to trust this is fixed and
> keep an eye out for post-release feedback mentioning this symptom.

I tried to reproduce this issue on Firefox 19.0.2 and Firefox 20.0b6, with 156 tabs open and with 177 tabs open (videos playing, games, simple pages, rich content pages etc). I got the same results for both tab counts:
- Fx 19.0.2: No flash of blank pages, just a flash of white background (all the page content is displayed, but on a white background. After a second it displays the dark blue background set in the page source).
- Fx 20.0b6: No flashes of any kind.

Unfortunately, I can't go further since my station completely freezes once I add more rich content, especially plugin one. I also doubt too many users have more tabs open and completetly loaded.

Comment 35

[u279076]

Thank you Ioana. I think you tried your best. I'm tagging this as [qa-] since QA is not able to verify this completely.