197520 - XHTML entity parsing is broken

Status: VERIFIED FIXED

(SeaMonkey :: General, defect, P1)

Description

[Frederic Bezies]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4a) Gecko/20030315
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4a) Gecko/20030315

Simple. About:mozilla is not working anymore.

I got instead :

"XML Parsing Error: undefined entity
Location: jar:resource:///chrome/en-US.jar!/locale/en-US/global/about.xhtml
Line Number 95, Column 15:<li>Copyright &copy; 1998-2003 by <a
href="about:credits">Contributors</a> to 
--------------^"

See details for more info.

This is not a duplicate of bug 197477 because I am using a trunk build, not a
1.3 branch build.

And of course, I made a clean install of my newly homemade build.

Reproducible: Always

Steps to Reproduce:
1.Grab sources
2.apply patches with CVS
3.build mozilla and choose help/about after you installed your new lizard

Actual Results:  
Error message !

Expected Results:  
Displaying infos :-)

My about:buildconfig

Build platform
target
i686-pc-cygwin

Build tools
Compiler 	Version 	Compiler flags
cl 	12.00.8804 for 80x86 	-TC -nologo -W3 -nologo -Gy -Fd$(PDBFILE)
cl 	12.00.8804 for 80x86 	-TP -nologo -W3 -nologo -Gy -Fd$(PDBFILE)

Configure arguments
--enable-extensions --enable-crypto --disable-debug --enable-optimize
--enable-calendar --disable-pedantic --disable-installer --enable-strip
--disable-tests

If this bug related to bug 194240 big patch ?

Comment 1

[Frederic Bezies]

Modifying URL.

All others about: seems to work.

Comment 2

[Boris Zbarsky [:bzbarsky]]

This is using the English UI?  Or the French localization?

Comment 3

[Frederic Bezies]

As I am using homemade builds, I am using english UI only.

No other xpi.

I deleted chrome and cache from my profile. Build was installed using a clean
install : deleting everything (keeping plugins directory).

Comment 4

[Boris Zbarsky [:bzbarsky]]

Is this a problem with a non-homemade build?  One from the mozilla.org ftp site?

Comment 5

[Frederic Bezies]

I am not using official build. I think I have an "half checking" problem. I will
grab all patches I don't grab, build it again and see if the problem is still alive.

Comment 6

[Stephen Walker]

Parsing entities seems to be broken for most (all?) xhtml files with the latest
m.o trunk build.

Add user_pref("browser.xul.error_pages.enabled", true); to your user.js or
prefs.js and then load http://www.blahblahblah.com/.

Comment 7

[Frederic Bezies]

Thanks ! I was not mad so !

Is bug 194240 the big breaker, here ?

Comment 8

[Stephen Walker]

dougt, can you look at this?

Comment 9

[Doug Turner (:dougt)]

Using this morning's windows and linux builds, i find no problem loading
about:mozilla.  wfm.

Comment 10

[Stephen Walker]

dougt, use about: or the steps I listed in comment 6. about:mozilla doesn't use
any entities (even though that's what Frederic listed in comment 0).

Comment 11

[Doug Turner (:dougt)]

Okay - i can see the problem using that prefs.js setting.

I landed the branch yesterday afternoon.  I don't think that my landing could
have caused this.

cc'ing alec.  he may have landed some jar changes which could have caused this?

Comment 12

[Frederic Bezies]

For my comment 0 :-)

I mean about:, not about:mozilla ;-]

I should have said help/about mozilla :-)

Comment 13

[Frederic Bezies]

Sorry for spamming, but bug is shown using build 2003031521.

Comment 14

[Josh Birnbaum]

*** Bug 197669 has been marked as a duplicate of this bug. ***

Comment 15

[Jo Hermans]

*** Bug 197665 has been marked as a duplicate of this bug. ***

Comment 16

[Gilles Durys]

*** Bug 197701 has been marked as a duplicate of this bug. ***

Comment 17

[Josh Birnbaum]

*** Bug 197712 has been marked as a duplicate of this bug. ***

Comment 18

[R.K.Aa.]

*** Bug 197738 has been marked as a duplicate of this bug. ***

Comment 19

[Boris Zbarsky [:bzbarsky]]

Do we have a regression date range here?  Something like 1 day or less?

Comment 20

[R.K.Aa.]

*** Bug 197757 has been marked as a duplicate of this bug. ***

Comment 21

[Masayasu Ishikawa]

I tried mozilla-win32-svg-GDI-mathml (Build ID: 2003031610) and encountered the
same problem. 1.3 on Windows XP / MacOS X and Windows nightly build (ID:
2003031408) worked without problem.

This error occurs when an XHTML document is served as application/xhtml+xml,
application/xml or text/xml.  When it's served as text/html, this error doesn't
occur.  Try the following tests:

http://www.w3.org/People/mimasa/test/xhtml/entities/

Comment 22

[Boris Zbarsky [:bzbarsky]]

Yes, yes.  text/html is NOT XHTML.  We know it's already broken by the 16th...

I'm working on narrowing the date range, now that I have local builds again...

Comment 23

[Boris Zbarsky [:bzbarsky]]

And if you're not on the list at
http://www.mozilla.org/roadmap.html#project-management please do not set any
"blocking a release" flags.  Thank you.

Comment 24

[Boris Zbarsky [:bzbarsky]]

OK, I just tested this in the following builds:

Linux .tar.gz: 2003-03-15-05, 2003-03-15-21, 2003-03-16-05
Linux installer: 2003-03-16-05

None of these shows the problem. Since all the duplicates are on Windows, I'm
assuming it's a Win32-specific problem.  So could someone who has Windows please
determine the regression date range?

Frederic, when did you pull the build that shows the problem?  What was the time
before that when you pulled?

Comment 25

[timeless]

w32 2003031504 not talkback, broken
w32 2003031408     talkback, worked

Comment 26

[Erik]

The last build which WFM was 2003-03-14-08.
The first build which shows the error is the 2003-03-15-04 using Windows nt4.0 

Comment 27

[Boris Zbarsky [:bzbarsky]]

Alright.... that's what timeless says too; he tested zip builds, so it's not an
installer problem.

to dougt, as the most likely culprit in that date range.....

Specifically, we may have a place where we switched from nsFileSpec to nsIFile
and the two parse some string differently on windows, with nsIFile not coming
out with a useful filename and hence not loading a DTD.  That's my current
hypothesis on what's going on.

I looked at the nsScanner changes and those look fine (not to mention that that
particular constructor does not seem to be called). Heikki, where would one look
to see the DTD-loading code?

Comment 28

[Frederic Bezies]

In answer to comment #24 : Bug appears soon after minimo landing. I am living in
Europe, so Mozilla.org time + 9 hours.

Well, last working homemade build : 14 march, just before minimo landing.

Should this be considered as a 1.4a potentially blocker ? (don't know if I can
set "?" for 1.4a blocker).

Comment 29

[Christopher Hoess (gone)]

*** Bug 197805 has been marked as a duplicate of this bug. ***

Comment 30

[Doug Turner (:dougt)]

Hi.  I landed on the 15th at around 4pm.  If the regression was in the 15th's
nightly build, then the regression wasn't caused by my landing, correct? 

Comment 31

[Doug Turner (:dougt)]

okay, Frederic confirms that it is my bustage. I will take a look.

Comment 32

[timeless]

fwiw bug 175394 is about mac classic having this sort of problem

Comment 33

[Boris Zbarsky [:bzbarsky]]

Doug, just for the record you landed on the _14th_ around 5pm.  According to
bonsai, at least.

Comment 34

[Boris Zbarsky [:bzbarsky]]

And yes, anyone can request blocker status..

Comment 35

[(not reading, please use [e-mail] instead)]

confirming that I see this on win32, debug.

I'll start debugging.

Comment 36

[Heikki Toivonen (remove -bugzilla when emailing directly)]

XHTML entity loading happens here:

http://lxr.mozilla.org/seamonkey/source/htmlparser/src/nsExpatDriver.cpp#188

Those of you that have a recent enough build to see this, could you please take
a look at what is going on?

Comment 37

[(not reading, please use [e-mail] instead)]

heikki, I'll look at what's going on in nsExpatDriver.cpp

here's a stack to the code that generates the error, might be useful.

CreateErrorText(const unsigned short * 0x0012f904, const unsigned short *
0x03a08ce8, const int 95, const int 15, nsString & {...}) line 719
nsExpatDriver::HandleError(const char * 0x03a04bd0, unsigned int 7314, int 0)
line 787 + 67 bytes
nsExpatDriver::ParseBuffer(const char * 0x03a04bd0, unsigned int 7314, int 0)
line 817
nsExpatDriver::ConsumeToken(nsExpatDriver * const 0x039fa65c, nsScanner & {...},
int & 0) line 918 + 32 bytes
nsParser::Tokenize(int 0) line 2543 + 26 bytes
nsParser::ResumeParse(int 1, int 0, int 1) line 1770 + 31 bytes
nsParser::OnDataAvailable(nsParser * const 0x0368166c, nsIRequest * 0x039dd258,
nsISupports * 0x00000000, nsIInputStream * 0x039dd894, unsigned int 0, unsigned
int 3657) line 2405 + 21 bytes
nsDocumentOpenInfo::OnDataAvailable(nsDocumentOpenInfo * const 0x039dce68,
nsIRequest * 0x039dd258, nsISupports * 0x00000000, nsIInputStream * 0x039dd894,
unsigned int 0, unsigned int 3657) line 243 + 46 bytes
nsJARChannel::OnDataAvailable(nsJARChannel * const 0x039dd260, nsIRequest *
0x039dd788, nsISupports * 0x00000000, nsIInputStream * 0x039dd894, unsigned int
0, unsigned int 3657) line 677 + 57 bytes
nsInputStreamPump::OnStateTransfer() line 406 + 65 bytes
nsInputStreamPump::OnInputStreamReady(nsInputStreamPump * const 0x039dd78c,
nsIAsyncInputStream * 0x039dd894) line 321 + 11 bytes
nsInputStreamReadyEvent::EventHandler(PLEvent * 0x0368f114) line 112
PL_HandleEvent(PLEvent * 0x0368f114) line 663 + 10 bytes
PL_ProcessPendingEvents(PLEventQueue * 0x00f47f08) line 593 + 9 bytes
_md_TimerProc(HWND__ * 0x0009028a, unsigned int 275, unsigned int 0, unsigned
long 2500515) line 963 + 9 bytes
USER32! 77e3a290()
USER32! 77e143da()
USER32! 77e1a752()
nsAppShellService::Run(nsAppShellService * const 0x00fca1c8) line 480
main1(int 1, char * * 0x00271ce0, nsISupports * 0x00f0ce90) line 1268 + 32 bytes
main(int 1, char * * 0x00271ce0) line 1639 + 37 bytes
mainCRTStartup() line 338 + 17 bytes
KERNEL32! 77e9ca90()

Comment 38

[(not reading, please use [e-mail] instead)]

© is a special internal entity for the copyright symbol, right?

I'm new to this code, but it isn't going to find it in an external dtd.

Comment 39

[Doug Turner (:dougt)]

*** Bug 175394 has been marked as a duplicate of this bug. ***

Comment 40

[Boris Zbarsky [:bzbarsky]]

© should be coming from xhtml11.dtd

Comment 41

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Looking at the nsExpatDriver.cpp changes, 

lfile->AppendRelativeNativePath(PromiseFlatCString(nsDependentCString(kDTDDirectory)
+ fileName));

is appending a path with a forward-slash in it.  Is that supposed to work on
Windows?

Comment 42

[Doug Turner (:dougt)]

found the bug.  nsIFile::AppendRelative does not as documented.  patch coming up.

Comment 43

[(not reading, please use [e-mail] instead)]

Attachment: a fix, based on dbaron's comment

Comment 44

[Doug Turner (:dougt)]

Attachment: loops on the path

Comment 45

[Josh Birnbaum]

*** Bug 197870 has been marked as a duplicate of this bug. ***

Comment 46

[Doug Turner (:dougt)]

Attachment: final version

Comment 47

[Doug Turner (:dougt)]

Checking in nsExpatDriver.cpp;
/cvsroot/mozilla/htmlparser/src/nsExpatDriver.cpp,v  <--  nsExpatDriver.cpp
new revision: 3.36; previous revision: 3.35
done


This is fixed.

Comment 48

[harishd]

Comment on attachment 117501 [details] [diff] [review]
final version

>? gkparser.dll
>? gkparser.exp
>? gkparser.ilk
>? gkparser.lib
>? gkparser.pdb
>? module.rc
>? module.res
>? x
>? xx
>Index: nsExpatDriver.cpp
>===================================================================
>RCS file: /cvsroot/mozilla/htmlparser/src/nsExpatDriver.cpp,v
>retrieving revision 3.35
>diff -u -1 -0 -r3.35 nsExpatDriver.cpp
>--- nsExpatDriver.cpp	15 Mar 2003 01:02:47 -0000	3.35
>+++ nsExpatDriver.cpp	17 Mar 2003 20:09:42 -0000
>@@ -46,21 +46,20 @@
> #include "nsIURL.h"
> #include "nsIUnicharInputStream.h"
> #include "nsNetUtil.h"
> #include "prprf.h"
> #include "prmem.h"
> #include "nsTextFormatter.h"
> #include "nsDirectoryServiceDefs.h"
> #include "nsCRT.h"
> 
> static const char* kWhitespace = " \r\n\t"; // Optimized for typical cases
>-static const char kDTDDirectory[] = "res/dtd/";
> 
> /***************************** EXPAT CALL BACKS *******************************/
> 
>  // The callback handlers that get called from the expat parser 
> PR_STATIC_CALLBACK(int)
> Driver_HandleStartElement(void *aUserData, 
>                           const XML_Char *aName, 
>                           const XML_Char **aAtts) 
> {
>   NS_ASSERTION(aUserData, "expat driver should exist");
>@@ -271,23 +270,26 @@
>   
>   nsCOMPtr<nsIFile> dtdPath;
>   NS_GetSpecialDirectory(NS_OS_CURRENT_PROCESS_DIR, 
>                          getter_AddRefs(dtdPath));
> 
>   if (!dtdPath)
>     return PR_FALSE;
> 
>   nsCOMPtr<nsILocalFile> lfile = do_QueryInterface(dtdPath);
> 
>-  nsCAutoString path;
>-  path = NS_LITERAL_CSTRING(kDTDDirectory) + fileName;
>-  lfile->AppendRelativeNativePath(path);
>+  // append res/dtd/<fileName>
>+  // can't do AppendRelativeNativePath("res/dtd/" + )
>+  // as that won't work on all platforms.
>+  lfile->AppendNative(NS_LITERAL_CSTRING("res"));
>+  lfile->AppendNative(NS_LITERAL_CSTRING("dtd"));
>+  lfile->AppendNative(fileName);
> 
>   PRBool exists;
>   dtdPath->Exists(&exists);
> 
>   if (exists) {
>     // The DTD was found in the local DTD directory.
>     // Set aDTD to a file: url pointing to the local DT
>     nsCOMPtr<nsIURI> dtdURI;
>     NS_NewFileURI(getter_AddRefs(dtdURI), dtdPath); 
>

Comment 49

[harishd]

Please ignore comment #48. That was an accident :-(

Comment 50

[Frederic Bezies]

I can see about: back now.

Thanks for fixing this bug.

Comment 51

[Bill Mason]

Verified 2003031804 PC/WinXP

Comment 52

[Peter Van der Beken [:peterv]]

*** Bug 197477 has been marked as a duplicate of this bug. ***

Comment 53

[Sébastien Delahaye]

*** Bug 198074 has been marked as a duplicate of this bug. ***

Comment 54

[Greg K.]

*** Bug 182204 has been marked as a duplicate of this bug. ***

Comment 55

[Jim Booth]

This appears to have regressed.  

In 2003091704 (Windows ME) the Help/About Mozilla page displays:

XML Parsing Error: undefined entity Location:
jar:resource:///chrome/en-US.jar!/locale/en-US/global/about.xhtml Line Number
95, Column 15:

<li>Copyright &copy; 1998-2003 by <a href="about:credits">Contributors</a> to
--------------^

(end of page display)

Comment 56

[Christian :Biesinger (don't email me, ping me on IRC)]

Jim: you are seeing bug 219355, not this bug.

Comment 57

[Matthias Versen [:Matti]]

verified

Please don't reopen such old VERIFIED fixed bugs