229024 - Source code truncated on some files

Status: RESOLVED FIXED

(Other Applications Graveyard :: Venkman JS Debugger, defect, P1)

Description

[Morgan Doocy]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6b) Gecko/20031208
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6b) Gecko/20031208

The 'Source Code' pane truncates some Javascript source files. In the example of
the page <http://devedge.netscape.com/>, line 76 of the included script
<theme-menu-2.js> is the last line shown, and it is truncated:

75|   document.documentElement.style.fontSize = newSize;
76|   documen

The file contains 415 lines of code.

When stepping through the script, the debugger still evaluates each line in the
file, but any lines past the point of truncation are not shown in the 'Source
Code' pane.

Another sample:
<http://bugzilla.mozilla.org/enter_bug.cgi?product=Browser&format=guided>

Last lines in source file <quicksearch.js>:
48|   function go_to (url) {
49|       if ( typeof sidebar 


Reproducible: Sometimes

Steps to Reproduce:
1. Launch debugger
2. Load/reload a page in a browser window
3. Double-click a Javascript source file in the 'Loaded Scripts' pane
4. Scroll to bottom of file in 'Source Code' pane
Actual Results:  
Source is truncated.

Expected Results:  
Displayed entire contents of source file.

Comment 1

[Robert Ginda]

*** Bug 229771 has been marked as a duplicate of this bug. ***

Comment 2

[jsp]

I've been seeing this (unpredictably) for some time.  I'm currently running
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6b) Gecko/20031208, but the
problem predates it.  Sometimes reloading the source helps, but more often it
doesn't.  Nor does clearing the cache and restarting the browser.  I can
reliably reproduce the problem with the URLs specified (today, at least), though
I get a few more characters on the final line.

Confirming.

Comment 3

[jsp]

I tested this with Moz 1.3/Venkman 0.9.48, Moz 1.4/Venkman 0.9.75, and Moz
1.5/Venkman 0.9.79.  All displayed the full source code in the two example URLs.
 This problem would appear to have been introduced by some change in Mozilla
since 1.5.

Comment 4

[Robert Ginda]

Darin, did something change in async url loading after 1.5?  The Venkman code
that loads urls is in
http://lxr.mozilla.org/mozilla/source/extensions/venkman/resources/content/venkman-url-loader.js

We had a problem like this before, where the web server broke the response up
into multiple blocks and venkman only loaded the first block.  I forget the
exact fix, but I think it had something to do with the fact that we were reusing
the scriptable input stream.

Comment 5

[Darin Fisher]

yes, some async stream code did change during 1.6, so it is possible that that
could have caused the problem you are seeing.

if someone can capture a HTTP log (see
http://www.mozilla.org/projects/netlib/http/http-debugging.html) while the
abnormal truncation occurs that would be great.  please attach it to this bug
report.

thx!

Comment 6

[Antoine Levy-Lambert]

Attachment: log of http request

this is the log file obtained for the http request for a page with a https://
url.
Only the 5 odd first lines of this page display in the venkman debugger, which
happens to know though that the page contains a java script at line 1260 or
something like that.
Cheers,
Antoine

Comment 7

[Darin Fisher]

antoine: thanks for the log file.  can you tell me which URL appears to be
partially loaded / truncated ??

thx!

Comment 8

[Antoine Levy-Lambert]

this URL https://cbs.wdf.sap.corp/pdb/mj/configurations.php appears truncated

Comment 9

[Antoine Levy-Lambert]

I have found a URL where this problem of truncation occurs :
http://nagoya.apache.org/bugzilla/query.cgi
Since this is really a public domain link, you can certainly gladly use it to
debug the failing parts.
Cheers,

Antoine

Comment 10

[Robert Ginda]

*** Bug 233006 has been marked as a duplicate of this bug. ***

Comment 11

[chris hofmann]

http://nagoya.apache.org/bugzilla/query.cgi seems to load fine for me.  does
connection speed or server loading, or anything else play a part in this?

Comment 12

[Robert Ginda]

no, this is a widely reported mozilla regression from 1.6.  It's likely to be
intermittent, though, and so it doesn't surprise me it doesn't repro exactly for
everyone.

Comment 13

[Darin Fisher]

accepting for 1.7b

Comment 14

[Robert Ginda]

err, that is, yes.  connection speed or server loading may play a part in this
bug.  my guess is that if necko has to deliver the file in more than one chunk,
for any reason, then we only see the first chunk.

We had this problem once before when we were reusing a scriptableInputStream
between responses, but we don't do that anymore.

Comment 15

[Darin Fisher]

-> 1.7b

Comment 16

[J. Akiyama]

I am also seeing this bug as well.

When trying to look through the source code of Javascript scripts using Venkman
0.9.79 (launched with Firefox 0.80), the source code gets truncated.  

For example, going to http://www.vbulletin.com/forum/ and opening up
vbulletin_global.js will show that it should have over 568 lines of code. 
However, double-clicking on the last function of vbulletin_global.js
(vBulletin_init) will only show up to 119 lines in the Source Code pane.

I have tried this out on two different installations of Firefox 0.80 and Venkman
0.9.79 running on two different Windows 2000 computers (one at home and one at
work so two completely different networks).

Comment 17

[Paulo Rosa]

This might help speeding up bug tracking a bit.

I'm using Mozilla 1.6 and Venkman 0.9.80. I have this script file, served by a
local apache server, that is truncated in the very beggining (line 107 of 1100+). 

The file's named 'htmlarea.js' and is retrievable through the following URL:
http://www.interactivetools.com/products/htmlarea/

It's version 3.0.

Regards,
Paulo Rosa

Comment 18

[Darin Fisher]

(In reply to comment #11)
> http://nagoya.apache.org/bugzilla/query.cgi seems to load fine for me.  does
> connection speed or server loading, or anything else play a part in this?

i'm able to repro the problem on this site, but the corresponding HTTP log shows
necko loading the URL completely without any errors or cancelation.  i cannot
explain the visible truncation from the necko HTTP log alone.

Comment 19

[Darin Fisher]

actually, i suspect this may be a regression caused by the fact that all necko
channel's (the HTTP one in particular) now support the synchronous
nsIChannel::open method (see bug 192284).

Comment 20

[Darin Fisher]

the problem i suspect is caused by this incorrect code:

function loadURLNow (url)
{
    var chan = _getChannelForURL (url);
    chan.loadFlags |= nsIRequest.LOAD_BYPASS_CACHE;
    var instream =
        Components.classes[SIS_CTRID].createInstance(nsIScriptableInputStream);
    instream.init (chan.open());

    return instream.read (instream.available());
}

this code assumes that the amount of data available is equal to the total length
of the data stream.  this code needs a loop to consume data until EOF on the stream.

however, that might not be sufficient to fix this problem since nsIChannel::open
and reading from the resulting stream can block the UI thread.  moreover, it
spins an event queue to allow UI events to continue being processed -- this can
cause the venkman code to be re-entered in potentially wierd ways.  like maybe
the venkman code would behave badly if re-entered while calling loadURLNow.

the best solution might be to make loadURLNow throw a dummy exception when
passed a HTTP or HTTPS URL.  this solution is not ideal.  what about FTP?  or
other remote protocols?  hmm...

Comment 21

[Darin Fisher]

Attachment: v1 patch

This patch makes it work, and it is needed for correctness.  However, I'm not
sure that it is sufficient.  Should we worry about the re-entrancy problem I
outlined in my previous comment?

Comment 22

[Robert Ginda]

> the best solution might be to make loadURLNow throw a dummy exception when
> passed a HTTP or HTTPS URL.  this solution is not ideal.  what about FTP?  or
> other remote protocols?  hmm...

Actually, I think necko used to do exactly that, and I relied on it.  IIRC,
necko used to throw an exception somewhere along the way (maybe the available()
call?) if the url could not be loaded in a single shot.  Higher up the codepath,
in
http://lxr.mozilla.org/mozilla/source/extensions/venkman/resources/content/venkman-static.js#1186,
if loadURLNow fails, we loadURLAsync instead.  Perhaps venkman should just stick
with loadURLAsync.

Comment 23

[Darin Fisher]

(In reply to comment #22)
> > the best solution might be to make loadURLNow throw a dummy exception when
> > passed a HTTP or HTTPS URL.  this solution is not ideal.  what about FTP?  or
> > other remote protocols?  hmm...
> 
> Actually, I think necko used to do exactly that, and I relied on it.  IIRC,
> necko used to throw an exception somewhere along the way (maybe the available()
> call?) if the url could not be loaded in a single shot.  Higher up the codepath,
> in
>
http://lxr.mozilla.org/mozilla/source/extensions/venkman/resources/content/venkman-static.js#1186,
> if loadURLNow fails, we loadURLAsync instead.  Perhaps venkman should just stick
> with loadURLAsync.

right, some channels... namely, the nsHttpChannel would throw a "not
implemented" exception from its .open method.

if you can call loadURLAsync, then you should almost always do that.  i figured
you must have wanted to synchronously load certain URL types for some reason.

NOTE: the patch i've attached is still necessary b/c the existing code in
loadURLNow is not right, and there are places where you don't failover from sync
load to async load.

Comment 24

[Greg K.]

*** Bug 234677 has been marked as a duplicate of this bug. ***

Comment 25

[Robert Ginda]

*** Bug 236166 has been marked as a duplicate of this bug. ***

Comment 26

[Asa Dotzler [:asa]]

Would be nice to get this but not going to block beta for it. If this is
reviewed and ready to go during the freeze, please request driver approval to land.

Comment 27

[Rick Mangi]

(In reply to comment #26)
> Would be nice to get this but not going to block beta for it. If this is
> reviewed and ready to go during the freeze, please request driver approval to
land.

Disagree. This in my mind this is a show stopper. Then again, I'm just a user :-)

Comment 28

[Antoine Levy-Lambert]

I also believe this is a show stopper.
By the way, when one chooses the menu option "View Source" in a browser, the
contents of the pages including the JavaScript is  not truncated.

Comment 29

[bht237]

I can only download any new beta when this is fixed. Currently I have gone back
from 1.6 to 1.4 because this bug does not exist in 1.4. However it might be
acceptable to for the affected users to not test the new beta and go back to 1.4
- or not. I don't care.

Comment 30

[seth]

This bugs me, too, but you have to remember that most of their users are not web
developers and so problems with venkman are going to have a lower priority. It's
extremely important to some of us, but utterly inconsequential to most.

If every bug like that (very important to a few, meaningless to most) was
allowed to be a blocker, then they'd never be able to ship *any* bugs!

Comment 31

[Robert Ginda]

Attachment: darin's patch, plus remove all but one loadURLNow callsite

Comment 32

[Darin Fisher]

Comment on attachment 143453 [details] [diff] [review]
darin's patch, plus remove all but one loadURLNow callsite

in one case you catch errors from loadURLAsync... in the other case you don't. 
does that matter?

r=darin

Comment 33

[jsp]

rginda, would you be willing to put up a new Venkman XPI with this patch at
http://www.hacksrus.com/~ginda/venkman/?  This would:

1) allow interested parties (like me) to test it,
2) allow Mozilla 1.6 users to get a working version of Venkman (I'm still using
1.5 because of this problem), and
3) make the question of whether the patch gets into 1.7 less pressing, since it
wouldn't be hard to get a working version.

Comment 34

[chris hofmann]

Comment on attachment 143453 [details] [diff] [review]
darin's patch, plus remove all but one loadURLNow callsite

a=chofmann for 1.7b

Comment 35

[Darin Fisher]

fixed on trunk for 1.7 beta

Comment 36

[Robert Ginda]

Thanks for the help darin!

Comment 37

[Robert Ginda]

*** Bug 237643 has been marked as a duplicate of this bug. ***

Comment 38

[Frank Wales]

(In reply to comment #30)  
> If every bug like that (very important to a few, meaningless to most) was  
> allowed to be a blocker, then they'd never be able to ship *any* bugs!  
  
A dissenting opinion on this: bugs that impede developers can potentially 
affect many ordinary users, so shouldn't be lumped in with bugs in 
non-development features deemed to affect only a few users. 
 
For example, this very bug has made it significantly harder to finish new 
Mozilla-based user interface code for a client, which in turn is forcing them 
to roll out a less-capable version of an application for the time being.  So a 
bug that ostensibly inconveniences a couple of developers has deleterious 
knock-on effects for hundreds of Mozilla users, precisely because it's a bug 
only developers would be affected by. 

Comment 39

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 265190 has been marked as a duplicate of this bug. ***