Closed Bug 81531 Opened 23 years ago Closed 23 years ago

page source takes long time to load the search result from this page

Categories

(Core :: Layout, defect)

Product:
Core
Component:
Layout
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
VERIFIED DUPLICATE of bug 55583

People

(Reporter: dave.close, Assigned: karnaze)

References

(
URL
)

Details

(Keywords: crash, hang, perf) 

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9) Gecko/20010505
BuildID:    2001050515 (0.9)

Attempting to view page source seems to lock-up browser.

Reproducible: Always
Steps to Reproduce:
1. Go to this page, enter "AT command" in search box, search.
2. On result screen, try to "view page source". Source appears, at least
partially, but hourglass never goes away. Browser appears dead and must be killed.
3. http://www2.elsa.de/Internet/Elsafilearea.nsf/$$search?openform&lang=en
Confirm with build 2001-05-16-04 using Win2k-SP1

The taskman shows mozilla using 100% of the CPU. Mem usage was at 55MB and
increasing fast. The page is 25KB long.

The browser or source window were not really hung, just very unresponsive.
My system is a Celeron400 with 320MB of RAM and a Voodoo Banshee.
Status: UNCONFIRMED → NEW
Ever confirmed: true
seeing this on linux cvs too.
Severity: normal → critical
Keywords: hang
OS: Windows NT → All
updating component and setting default owners.
Assignee: asa → blakeross
Component: Browser-General → XP Apps: GUI Features
QA Contact: doronr → sairuh
using 2001.05.18.13 comm verif bits on linux and winnt, i don't get a hang.
however, it seems to take forever for the source to load --and after a couple of
minutes i just hit ctrl+W [or clicked the close widget]. the source window did
[finally] close, and i was returned to my browser as before.

however, when i tried this on the mac, i got a crash in
SinkContext::OpenContainer. i'll search and see if that's reported elsewhere,
though...
Keywords: hang → crash, perf
Hardware: PC → All
Summary: Can't open page source → page source takes "forever" to load the search result from this page
i'm gonna see if actually waiting longer for the source to load will crash... in
the meantime, here's the talkback info for the mac crash i mentioned above:

Incident ID 30635614 
 Trigger Time               2001-05-18 16:23:49
 Platform ID                MacOS 
 Stack Trace
0x80011960 
SinkContext::OpenContainer() [nsHTMLContentSink.cpp, line 1276] 
HTMLContentSink::OpenContainer() [nsHTMLContentSink.cpp, line 3081] 
CViewSourceHTML::WriteTag() [nsViewSourceHTML.cpp, line 926] 
CViewSourceHTML::WriteAttributes() [nsViewSourceHTML.cpp, line 873] 
CViewSourceHTML::WriteTag() [nsViewSourceHTML.cpp, line 962] 
waiting a nice long time, and the source did complete loading on winnt
[2001.05.18.13] and linux [using a 13:00-ish cvs build]. for kicks i scrolled to
the bottom of the source page, which worked fine, and then closed the window via
ctrl+W which again was fine. then i quit the app. on winnt, quitting went fine,
but on my linux debug, it crashed --not sure if that's related here, though:

#0  0x400eee6b in nsThreadPool::GetRequest (this=0x8157dd8, 
    currentThread=0x42369d28) at nsThread.cpp:600
#1  0x400eff58 in nsThreadPoolRunnable::Run (this=0x42369d10)
    at nsThread.cpp:833
#2  0x400ed8e3 in nsThread::Main (arg=0x42369d28) at nsThread.cpp:105
#3  0x40230e10 in PR_Select () from /builds/sairuh/mozilla/dist/bin/libnspr4.so
#4  0x4024cb85 in pthread_start_thread (arg=0xbddffe40) at manager.c:241

looking at my mac now: the watch cursor is still there. cannot scroll... hit
cmd+W, wait a couple minutes: no response. click the close widget, wait a few
minutes: no response... *sigh* jump into Macsbug and hit 'es'... but no crash.

anyhow, not sure who should get this bug. perhaps the layout group?
Assignee: blakeross → karnaze
Component: XP Apps: GUI Features → Layout
QA Contact: sairuh → petersen
Summary: page source takes "forever" to load the search result from this page → page source takes long time to load the search result from this page
Reassigning to harishd based on 2001-05-18 16:35 comments.
Assignee: karnaze → harishd
The problem could be related to view source coloring.
Status: NEW → ASSIGNED
JPatel - Pls look through Talkback data and see if there is anything there, per PDT.
Jpatel - Per PDT, pls look through stack to see if there is other instances of
this crash in the data base.
There isn't much data for this crash in Talkback...I queried for the following:

- "0x80011960", the stack signature sairuh submitted after crashing on mac and 
her entry was the only one that came up.

- "nsThreadPool::GetRequest", the stack signature from the stack sairuh 
submitted after crashing with a debug build on linux.  The only entry in 
Talkback was with build 2001050915, and the stack was different...that crash 
seemed to be a plugin related crash on exit.

- "SinkContext::OpenContainer", the function that this crash seems to occur in.  
There were no entries in Talkback with that as the stack signature.

If we can get easy steps to reproduce and get more Talkback entries submitted, 
we might be able to get some more info...but for now, there isn't much.
From the point-of-view of the original reporter, there is no talkback because
the browser didn't actually crash. It appeared to lock-up and had to be killed
with task manager. Is there a way in such a case to force a talkback entry? If
so, I'll be glad to run it again.
Looks like a lot of time is spent in resolving the style associated with 
viewsource coloring. With syntax highlighting disabled I was able to view-source 
without any problem. 

Reassigning bug to stephend who enabled coloring.
Assignee: harishd → stephend
Status: ASSIGNED → NEW
I didn't actually code view source's coloring, I just checked it in for someone 
else.  I can't remember if it was Gerv or Boris though. 
per stephend's comment reassigning to defaults.

stephend: can you profile this code to see what functions are eating all of the 
cpu cycles?

hyatt: is this page's view-source mode useful for your testing?  if not, is it 
a reason for you to implement the other style sharing that you didn't already 
do? [i'm tired so if this is all offbase correct me or ignore me]
Assignee: stephend → karnaze
Keywords: hang
So, like, no one noticed that the document returned to view source is 
about 1MB in size?

Just to clarify (or obfuscate?) there are a few problems here:

1) view source takes a long to show a 1MB document, with link coloring, etc.
2) view source is pinging the server for the contents to display (why not 
   pull it from the cache?). [This is bad for (a) efficiency, and (b) server
   side effects].
3) view source sends a GET to the server to retrieve a document that was 
   generated by a POST with parameters.
4) the Domino server treats the GET as an excuse to dump all the records in 
   its db to the client (Well, that's just dumb on the server side, but 
   nothing we can do about it). 

Law has open bugs on (2) and (3):
 "view-source doesn't work for pages generated via forms with method=POST"
   http://bugzilla.mozilla.org/show_bug.cgi?id=64100  
 "view-source should show original source (use cached source)" 
  http://bugzilla.mozilla.org/show_bug.cgi?id=55583

So, that leaves point (1) (and possibly the associated crash) for this
bug.  But, note that this is an edge case: 1MB documents are not the
normal document for 'view source'.

I had just noticed that the obtained viewsource has over 1MB.

There is a debugging #ifdef to dump the viewsource to a file, and I have dumped 
the two corresponding files: one in which syntax highlighting is enabled, and 
the other where it is disabled. But the files are so big that they execeed the 
allowable POST limit in my configuration. And it is because I was having 
troubles attaching them, that I noticed to my surprise that they where over 
1MB...

Yep, jrgm has summarized what is happening... The real culprit here is
actually bug 55583. The document being viewed here is something else.

*** This bug has been marked as a duplicate of 55583 ***
Status: NEW → RESOLVED
Closed: 23 years ago
Resolution: --- → DUPLICATE
Note that the original page was NOT ~1 MB in size. Apparently, the server
responded badly to Mozilla's GET. Of course, the user had no way to know that in
advance. This begs the question, why ever re-fetch a page when it's in the
cache? NS4 did that when requested to print, to my everlasting annoyance.
Yes, it's wrong to pull it from the server again (and even more wrong to 
use a GET, when the original document was fetched with POST + form info). 

Um, but I don't agree that this is a duplicate. In this case, if that other
bug were fixed, then this "view source lockup" would not have occurred. 
And while a 1MB document is out of the norm, we shouldn't get trashed this
badly doing view source. 

So, reopening (with the expectation that this gets punted past mozilla1.0, 
since it won't be a priority in the next couple of cycles.).
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
(... I remember when view source was just whatever text editor that was set 
up as the helper application. I remember how annoying the "new" view source
window was when nav4.x came out. I see view source taking up cycles from 
developers, but it's still no more useful to me than Notepad. Does anyone 
know why we are doing this?).
In some ways, view source is LESS useful than Notepad (or VI or whatever). With
a real editor, I can search, save, and revise the data. Searching in particular
has always been something I sorely miss with view source.
In principle, view-source could be minimal but somewhow there are lots of RFEs 
filed about it, and that's why it takes time away from developers. Despite that,
users are never satisfied... If there was the possibility of passing it to a
separate helper application, that might perhaps cut down on the number of RFEs.

It is a known perf problem that a syntax highlighted view-source on very large 
documents is slow. I think this bug has rather illuminated in a very clear way 
the unexpected nasty effects of the bad re-fetch, and the fact that this can 
bite a good number of people (including experienced hackers), and this over a 
long time, without them remembering it :-)

Re-resolving as DUPLICATE (as you said, this bug wouldn't occur if the other was 
fixed). Please feel free to open a perf bug with a very large static document 
that might help to track/improve the speed of viewsource in a more focused way. 
I did spent some time in the rework of view-source and at this stage, I can only 
bet on hyatt's style changes for performance improvements w.r.t. syntax 
highlighting.

*** This bug has been marked as a duplicate of 55583 ***
Status: REOPENED → RESOLVED
Closed: 23 years ago → 23 years ago
Resolution: --- → DUPLICATE
I apologize for the additional noise but I wanted to say clearly that my 
comment about viewsource (1) should never have been added to this bug report,
since comments of that type do not belong in bug reports, and (2) I apologize
for offending anyone who has done the heavy lifting to implement view source; 
I was wrong in what I said. 

(All I can say in my defense, is that I was abused by a Lotus Domino server 
when I was a small child, and I was transferring my anger at Domino to the
closest thing at hand :-]).
Status: RESOLVED → VERIFIED
No worries. It is a fact that a syntax higlighted view-source can be slow :-) 
Technically, it is slow because the coloring is achieved by inserting 
<span class="something"> on _each_ relevant substring. For example, the 
about:blank document:
<HTML>
  <BODY>
  </BODY>
</HTML>

is transformed as follows (here each class="." stands for the appropriate
coloring class in viewsource.css):

<html><body>
<pre class="viewsource">

&lt;<span class="start-tag">HTML</span>&gt;
  &lt;<span class="start-tag">BODY</span>&gt;
  &lt;/<span class="end-tag">BODY</span>&gt;
&lt;/<span class="end-tag">HTML</span>&gt;

</pre>
</html></body>

And when you add to this the niceties involved with coloring elements of 
attributes: name="value", the version that is handled internally becomes quite 
mouthful (in the DOM sense, the HTML typesetting is not there). And as if that 
wasn't enough, lots of style resolutions are going on. This could scale 
exponentially on some large documents. (Fortunately, there are not really too 
many view-source CSS rules -- hyatt's new style magic which is based on a rule 
tree might perhaps be well suited for view-source, but let's not rejoice too 
early...)

There is however a side-benefit. Because view-source is rendered through the
'normal' Gecko system, functions like 'Find in this page' are _already_ 
possible, but these aren't yet hooked up in the UI (there are bugs filed around 
to eventually allow that). In the meantime, there is a known exploit :-) One can 
'Find in this page' by copy-pasting its URL in the Location bar as:

data:text/html,<A HREF="view-source:http://www.mozilla.org/">mozilla.org</A>
Actually, i think ctrl-f [accel-f] should work even now.  Doron and bz were 
working on that. [fwiw it worked in nc4 too, you just had to know it would]
Here's my $.02.

There are 3 issues here:

1. Handling pages with POST data.
2. Re-fetching from the cache.
3. Slow with coloring.

I can address 1 and 2.  Way back when, there was a bug about view-source
refetching pages.  I fixed that, except in the case of POST data, by having the
docshell/webshell issue the load request with load flags that forced the data to
come from the cache (if at all possible).

POST was still a problem.  I opened bug 55583 to deal with that later (it was
kind of hard because we had to pass the post data from the window that initiated
the view-source to the view-source .xul/.js).

People commandeered bug 55583 with some grandiose (my term, no offense intended)
scheme to always keep the source in memory/cache so we would never go back to
the server.  I got mad and opened bug 64100 to track the issue of POST
independently (that has to be fixed regardless of the grandiose scheme, because
how else are we going to find the right page in memory/cache?).

Then, we fixed something else (bug #?) and changed the way we did view-source
entirely.

Now we're stuck because even if viewsource.js *had* the post data stream, it
apparently has no way to pass that to the view-source: protocol handler so it,
in turn, could use it to fetch the proper entry from the cache.

If somebody can figure a way out of this predicament, please advise.  It seems
we will have to add an optional POST data stream attribute to the view-source
channel, and have it pass that along to its http channel.  But there's still the
problem of getting to the view-source channel because that's hiding underneath
the magic of the nsIWebNavigation interface that we issue the load request to.
I haven't had time to work on this hard enough to figure out the details of how
it should be fixed.
Bug 66334 is the one law refers to (on making view-source a protocol handler).

Should we just file a separate bug on making it possible to pass a POST data 
stream to the view-source: handler?
I tried a (debug) build with hyatt's style changes. I used the local 
viewsource.html that is dumped when '#define DUMP_TO_FILE' is enabled in 
htmlparser/src/nsViewSourceHTML.cpp.  No miracles though. +1MB of flashy
view-source remains a pathological case. Moreover, viewing (not view-source) of 
the original HTML file that was the parent of the generated viewsource version 
was taking even much longer than its view-source (The HTML version has tables).
*** Bug 86355 has been marked as a duplicate of this bug. ***
You need to log in before you can comment on or make changes to this bug.