17586 - file:// directory listing doesn't reflect reality

Status: VERIFIED FIXED

(Core :: XPCOM, defect, P1)

Description

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

DESCRIPTION:  The file:/// directory listings don't reflect reality.  Without
rhyme or reason, some files show up and others don't.  The listing seems to bear
significant resemblence to the state of the directory a month or two ago.

STEPS TO REPRODUCE:
 * load a file:///something/on/your/drive/ in apprunner

ACTUAL RESULTS:
 * many files not listed

EXPECTED RESULTS:
 * all files listed

DOES NOT WORK CORRECTLY ON:
 * Linux, apprunner, 1999-10-29-10-M11 and many prior

ADDITIONAL INFORMATION:
Two attachments to be attached:
 * a screenshot of apprunner
 * an "ls -l" of the directory it's showing

Comment 1

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Attachment: screenshot of directory listing

Comment 2

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Attachment: the real directory listing

Comment 3

[leger]

Directory listing is not for dogfood.  Putting on PDT- radar.

Comment 4

[Mo DeJong]

I just ran into this problem and I came across this bug report.
I am also going to attach a screen shot I got from todays
mozilla build (Sun Nov 14) and compare it to the output of
ls in the same directory.

Could someone point me to the code that is responsible
for this uglyness. I might be able to figure out what
the problem is, but I have no idea where to begin looking.

Comment 5

[Mo DeJong]

Attachment: The directory contents displayed by mozilla

Comment 6

[Mo DeJong]

Attachment: The output of ls in the directory.

Comment 7

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

One place to start looking might be:

http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/file/src/nsFileChannel.cpp

Comment 8

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

cc:ing [e-mail] .  See my previous comment...

Comment 9

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Or maybe...
http://lxr.mozilla.org/seamonkey/source/netwerk/base/src/nsDirectoryIndexStream.cpp

... but the core of this thing has got to be elsewhere still...

Comment 10

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Aha.  Here's a good place to poke around:

http://lxr.mozilla.org/seamonkey/search?string=http-index-format

Comment 11

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

This bug seems to have been fixed recently.

(Since it was platform specific (right?), I would guess it was nsFileSpec or
something...)

Comment 12

[Chris Waterson]

I checked in a fix for 16797; it could be that this was a dup of that bug. Or,
as you note, it could be some nsFileSpec voodoo. I'll mark it fixed, we can
re-open, etc. if it resurfaces.

Comment 13

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

This bug has resurfaced in Linux mozilla 1999-11-28-08-M12.  The current
symptoms are indistinguishable from those in the original bug report.

Reopening.

Comment 14

[leger]

Clearing FIXED resolution due to reopen.

Comment 15

[Hugh Kennedy]

This is also present in M11 on Solaris 2.6, and in a different variation under
M11 on NT.

On WinNT4 sp5 (Build ID: 199911520) opening a directory listing via
file:///C|/
fails to list any files or directories at all.
The header (Name Size Last Modified) is there, but there are no contents.

Comment 16

[Chris Waterson]

This works for me in the build I just pulled. 11/30/1999.

Comment 17

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

I'm still seeing this in Linux mozilla 1999-12-01-13-M12.  Reopening.

Comment 18

[Chris Waterson]

Hmm. I'm still not able to reproduce this :-/. Is it deterministic? I.e., If
you press "reload", do the same files show up missing? I'm wondering if maybe
the buffer quantization of the HTTP/Index stream is causing this...

Comment 19

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

The output seems the same every time.  However, if I modify the file list in the
directory, the same NUMBER of files seem to be listed all the time.  This number
does vary a little.  In the directory I counted, it was 29.  In another, I think
it was slightly higher (but the browser crashed when i tried to scroll).

Could this have to do with packets of the http-index format?

Comment 20

[leger]

Clearing WORKSFORME resolution due to reopen.

Comment 21

[Judson Valeski]

this is a tree filling issue. cc'ing hyatt.

Comment 22

[Chris Waterson]

I don't think it's a bug with the tree code. I think it's either in the
generation of the HTTP/Index stream, or the datasource that converts the
HTTP/Index stream into a graph....

Comment 23

[Chris Waterson]

*** Bug 19225 has been marked as a duplicate of this bug. ***

Comment 24

[Chris Waterson]

*** Bug 22592 has been marked as a duplicate of this bug. ***

Comment 25

[Chris Waterson]

Not an M13 stopper. Moving to M14; with luck, may fix it in time for the M13
train.

Comment 26

[Asa Dotzler [:asa]]

*** Bug 25721 has been marked as a duplicate of this bug. ***

Comment 27

[Sitsofe Wheeler]

Does this bug also apply to when Mozilla doesn't show a listing at all? 
Pressing the reload goes on to show a listing though....

I also have a suggestion: could this bug's summary be changed from file:// to 
file:/// because this would come in a search for file:// too.

Comment 28

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

I received the following email message from Morten Welinder <[e-mail]>. 
Note that "ls -f" will also list files unsorted.  However, now I'm seeing
nothing at all, so I can't verify that it's truncation...

----

I see the same thing.

The file list appears to be truncated.  Yes, truncated, but you wouldn't
know unless you used dirty tricks like

  perl -e 'opendir (DIR,".netscape"); print join (":", readdir (DIR)), "\n";'

to print the directory contents.  "ls" sorts, so it is hard to tell the
order.

Could you please verify that you are seeing truncation too?  (Note, that
"the state of the directory a month or two ago" will include a lot of
files in the true beginning of the directory.)

Comment 29

[Chris Waterson]

from private email. nsIFile guys, does this look like it may cause trouble?

Subject: Mozilla 17586
   Date: Sun, 30 Jan 2000 01:49:23 +0100 (MET)
   From: Morten Welinder <[e-mail]>
     To: [e-mail]

I am seeing http://bugzilla.mozilla.org/show_bug.cgi?id=17586 every
single time I go to my ~/.netscape/ directory which contains the
following files:

welinder@anemone:~ > perl -e 'opendir (DIR,".netscape"); print join (":", 
readdir (DIR)), "\n";'
.:..:lock:cert5.db:key.db:history.db:plugin-list:summary2.dat:cache:cookies:pab.
na2:bookmarks.html:plugin-list.BAK:registry:cert7.db:key3.db:secmodule.db:abook.
nab:preferences.js:archive:xover-cache:main.html:history.list:custom.dic:history
.dat:liprefs.js

The files "abook.nab", "archieve", "custom.dic", "history.list", "liprefs.js",
"main.html", "preferences.js", "history.dat", and "xover-cache" do not
show up.

NOTE: that's clearly not random!  It's the tail of the list, but a normal
"ls" will not show that.  I am using M13 (Linux, SuSE 6.3).  Similar
things seen earlier on Solaris 2.7 with M12.

Wild guess: nsDirectoryIterator doesn't have a copy constructor or an
assignment operator.  The defaults just will not work on Unix, so it
should have one.  (In particular, if closedir was hit because of a copy,
that would truncate the list rather effectively.)

Also, the ++ operator on Unix assumes that it will never see ".." and
"." in that order.  Is that safe?

Morten

Comment 30

[Chris Waterson]

sitsofe: see bug 25303

Comment 31

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

I confirm I am seeing truncation.  The listing shown in mozilla perfectly
matches "ls -f | head -30 | sort".

Comment 32

[Chris Waterson]

although this was PDT- for DOGFOOD, re-nominating as potential Beta1.

Comment 33

[leger]

Moving [DOGFOOD] to keyword field as dogfood.

Comment 34

[leger]

Putting on PDT+ radar.  Actually, yes needed for dev dogfood.

Comment 35

[Chris Waterson]

Attachment: partial (?) fix

Comment 36

[Chris Waterson]

The above patch fixes bustage that occurred when nsIFile landed. I was relying
on the "length" of a directory being zero (bogus). When nsIFile landed, Unix
actually started coughing up the inode size for directories. The fix is to
explicitly set the stream length to -1 if we're looking at a dir. (Also added a
bunch of debugging code.)

Comment 37

[Judson Valeski]

patch looks good to me.

Comment 38

[Chris Waterson]

Attachment: changes to nsDirectoryViewer.cpp

Comment 39

[Chris Waterson]

Between nsIFile and this cleanup, I'm not able to reproduce the "spotty
directory" problems. Marking as fixed. N.B. that bug 25303 is still open though!

Comment 40

[Judson Valeski]

ouch. I'm seeing this on winnt this morning. Steps to repro. create a new dir in 
c:\ , then view file://c:\ that new dir doesn't appear. Note that I added a 
single char dir "b" and taht one showed up :-/

Reopening.

Comment 41

[Suresh Duddi (gone)]

I dont see this bug on linux 2/13 (sunday) 11am build.

I created new directories and new files and all of them showed up. I had to give
it a resize to have it show anything though :-( i think that is a separate bug.

Can someone check this on windows too. 2/9 was just about when we went into
regression hell. So I am optimistic this is actually fixed.

Comment 42

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Is that "separate bug" filed?  (That is, the bug that the file directory listing
requires a reflow before it shows up.)  I'd say it's a beta blocker...

Comment 43

[Judson Valeski]

yes, either waterson or hyatt has it.

Comment 44

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

For the record, it's bug 25303.

Comment 45

[Chris Waterson]

Just so I'm clear, this bug now means:

"When I type 'file://c:\' into the URL bar, nothing shows up. Even after I 
resize the window. But other URLs are ok."

If so, could we please close this and open another bug (assigned to gagan for 
URL parsing)? (Don't mean to be bureaucratic, but this bug kinda meant 
something else...)

Comment 46

[Chris Waterson]

ok, this bug is fixed. Opened bug 27764 to track problems with root directories 
on Win32.

Comment 47

[Tom Everingham]

verified:
NT 2000021708