39036 - Linux filepicker doesn't show hidden directories or files (dotfiles)

Status: VERIFIED FIXED

(Core :: XUL, enhancement, P2)

Description

[hacksaw]

Go to mail/news
Select Import... from the file menu
Try and find .netscape or .exmh or .any_damn_thing

Comment 1

[scottputterman]

reassigning to chuang.

Comment 2

[scottputterman]

cc'ing pavlov.  Is this controlled by the code calling the file picker or is
this a file picker bug? Marking M18.

Comment 3

[chuang]

pass to tonyr.

Comment 4

[tonyr]

I think my import tools are only accessed from Tools->Import, not from the File 
menu.  I'm not sure what File->Import... calls.

Comment 5

[chuang]

File->Import... calls the same function as Tools->Import.

Comment 6

[scottputterman]

cc'ing tonyr.  Please take this back if it's yours.

Comment 7

[lchiang]

+ per mail triage

Comment 8

[lchiang]

reassign back to Tony.  See candice's comments.

Comment 9

[Stuart Parmenter]

this should be marked as a dup of the convert nsIFileSpecWithUI to nsIFilePicker
for mailnews bug that sspitzer has.  nsIFileSpecWithUI is going away, and this
bug is invalid with the new file picker.

Comment 10

[tonyr]

*** Bug 36483 has been marked as a duplicate of this bug. ***

Comment 11

[tonyr]

Anyone want to review the fix and I'll check it in?  I just switched to 
nsIFilePicker and converted the returned file to an nsIFileSpec for the 
underlying import code which still requires nsIFileSpec's.

Index: importDialog.js
===================================================================
RCS file: /cvsroot/mozilla/mailnews/import/resources/content/importDialog.js,v
retrieving revision 1.11
diff -r1.11 importDialog.js
423a424,435
> function CreateNewFileSpecFromPath( inPath)
> {
> 	var file = 
Components.classes["component://netscape/filespec"].createInstance();
> 	if (file != null) {
> 		file = file.QueryInterface( Components.interfaces.nsIFileSpec);
> 		if (file != null) {
> 			file.nativePath = inPath;
> 		}
> 	}
> 
> 	return( file);
> }
451c463
< 			var filePicker = 
Components.classes["component://netscape/filespecwithui"].createInstance();
---
> 			var filePicker = 
Components.classes["component://mozilla/filepicker"].createInstance();
453c465
< 				filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFileSpecWithUI);
---
> 				filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFilePicker);
455c467
< 					// filePicker.create( window.top, 
GetBundleString( 'ImportSelectSettings'), filePicker.modeLoad);
---
> 					var file = null;
457,459c469,485
< 						filePicker.chooseInputFile( 
GetBundleString( 'ImportSelectSettings'), filePicker.eAllFiles, null, null);
< 						setIntf.SetLocation( 
filePicker.QueryInterface( Components.interfaces.nsIFileSpec));
< 					} catch( ex) {
---
> 						filePicker.init( top.window, 
GetBundleString( 'ImportSelectSettings'), 
Components.interfaces.nsIFilePicker.modeOpen);
> 						filePicker.appendFilters( 
Components.interfaces.nsIFilePicker.filterAll);
> 						filePicker.show();
> 						if (filePicker.file && 
(filePicker.file.path.length > 0))
> 							file = 
CreateNewFileSpecFromPath( filePicker.file.path);
> 						else
> 							file = null;
> 					}
> 					catch(ex) {
> 						file = null;
> 						error.value = null;
> 						return( false);
> 					}					
> 					if (file != null) {
> 						setIntf.SetLocation( file);
> 					} 
> 					else {
524c550
< 			var filePicker = 
Components.classes["component://netscape/filespecwithui"].createInstance();
---
> 			var filePicker = 
Components.classes["component://mozilla/filepicker"].createInstance();
526c552
< 				filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFileSpecWithUI);
---
> 				filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFilePicker);
529,530c555,561
< 						filePicker.chooseDirectory( 
GetBundleString( 'ImportSelectMailDir'));
< 						mailInterface.SetData( 
"mailLocation", CreateNewFileSpec( filePicker.QueryInterface( 
Components.interfaces.nsIFileSpec)));
---
> 						filePicker.init( top.window, 
GetBundleString( 'ImportSelectMailDir'), 
Components.interfaces.nsIFilePicker.modeGetFolder);
> 						filePicker.appendFilters( 
Components.interfaces.nsIFilePicker.filterAll);
> 						filePicker.show();
> 						if (filePicker.file && 
(filePicker.file.path.length > 0))
> 							mailInterface.SetData( 
"mailLocation", CreateNewFileSpecFromPath( filePicker.file.path));
> 						else
> 							return( false);
615c646
< 		var filePicker = 
Components.classes["component://netscape/filespecwithui"].createInstance();
---
> 		var filePicker = 
Components.classes["component://mozilla/filepicker"].createInstance();
617c648
< 			filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFileSpecWithUI);
---
> 			filePicker = filePicker.QueryInterface( 
Components.interfaces.nsIFilePicker);
635,636c666,672
< 				filePicker.chooseDirectory( GetBundleString( 
'ImportSelectAddrDir'));
< 				file = filePicker.QueryInterface( 
Components.interfaces.nsIFileSpec);
---
> 				filePicker.init( top.window, GetBundleString( 
'ImportSelectAddrDir'), Components.interfaces.nsIFilePicker.modeGetFolder);
> 				filePicker.appendFilters( 
Components.interfaces.nsIFilePicker.filterAll);
> 				filePicker.show();
> 				if (filePicker.file && 
(filePicker.file.path.length > 0))
> 					file = filePicker.file.path;
> 				else
> 					file = null;
644,645c680,686
< 				filePicker.chooseInputFile( GetBundleString( 
'ImportSelectAddrFile'), filePicker.eAllFiles, null, null);
< 				file = filePicker.QueryInterface( 
Components.interfaces.nsIFileSpec);
---
> 				filePicker.init( top.window, GetBundleString( 
'ImportSelectAddrFile'), Components.interfaces.nsIFilePicker.modeOpen);
> 				filePicker.appendFilters( 
Components.interfaces.nsIFilePicker.filterAll);
> 				filePicker.show();
> 				if (filePicker.file && 
(filePicker.file.path.length > 0))
> 					file = filePicker.file.path;
> 				else
> 					file = null;
655c696
< 		file = CreateNewFileSpec( file);
---
> 		file = CreateNewFileSpecFromPath( file);

*****CVS exited normally with code 1*****

Comment 12

[lchiang]

cc: putterman

Comment 13

[tonyr]

Converted the dialog to use nsIFilePicker rather than nsIFileSpecWithUI.  The 
underlying import code still uses nsIFileSpec though - it would be a major 
effort to switch to the new file classes.  (After all, import does a lot of work 
navigating directories and reading/writing/creating/deleting files.)

Comment 14

[Brian Ryner (not reading)]

Um, so.... this may break again when I land the rdf filepicker branch.  The
filesystem datasource by default ignores hidden files entirely.

I've talked some with rjc about how we can solve this.  I am willing to fix it
but I'd request this be re-triaged to see if we really need it for beta3.

Comment 15

[Brian Ryner (not reading)]

Actually, I will go ahead and reopen this and assign it to myself until we get
this resolved, since I am landing the branch now.

Comment 16

[Brian Ryner (not reading)]

-> me

Comment 17

[Peter Trudelle]

clearing nsbeta3+ for triage by new owners.
Is opening the old AB in a .directory the typical case for import, or does that
normally get converted on profile migration?  Also, isn't Import usually done on
an AB that is explicitly Exported, or Saved?  I'm trying to determine how
critical a need we have for displaying .directories. Also, is the workaround of
typing the explicit path in sufficient?

Comment 18

[Peter Trudelle]

nsbeta3-/future, 'native' file pickers don't have any easy way to show these either.

Comment 19

[Brian Ryner (not reading)]

better summary

Comment 20

[sairuh (rarely reading bugmail)]

i'll take this (unless esther minds ;).

Comment 21

[sairuh (rarely reading bugmail)]

*** Bug 56991 has been marked as a duplicate of this bug. ***

Comment 22

[Brian Ryner (not reading)]

mozilla 0.9... we should introduce some sort of "Show hidden files" checkbox in
the filepicker dialog.

Comment 23

[Gervase Markham [:gerv]]

If other apps using the native filepicker are also unable to show hidden files, 
surely this doesn't merit a relnote? Please nominate if you disagree...

Gerv

Comment 24

[Peter "jag" Annema]

bryner: I suggest we introduce some XP notion of what the OS settings are on
showing/hiding "hidden" files, then use that information in the filesystem
datasource.

Of course, we can always to the checkbox (or whatever) thing (too) :-)

Comment 25

[sairuh (rarely reading bugmail)]

relnote to endusers: summary sez it all: you'll not be able to view your hidden
files or dirs [eg, dot files, dot directories] using either the linux [xul]
filepicker --or by viewing a directory listing [file://] in the browser's
content window.

this problem is specific to unix/linux, and doesn't occur with the native
filepickers or dir views on Mac or Win32.

Comment 26

[John Morrison]

*** Bug 62662 has been marked as a duplicate of this bug. ***

Comment 27

[Ben Goodger (use ben at mozilla dot org for email)]

*** Bug 60293 has been marked as a duplicate of this bug. ***

Comment 28

[sairuh (rarely reading bugmail)]

with bryner's rewrite of the file picker [see bug 75838], you can easily turn on
display of dot files/dirs by changing 'false' to 'true' in the following line of
xpfe/components/filepicker/res/content/nsFileView.js

  this.mShowHiddenFiles = false;

bryner, could this be turned on by default? i find it quite useful to view and
trowel through dot files and dirs... you had suggested adding a checkbox in the
file picker dialog as an alternative --mpt, would it be better to just have this
turned on, or have the checkbox there? [i'd prefer a less cluttered dialog
without the checkbox, but don't have strong feelings either way.]

Comment 29

[hacksaw]

I have like 226 .* files. It should be a check box.

Even better would be a regexp matcher that works better than the very old
sunview one.

Its default regexp would be [^\.].* , that is, anything that doesn't start with
a '.'.

Perhaps the list of patterns could be replaced by this, or perhaps it should pop
up if the user selects the entry "other" from that list.

A side comment: perhaps the button at the top of the dialog should have an
left-up arrow instead of ".."?

  A
  |___ instead of ..

Comment 30

[Peter Trudelle]

This is starting to smell like feature creep to me.  I doubt we have much real
user need for this in the product; nobody has offered any typical end-user
scenarios where this feature would be needed.  However, if it is as trivial as
it sounds, I would caution that it be added in as non-intrusive a manner as
possible.  I concur with Jag's earlier suggestion of basing it on a system pref,
where one exists. I doubt it is worth complicating the UI with a pref, let alone
a button in the file picker. I am very much against having it enabled by
default, and God forbid we should have users editing regular expressions! 
However, this capability obviously appeals to mozilla developers (and sure, I'd
use it once in a while too), so perhaps any such UI could be added as an
overlay?

Comment 31

[sairuh (rarely reading bugmail)]

i certainly wouldn't mind having a backend pref for this, nor would i mind [esp
after reading hacksaw's comment] having said pref off by default. how doable
would that be?

Comment 32

[hacksaw]

Having it be a pref (as in "changeable with the preferences dialog") is a lose.

It almost always going to be a temporary thing, so having to change it in the
prefs dialog is just annoying, because then you'll want to change it back.

That's why a button in the dialog is the best solution.

However, RE's, or at least file globs would be preferable.

Think about using composer. I'd like to edit all the file from my old MS server,
all with a ".htm" suffix. Just the .htm's thanks, the other are done.

I'd like to look at all my files called my*.html.

I'd like to look at the files ending in .doc (which I use to indicate complete
documents, not damned MS word files... but I digress)

I'd like to see all the file ending in html and txt, but not the 400 jpegs.

And, importantly, I'd like to be able to find all the cute little iconized
jpeg's left by xv and ee, some would make great icons. 

If I had my druthers, I'd want a checkbox on the page for file:// listing as well.

Another side comment: The history thing at the top? I think that it'd be better
as a cascade listing of the path, ala the gimp or Mac OS finder window titles.

Comment 33

[Matthew T (active 1999-2002)]

> --mpt, would it be better to just have this turned on, or have the checkbox
> there?

In the filepicker, I think `Show Hidden Files' should be a checkbox item at the
bottom of the shortcut menubutton (the same menubutton which provides shortcuts
to your home directory, and to mounted devices determined through mntent and
friends). It would default to off.

For a file:// listing, I think hidden files should always be shown, with no
exceptions. Showing a file listing in your Web browser is an advanced task
anyway, so showing hidden files is quite acceptable.

Peter, I think the necessity for this feature is demonstrated quite well by the
original bug report.

Comment 34

[Peter Trudelle]

MPT: if that were the case, I would have expected answers to the questions I
asked in here last September.  I don't think normal users should have to be
fishing around in hidden directories they probably don't even know exist.  If
they are, then I think some other solution is required, such as offering to
auto-import from competing products.
BTW, putting the check for this at the bottom of the generated directory menu
seems wrong to me; unintuitive, yet easy to hit by accident.  If someone adds it
(and I can't commit any of my team's engineering time for that), I think it
should be a easily clickable, since you'll typically toggle it both ways.

Comment 35

[hacksaw]

This is the generic file picker, right? We're no longer talking about a file
picker just for bookmark management.

The generic file picker must be able to see *every* file available to the user.

Don't presume to lump dot files in with the MacOS hidden file, it's not the same
thing. Dot files are a convenience to the user. The feature allows users to hide
files and directories from regular listings. However, Unix users learn to view
those files very shortly after they learn how to show detailed listings.

For one thing, folks I know routinely have a backup of important files in a
.backup or somesuch. I'd certainly have resume.html hiding their.

Having the program (and not the installer!) convert other bookmark files and
address lists is fine, but that a separate function. And you'd still better be
able to find the bookmark file. I might have moved it to .netscape.old...

Comment 36

[Peter Trudelle]

Yes, this is the generic filepicker. However, I don't agree that .files are
primarily a convenience for the user, so much as a way for the system and apps
to hide files that end users shouldn't usually need to see, and aren't normally
the object of commands.  Many Unix users are advanced enough to be aware of them
and use them, which may also be true to a lesser extent on Windows and even Mac.
 That doesn't mean that an application intended for end users "must" show them.
 User that are savvy about them can always open them from the command line or
rename them to be visible, or type the path in the filepicker, which works just
fine right now.  This just doesn't fit our profile of things we can work on in
the current release cycle, although someone else could certainly do the work and
submit it to drivers.

Comment 37

[hacksaw]

>Many Unix users are advanced enough to be aware of them
>and use them

That doesn't mean we're in love with having to work around deficiencies in the
apps we use.

Deciding to hide things in Unix isn't advanced activity. In Mac and Windows you
need a special app to set a bit somewhere. In Unix it's just '.' in front of the
name. End users do this. 

Comment 38

[Brian Ryner (not reading)]

*** Bug 105866 has been marked as a duplicate of this bug. ***

Comment 39

[André Dahlqvist]

Peter Trudelle wrote:
> This is starting to smell like feature creep to me.  I doubt we have much real
> user need for this in the product; nobody has offered any typical end-user
> scenarios where this feature would be needed.

Let me give you an example that I have run into: The standard UNIX file for
storing your email signature is ~/.signature, and that's where I have it. I
wanted to setup this in the Mail/News setup, and browsed for it in the
filepicker. Then I noticed it didn't show dotfiles. So instead of browsing for
it I had to type in the path by hand.

Comment 40

[Peter Trudelle]

->enhancement/untargetted, adding nsbeta1 keyword to get triage team to look at
it (can't commit to doing the work without that)

Comment 41

[R.K.Aa.]

*** Bug 118472 has been marked as a duplicate of this bug. ***

Comment 42

[Travis Crump]

end-user scenario that I just ran into:  I wanted to save an image as my desktop
background.  The  expected procedure for this is to  right click on the image
and do Save as, and then drop it in ~/.enlightenment/backgrounds, but I couldn't
find the .enlightenment directory.  I briefly considered ln -s it to a visible
directory, but decided that this would be a hastle and add to the clutter of my
home directory. I then hit upon the solution of typing in the whole path.  The
problem with this is that a) I wasn't completely sure it would work even as I
was doing it b) I wasn't completely sure of the exact path I wanted(ie was it
background or backgrounds?) so I had to go to a terminal to figure out the exact
path.

I like the checkbox idea, I think that is the way to go.  As for users not
usually interacting with .* files, besides this example what about .login,
.logout, .forward, .alias, .<shell>rc...(though I am not suggesting that a user
would ever want to access these files from within mozilla, it is still a precedent)

Comment 43

[Bob Atkins]

Another end user scenario:

I routinely setup my home page to be my bookmarks file. I can't do this with the
file picker since I can't see the ~/.mozilla directory. I can either type it in
manually or create a soft link from a visible directory.

I sorely miss the ability to show all files - including . files. This could
esily be fixed when I select 'All Files (*) from the the 'Files of Type' drop
down menu. An additional check box is not required. For the real purists perhaps
an additional item in the drop down list called 'All + Hidden files'.

The bottom line - I don't have this issue with any other Unix app and I don't
see why Mozilla has to be the odd one out. I thought the idea was to have
mozilla appeal to the broadest range of users. In its current form the
filepicker (and several other mozilla features) only appeal to the basic novice
user. Having used Netscape since the very first version and the original Mosaic
in its early alpha days, it is dissapointing to see how several advanced
features from those versions have yet to make it into mozilla.

For the purists that don't want to clutter dialogs and menus for novice users
while still supporting intermediate and advanced users - take a page from
Microsloth's book - offer an option for Basic, Intermediate and Advanced user
modes which alters just how much a user will see on the menus and dialogs.

Comment 44

[André Dahlqvist]

Peter Trudelle, do you need more end-user scenarios than the ones we have
already provided? Here is another one:

At my university we should place files in ~/.html to make them automagically
show up as our webpages, and I think this is sort of standard for unix systems.
Instead of being able to browse for it I had to type it in manually. A novice
user of mozilla and UNIX would not even realize that the directory is there.

Peter, do you still think it is feature creep?

Comment 45

[Peter Trudelle]

It still has a whiff of feature creep around it (regular expressions, UI modes
and such), but the scenarios here are compelling.  I'd be okay with adding
something simple, such as a discrete checkbox.  File view considerations should
be kept separate though, this bug only applies to the filepicker.  How much
time/risk would be involved?  Is anyone able to attach a patch? I'd be happy to
pay Brian to review and integrate, but I doubt I could authorize anything
non-trivial or risky.

Comment 46

[Brian Ryner (not reading)]

The good news here is that in the latest rewrite of the filepicker backend, I
built in support for a "show hidden files" mode, and the ability to switch
to/from that mode on the fly.  So, the frontend work here should be pretty
trivial, once we decide on an appropriate UI.

Comment 47

[hacksaw]

I'd agree that regexps are non-trivial, though I think they'd still be nice in a
full featured file-picker. But the real issue is unix dot files. As there
appears to be a backend solution, I'd say a checkbox would be good enough to
close this issue. The other stuff could then be moved into the realm of "when
someone is sufficiently motivated to submit a patch."

Comment 48

[Peter Trudelle]

Okay Brian, if you or someone else can add this in 0.9.9, then it can still make
moz1.0 and MachV, otherwise it may have to be treated as an exception. 
->0.9.9/P2/nsbeta1+

Comment 49

[Brian Ryner (not reading)]

Nominating for nsbeta1 triage.

Comment 50

[Brian Ryner (not reading)]

Attachment: patch

Comment 51

[jag (Peter Annema)]

Comment on attachment 69005 [details] [diff] [review]
patch

sr=jag

Check if you really need that pack="end".

Comment 52

[Ben Goodger (use ben at mozilla dot org for email)]

Comment on attachment 69005 [details] [diff] [review]
patch

r=[e-mail]

Comment 53

[Brian Ryner (not reading)]

checked in.

Comment 54

[André Dahlqvist]

With 2002-02-12-06 on Linux I can't scroll quiet all the way down with the
scrollbar. The scrollbar stops about 0.5 centimeters before it has reached it's
bottom position. This happens with this new option on or off, so it might have
been in the dialog before this.

Can someone verify?

Comment 55

[Christian Reis]

Why were the alignments to the  buttons changed from right to center? 
(At least Linux has default right-aligned as of 2002021121)

Comment 56

[André Dahlqvist]

And now I can't reproduce (at least not all the time) what I said in comment 54.

Comment 57

[Brian Ryner (not reading)]

[e-mail], can you explain what you mean?   The OK and Cancel buttons
should still be on the right side of the dialog, as seen in:

http://homepage.mac.com/bryner1/.cv/bryner1/Public/fp%2Dhidden2.png-link.png

Comment 58

[Christian Reis]

Comment on attachment 69005 [details] [diff] [review]
patch

>+<hbox class="dialog-button-box" align="center" pack="end">
>+  <checkbox label="&showHiddenFiles.label;" oncommand="toggleShowHidden();"
>+            flex="1" align="left"/>
>+  <button dlgtype="accept" class="dialog-button"/>
>+  <button dlgtype="cancel" class="dialog-button"/>
>+</hbox>

Sorry, never mind really. I don't know XUL's positioning rules very well, but I
was under the assumption that, from the code, the button would inherit the 
parent's alignment, and thus, both buttons would be centered. I guess the XUL
for
the buttons already defines them as right-aligned (which strikes me as odd?).
(My experience being with GTK+'s pos model, which isn't exactly different from
XUL's, but different enough to cause confusion like this)

Comment 59

[Brian Ryner (not reading)]

The buttons aren't specifically right aligned, but they don't have any flex set.
 The checkbox flexes, with left alignment, so it takes up all of the extra space.

Comment 60

[sairuh (rarely reading bugmail)]

vrfy'd fixed with 2002.02.21.08 comm bits on linux rh7.2.

Comment 61

[Dawn Endico]

removed the item for this bug from the release notes for 0.9.9 and beyond,
because the bug is fixed now. Let me know if you think the item should be 
re-added.

Comment 62

[hacksaw]

Looks good to me. Thanks folks. It's nice and fast too.
BTW, I'm using it in RH6.2.