900117 - From command line, cannot attach file(s) if file name contains commas (,), single quotes ('), or equal signs (=)

Status: NEW

(Thunderbird :: OS Integration, defect)

Description

[omidlink]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0 (Beta/Release)
Build ID: 20130627185035

Steps to reproduce:

The two files $HOME/test test.txt and $~/test, test.txt exists. Trying to attach them from command line:
thunderbird -compose "attachment='$~/test test.txt'" works. thunderbird -compose "attachment='$~/test, test.txt'" does not work and gives a file does not exist error.


Actual results:

The file ~/test does not exist so could not be attached to the message.


Expected results:

Both files should have been attached. Attaching works when the file with comma(s) in its name is selected using Thunderbird's Attach File(s) GUI.

Note:
In Thunderbird 3+, specifying the protocol file:// is not required any more.

Both

thunderbird -compose "attachment='$HOME/test test.txt'"

and

thunderbird -compose "attachment='file://$HOME/test test.txt'"

work.

Neither

thunderbird -compose "attachment='$HOME/test, test.txt'"

nor

thunderbird -compose "attachment='file://$HOME/test, test.txt'"

works.

Comment 3

[:aceman]

Not sure this belongs into "OS Integration".
Maybe the parsing is done by the compose code.

Comment 4

[Magnus Melin [:mkmelin]]

http://mxr.mozilla.org/comm-central/source/mail/components/compose/content/MsgComposeCommands.js#1918

Comment 5

[Thomas D. (:thomas8)]

Confirming as a problem. Can't find a dupe right now, but there should/might be one.

Looking at documentation, I think there's a design bug somewhere in this highly complex construction of parsing switches:

https://developer.mozilla.org/en-US/docs/Mozilla/Command_Line_Options?redirectlocale=en-US&redirectslug=Command_Line_Options#Syntax_Rules

> Multiple message options are separated by comma (,), for example:
> "to=[e-mail],subject=cool page" . Comma separators must not follow or
> precede spaces ( ). To assign multiple values to a field, enclose the values
> in single quotes ('), for example:
> "to='[e-mail],[e-mail]',subject=cool page"

Note the extra hint that "comma separators must not follow or precede spaces" maybe that's where the problem already starts.

So per this wrong/incomplete design, how does TB parse reporter's case?

> thunderbird -compose "attachment='$HOME/test, test.txt',subject=test"

* outer double quotes delimit the string of parameters for -compose command (this is required bc otherwise you could not have parameters starting with " -" because they would be parsed as command line options.
* the right-most comma before subject delimits different message composition fields, here attachment vs. subject
* multiple values for a single fields, by this design, are also separated with comma; however, that requires an extra level of quotes to avoid the following ambiguity:

"attachment=test1.txt,test2.txt,subject=test" (wrong syntax: ambiguous)

When parsing this, it would be hard to figure out that subject=test is not a 3rd attachment. So that's why they introduced single quotes to delimit multiple values for single field:

"attachment='test1.txt,test2.txt',subject=test"

Now it's less ambiguous because it's clear that the first (inner) comma separates multiple values for attachment field, whereas second (outer) comma separates attachment vs. subject field. However, per this bug, what's missing now is a third layer of quotes or some other delimiter to handle long file names containing comma (plus space) in this syntax:

"attachment='{test1,foo.txt},test2.txt',subject=test"

In the current syntax, something like that, not exactly using { and } obviously because we'd reproduce the problem with other characters, would be required to delimit the two file names, 
test1,foo.txt (1st file)
text2.txt (2nd file)

That's reporters case, so what reporter considers one file, TB considers two files because of current incomplete syntax. I'm not sure if there's any workaround that would currently allow reporter's case.

For solution, I wonder if we could handle nested quotes, but it seems somewhat orthogonal to the current seemingly left-to-right parsing logic. I suppose we have more cases that actually fail, what if the file name itself contains single quotes?

"attachment='test1'foo.txt,test2.txt',subject=test"
Does that attach these two files?
test1'foo.txt
test2.txt

Comment 6

[Thomas D. (:thomas8)]

(In reply to Thomas D. from comment #5)
> Looking at documentation, I think there's a design bug somewhere in this
> highly complex construction of parsing switches:
> "attachment='test1.txt,test2.txt',subject=test"

In fact, that currently intended syntax is highly flawed because the entire string (including all single quotes and equal sign) is actually a valid filename for a *single* file under Windows:

Valid single filename: 'test1.txt,test2.txt',subject=test
Valid single filename: 'test1.txt,test2.txt',subject=test.txt
Valid single filename: 'foo's notes, with bars' comments,subject=test.txt

I guess there's currently no way of attaching such a file via command line (hope it works from the UI at least), but there should be. All permitted filenames should be attachable via command line.

Comment 7

[Thomas D. (:thomas8)]

Can't find dupe, let's confirm

Comment 8

[Thomas D. (:thomas8)]

Adding equal signs to summary for verification, suppose that also fails under the right circumstances as in comment 6, but didn't explicitly test.

Comment 9

[Thomas D. (:thomas8)]

From command line, not being able to attach a file with apostrophe (') or (,) in file name sounds like a pretty bad limitation, because those are really regular characters which can easily occur in any Windows filename.

So any clues as to how to solve this bug? Do we need to define our own escape character for the command line? Maybe existing escape characters from OS?

(In reply to Thomas D. (currently busy elsewhere) from comment #6)
> (In reply to Thomas D. from comment #5)
> > Looking at documentation, I think there's a design bug somewhere in this
> > highly complex construction of parsing switches:
> > -compose "attachment='test1.txt,test2.txt',subject=test"
> 
> In fact, that currently intended syntax is highly flawed because the entire
> string (including all single quotes and equal sign) is actually a valid
> filename for a *single* file under Windows:
> 
> Valid single filename: 'test1.txt,test2.txt',subject=test
> Valid single filename: 'test1.txt,test2.txt',subject=test.txt
> Valid single filename: 'foo's notes, with bars' comments,subject=test.txt
> 
> I guess there's currently no way of attaching such a file via command line
> (hope it works from the UI at least), but there should be. All permitted
> filenames should be attachable via command line.

Comment 10

[Magnus Melin [:mkmelin]]

I think it's not fixable in any good way with the current syntax, which is flawed. 
Usually command line flags take one argument, not many, so they don't get into this situation. 

You'd have to redesign it all to take --attachment=foo.doc --attachment=bar.doc and so on.

Comment 11

[Stéphane Guillou]

Isn't this the same issue as bug 258887? Should it be consolidated there?