Open Bug 1223618 Opened 9 years ago Updated 2 years ago

Newlines are being inserted into IMAP request for quota (when hierarchy delimiter is "\"=BackSlash, Tb appends 0x0D0A0A to mboxname in quoted text part of IMAP command)

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86_64
Linux
Type:
defect

Tracking

(Not tracked)

People

(Reporter: ryan.martindale, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

tb.tcpdump
4.55 KB, text/plain
Details
tb.log
10.18 KB, text/plain
Details 
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150922203042

Steps to reproduce:

When I place a folder underneath another folder, it no longer shows any messages (IMAP server). This is using Thunderbird 38.2.0 (CentOS 6.7 64bit).


Actual results:

Digging into the communications (using tcpdump) I saw that the "getquotaroot" command response was sending an extra bunch of newlines. From the thunderbird logging (with the names of the folders changed):

...
        0x0030:  0073 ab9b 0000 3320 7365 6c65 6374 2022  .s....3.select."
        0x0040:  6262 6262 6262 6262 6262 625c 5c62 6262  bbbbbbbbbbb\\bbb
        0x0050:  6222 0d0a                                b"..
...
        0x0030:  007b aba4 0000 3420 6765 7471 756f 7461  .{....4.getquota
        0x0040:  726f 6f74 2022 6262 6262 6262 6262 6262  root."bbbbbbbbbb
        0x0050:  625c 5c62 6262 620d 0a0a 220d 0a         b\\bbbb..."..




Expected results:

I dug around a bit on comm-central and found a potential cause in nsImapServerResponseParser::PreProcessCommandToken. It appears that when an escape character is found, there is a strcpy used to do an overlapped copy:

http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapServerResponseParser.cpp#309

According to the C standard, this is undefined behavior (C99 §7.21.2.3 ¶2).

Otherwise, if I move the folder back to the top level, the messages re-appear.
I meant to post the thunderbird log (and I posted the relevant tcpdump packet data - both with names changed). Here is the relevant thunderbird log clip (pay particular attention to the NEWLINES present in the data as they ARE signficant):


2015-11-11 00:00:39.611972 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:ProcessCurrentURL: entering
2015-11-11 00:00:39.611988 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:ProcessCurrentURL:imap://<FNAME>%2EN%2E<LNAME>@<server>:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb:  = current
Url
2015-11-11 00:00:39.622658 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:SendData: 11 select "bbbbbbbbbbb\\bbbb"
2015-11-11 00:00:39.664159 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=12 needmore=0]
2015-11-11 00:00:39.664186 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * 5 EXISTS
2015-11-11 00:00:39.664207 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=12 needmore=0]
2015-11-11 00:00:39.664211 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * 0 RECENT
2015-11-11 00:00:39.664215 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=43 needmore=0]
2015-11-11 00:00:39.664229 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * OK [UNSEEN 1] Message 1 is first unseen
2015-11-11 00:00:39.664238 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=34 needmore=0]
2015-11-11 00:00:39.664242 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * OK [UIDVALIDITY 68] UIDs valid
2015-11-11 00:00:39.664247 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=37 needmore=0]
2015-11-11 00:00:39.664251 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * OK [UIDNEXT 6] Predicted next UID
2015-11-11 00:00:39.664257 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=52 needmore=0]
2015-11-11 00:00:39.664260 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * FLAGS (\Flagged \Seen \Answered \Deleted \Draft)
2015-11-11 00:00:39.664276 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=85 needmore=0]
2015-11-11 00:00:39.664280 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: * OK [PERMANENTFLAGS (\Flagged \Seen \Answered \Deleted \Draft \*)] Permanent flags
2015-11-11 00:00:39.664287 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=37 needmore=0]
2015-11-11 00:00:39.664291 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:A:CreateNewLineFromSocket: 11 OK [READ-WRITE] SELECT completed
2015-11-11 00:00:39.664332 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb

:SendData: 12 getquotaroot "bbbbbbbbbbb\\bbbb

"
2015-11-11 00:00:39.711497 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=50 needmore=0]
2015-11-11 00:00:39.711517 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb

:CreateNewLineFromSocket: 12 NO GETQUOTAROOT Invalid IMAP command argument
2015-11-11 00:00:39.711535 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb

:SendData: 13 UID fetch 1:* (FLAGS)
2015-11-11 00:00:39.711548 UTC - 1429194496[7f3a6c0d61a0]: ReadNextLine [stream=6be76f80 nb=22 needmore=0]
2015-11-11 00:00:39.711552 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb

:CreateNewLineFromSocket:  BAD unknown command
2015-11-11 00:00:39.714050 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:NA:SendData: 14 logout
OS: Unspecified → Linux
Hardware: Unspecified → x86_64
select command/getquotaroot command is generated here.
http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#8568
> 3153 void nsImapProtocol::SelectMailbox(const char *mailboxName)
> 
> 3160   nsCString escapedName;
> 3161   CreateEscapedMailboxName(mailboxName, escapedName);
> 3162   nsCString commandBuffer(GetServerCommandTag());
> 3163   commandBuffer.Append(" select \"");
> 3164   commandBuffer.Append(escapedName.get());
> 3165   commandBuffer.Append("\"");
http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#8568
> 8568 void nsImapProtocol::GetQuotaDataIfSupported(const char *aBoxName)
> 
> 8581   nsCString escapedName;
> 8582   CreateEscapedMailboxName(aBoxName, escapedName);
> 
> 8586   nsAutoCString quotacommand(GetServerCommandTag());
> 8587   quotacommand.Append(NS_LITERAL_CSTRING(" getquotaroot \""));
> 8588   quotacommand.Append(escapedName);
> 8589   quotacommand.Append(NS_LITERAL_CSTRING("\"" CRLF));

Following is needed?
  8588   quotacommand.Append(escapedName.get());

Confirming per tcpdump/NSPR log.
Status: UNCONFIRMED → NEW
Component: Untriaged → Networking: IMAP
Ever confirmed: true
Product: Thunderbird → MailNews Core
"extra new line" is 0x0D0A0A in this bug's case.
aBoxName already has 0x0D0A0A at end of string at gere?
  8582   CreateEscapedMailboxName(aBoxName, escapedName);
Tb issued 11 select "bbbbbbbbbbb\\bbbb", and log line/log heading is as follows.
  6f246800:<server>:S-bbbbbbbbbbb\bbbb(log line is split here. 0x0D0A0 here?)
  :SendData: 12 getquotaroot "bbbbbbbbbbb\\bbbb(0x0D0A0 exists here, because it's logged in tcpdump)
  "

This indicates that "Mboxname passed from server via LIST response or LSUB response" was "bbbbbbbbbbb\bbbb" or "bbbbbbbbbbb\\bbbb"(\ is escaped by \) instead of "modified utf-7 for string which contains \".

Mboxname data in IMAP is defined as "modified utf-7" to support non 7bis-ascii mboxname.
  http://tools.ietf.org/html/rfc3501#section-5.1.3
  http://tools.ietf.org/html/rfc2152

What string for mboxname does your IMAP server return to LIST/LSUB response?
FYI.
Following is NSPR log for internal request of "select mbox",
> 6f246800:<server>:A:ProcessCurrentURL:imap://<FNAME>%2EN%2E<LNAME>@<server>:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb: = current Url
> 6f246800:<server>:A:SendData: 11 select "bbbbbbbbbbb\\bbbb"
For Thunderbird, the mbox seems to correspnd to file named "\bbbbbbbbbbb\bbbb.msf" in file system of OS of your PC.
"top/first \" looks removed upon mboxname generation for IMAP command oor internal folder name in Tb.
"mid \" in file name(folder name in Tb) looks escaped by "\" when transfer to server as "mboxname to be selected".
"mapping string of bbbbbbbbbbb\\bbbb to which directory/file" is all up to IMAP server, and your server returned OK response to the IMAP select command.
To bug opener.
Please *attach* log file to this bug instead of *paste* to comment, if many long log lines are needed to know what occurs in your environment.
(In reply to ryan.martindale from comment #0)
> Steps to reproduce:
> When I place a folder underneath another folder, it no longer shows any messages (IMAP server).

Does it mean following?
- When mbox is /ABC/\bbbbbbbbbbb\bbbb in file system of your IMAP server, no problem occurs.
- When mbox is /\bbbbbbbbbbb\bbbb in file system of your IMAP server, problem of this bug occurs.

> This is using Thunderbird 38.2.0 (CentOS 6.7 64bit).

Does it mean following?
  No problem in previous release of Tb such as Tb 38.1.0, Tb 31.x.y. i.e. New problem in Tb 38.2.0.
(In reply to WADA from comment #2)
> select command/getquotaroot command is generated here.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.
> cpp#8568
> > 3153 void nsImapProtocol::SelectMailbox(const char *mailboxName)
> > 
> > 3160   nsCString escapedName;
> > 3161   CreateEscapedMailboxName(mailboxName, escapedName);
> > 3162   nsCString commandBuffer(GetServerCommandTag());
> > 3163   commandBuffer.Append(" select \"");
> > 3164   commandBuffer.Append(escapedName.get());
> > 3165   commandBuffer.Append("\"");

You are correct - but I'm not sure that is the "source" of the problem. I think the source of the problem may be in the actual storage of the name - where I pointed you to (and used by CreateEscapedMailboxName). Specifically, I am asserting that using strcpy to do an overlapped operation is BAD and has UNDEFINED behavior (which could produce a behavior such as I saw occur).

> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.
> cpp#8568
> > 8568 void nsImapProtocol::GetQuotaDataIfSupported(const char *aBoxName)
> > 
> > 8581   nsCString escapedName;
> > 8582   CreateEscapedMailboxName(aBoxName, escapedName);
> > 
> > 8586   nsAutoCString quotacommand(GetServerCommandTag());
> > 8587   quotacommand.Append(NS_LITERAL_CSTRING(" getquotaroot \""));
> > 8588   quotacommand.Append(escapedName);
> > 8589   quotacommand.Append(NS_LITERAL_CSTRING("\"" CRLF));
> 
> Following is needed?
>   8588   quotacommand.Append(escapedName.get());

No - the strcpy call needs to be replaced with something that can perform overlapped string copy (maybe just a simple "for" loop). The problem is that somebody assumed strcpy provided some sort of guarantee to work IF the two memory ranges overlap, but it DOES NOT.

> 
> Confirming per tcpdump/NSPR log.
(In reply to WADA from comment #6)
> To bug opener.
> Please *attach* log file to this bug instead of *paste* to comment, if many
> long log lines are needed to know what occurs in your environment.

Noted - I didn't think it was bad, but OK.
Attached file tb.tcpdump — 
This is a more informative communications breakdown between TB and the server (names replaced). It shows the TB request, and the server response. Originally, the folder starts just under the account, but is beneath a folder during the sequence of actions, at which point messages in that folder no longer show in Thunderbird. When the folder is moved back out to the top level (just under the account), then messages show in the folder again.
(In reply to WADA from comment #7)
> (In reply to ryan.martindale from comment #0)
> > Steps to reproduce:
> > When I place a folder underneath another folder, it no longer shows any messages (IMAP server).
> 
> Does it mean following?
> - When mbox is /ABC/\bbbbbbbbbbb\bbbb in file system of your IMAP server, no
> problem occurs.
> - When mbox is /\bbbbbbbbbbb\bbbb in file system of your IMAP server,
> problem of this bug occurs.
> 

Hopefully my previous attachment helps with this.

> > This is using Thunderbird 38.2.0 (CentOS 6.7 64bit).
> 
> Does it mean following?
>   No problem in previous release of Tb such as Tb 38.1.0, Tb 31.x.y. i.e.
> New problem in Tb 38.2.0.

I'm not sure how long this error may have existed. Just identifying what I have.
(In reply to ryan.martindale from comment #10)
> tb.tcpdump

 0x0030:  00e3 ab93 0000 3137 206c 7375 6220 2222  ......17.lsub.""
 0x0040:  2022 6262 6262 5c5c 2a22 0d0a            ."bbbb\\*"..

If delimiter is ".", lsub command is one like next.
  4 lsub "" "INBOX.*"    Parent_Mbox + Delimiter + WildCrd(* or %. if lsub, * is used by Tb)
What is actual mbox name/folder name?
Or mbox="bb...bb\\" and Tb doesn't add delimiter because delimiter="\"?
Or mbox="bb...bb\" and delimiter="\"?
Or mbox="bb...bb" and delimiter="\" is added and the "\" is escaped by "\"?  

 0x0030:  00e3 aba3 0000 3139 2072 656e 616d 6520  ......19.rename.
 0x0040:  2262 6262 6222 2022 6262 6262 6262 6262  "bbbb"."bbbbbbbb
 0x0050:  6262 625c 5c62 6262 6222 0d0a            bbb\\bbbb"..

What string did you type at rename dialog for "bbbbbbbb"?
"bbbbbbbbbbb\\bbbb"?
"bbbbbbbbbbb\bbbb"?

Anyway, tcpdump is mandatory to see 0x0D0A0A after ...bbbb before doublequot+CRLF in getquotaroot command.
However, imap log(NSPR log) is far better to know all other commands/responses such as capability, namespace, list, lsub etc.
Can you attach imap log file(NSPR log file)?
Attached file tb.log — 
(In reply to WADA from comment #12)
> (In reply to ryan.martindale from comment #10)
> > tb.tcpdump
> 
>  0x0030:  00e3 ab93 0000 3137 206c 7375 6220 2222  ......17.lsub.""
>  0x0040:  2022 6262 6262 5c5c 2a22 0d0a            ."bbbb\\*"..
> 
> If delimiter is ".", lsub command is one like next.
>   4 lsub "" "INBOX.*"    Parent_Mbox + Delimiter + WildCrd(* or %. if lsub,
> * is used by Tb)
> What is actual mbox name/folder name?
> Or mbox="bb...bb\\" and Tb doesn't add delimiter because delimiter="\"?
> Or mbox="bb...bb\" and delimiter="\"?
> Or mbox="bb...bb" and delimiter="\" is added and the "\" is escaped by "\"?  
> 
>  0x0030:  00e3 aba3 0000 3139 2072 656e 616d 6520  ......19.rename.
>  0x0040:  2262 6262 6222 2022 6262 6262 6262 6262  "bbbb"."bbbbbbbb
>  0x0050:  6262 625c 5c62 6262 6222 0d0a            bbb\\bbbb"..
> 
> What string did you type at rename dialog for "bbbbbbbb"?
> "bbbbbbbbbbb\\bbbb"?
> "bbbbbbbbbbb\bbbb"?
> 

I didn't actually type anything - I just dragged the folder from the account tree into another folder in the tree (just mouse clicks accompanied with mouse movement). The "bbbbbbbbbbb" folder exists throughout. The "bbbb" folder is simply dragged in under it and then back out.

> Anyway, tcpdump is mandatory to see 0x0D0A0A after ...bbbb before
> doublequot+CRLF in getquotaroot command.
> However, imap log(NSPR log) is far better to know all other
> commands/responses such as capability, namespace, list, lsub etc.
> Can you attach imap log file(NSPR log file)?

This attachment is an edited log fragment - its too hard to otherwise sanitize the entire log.

NOTE: I have added a bit of whitespace (newlines) manually to break it up into sections.
Hierarchy delimiter(single char) is passed by lsub response, but I can't see lsub response in your imap log.

If "Show only subscribed folders" is unchecked, list is issued and delimiter is returned.	
> bc400:imap.gmail.com:A:SendData: 11 list "" "INBOX" 	
> bc400:imap.gmail.com:A:CreateNewLineFromSocket: * LIST (\HasChildren) "/" "INBOX" 	
> bc400:imap.gmail.com:A:CreateNewLineFromSocket: 11 OK Success
IIRC, "list *" was issued by Thunderbird before lsub when "Show only subscribed folders" is checked.
But I'm not sure.
So I expected log for namespace, because I can't get your answer about "actual mboxname" and delimiter.
> 86bc400:imap.gmail.com:A:SendData: 4 namespace 	
> ReadNextLine [stream=867fb08 nb=32 needmore=0]	
> 86bc400:imap.gmail.com:A:CreateNewLineFromSocket: * NAMESPACE (("" "/")) NIL NIL 	
> ReadNextLine [stream=867fb08 nb=14 needmore=0]	
> 86bc400:imap.gmail.com:A:CreateNewLineFromSocket: 4 OK Success 	
And, "NAMESPCE" is returned to capability command in your case.

No such log in your original imap log?

Namesace response data is held in pres.js and is shown in Server Settings/Advanced panel.
What string is set as "namesace data"?

Because 42 lsub "" "bbbbbbbbbbb\\bbbb\\*" is seen in your imao log, and because "...@<server>:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb:" is seen in imap log, and because delimiter is "single character", I think that "\\" is delimiter="\" and the "\" is escaped by "\".
But I'm not sure, because I don't know about imap spec on "\ in mbox name hierarchy".

I can't understand reason of "two newlines" between "S-bbbbbbbbbbb\bbbb" and ":SendData: 12 getquotaroot ..." in imap log.　
> 2015-11-11 00:00:39.664332 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb
> 
> :SendData: 12 getquotaroot "bbbbbbbbbbb\\bbbb
> 
> "
If mbox=bbbb, <server>:S-bbbb:SendData: 12 getquotaroot "bbbb"[CRLF] (single log line).
Is "mboxname part of <server>:S-<mboxname>: in imap log" obtaained from finally sent imap command(getquotaroot in this case)?
It's set from "currently selected mbox at a cached connection", isn't it?
(In reply to WADA from comment #14)
> Hierarchy delimiter(single char) is passed by lsub response, but I can't see
> lsub response in your imap log.
> 
> If "Show only subscribed folders" is unchecked, list is issued and delimiter
> is returned.	

"Show only subscribed folders" for this account is checked.

> > bc400:imap.gmail.com:A:SendData: 11 list "" "INBOX" 	
> > bc400:imap.gmail.com:A:CreateNewLineFromSocket: * LIST (\HasChildren) "/" "INBOX" 	
> > bc400:imap.gmail.com:A:CreateNewLineFromSocket: 11 OK Success

Here's the beginning of the "list" request:

2015-11-12 16:49:38.775534 UTC - -1191184640[7f09b9eaf800]: b9c7f000:<server>:A:SendData: 4 list "" "*"
2015-11-12 16:49:38.814155 UTC - -1128270080[7f09b9eaf380]: ReadNextLine [stream=b9ebe880 nb=40 needmore=0]
2015-11-12 16:49:38.814186 UTC - -1128270080[7f09b9eaf380]: b9e69800:<server>:A:CreateNewLineFromSocket: * LIST (\HasChildren) "\\" bbbbbbbbbbb
2015-11-12 16:49:38.814232 UTC - -1128270080[7f09b9eaf380]: ReadNextLine [stream=b9ebe880 nb=35 needmore=0]

> IIRC, "list *" was issued by Thunderbird before lsub when "Show only
> subscribed folders" is checked.
> But I'm not sure.
> So I expected log for namespace, because I can't get your answer about
> "actual mboxname" and delimiter.
> > 86bc400:imap.gmail.com:A:SendData: 4 namespace 	
> > ReadNextLine [stream=867fb08 nb=32 needmore=0]	
> > 86bc400:imap.gmail.com:A:CreateNewLineFromSocket: * NAMESPACE (("" "/")) NIL NIL 	
> > ReadNextLine [stream=867fb08 nb=14 needmore=0]	
> > 86bc400:imap.gmail.com:A:CreateNewLineFromSocket: 4 OK Success 	
> And, "NAMESPCE" is returned to capability command in your case.
> 

Sorry I'm not understanding the terminology well. Does this give you what you need?

2015-11-12 16:49:38.774875 UTC - -1191184640[7f09b9eaf800]: ReadNextLine [stream=b9c43580 nb=78 needmore=0]
2015-11-12 16:49:38.774905 UTC - -1191184640[7f09b9eaf800]: b9c7f000:<server>:A:CreateNewLineFromSocket: * NAMESPACE (("" "\\")) (("Other Users\\" "\\")) (("Public Folders\\" "\\"))
2015-11-12 16:49:38.775104 UTC - -1191184640[7f09b9eaf800]: ReadNextLine [stream=b9c43580 nb=26 needmore=0]
2015-11-12 16:49:38.775128 UTC - -1191184640[7f09b9eaf800]: b9c7f000:<server>:A:CreateNewLineFromSocket: 3 OK NAMESPACE completed

> No such log in your original imap log?
> 
> Namesace response data is held in pres.js and is shown in Server
> Settings/Advanced panel.
> What string is set as "namesace data"?
> 

The "Personal namespace" is set to "".

> Because 42 lsub "" "bbbbbbbbbbb\\bbbb\\*" is seen in your imao log, and
> because "...@<server>:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb:" is seen in imap
> log, and because delimiter is "single character", I think that "\\" is
> delimiter="\" and the "\" is escaped by "\".
> But I'm not sure, because I don't know about imap spec on "\ in mbox name
> hierarchy".
> 
> I can't understand reason of "two newlines" between "S-bbbbbbbbbbb\bbbb" and
> ":SendData: 12 getquotaroot ..." in imap log.　
> > 2015-11-11 00:00:39.664332 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb
> > 
> > :SendData: 12 getquotaroot "bbbbbbbbbbb\\bbbb
> > 
> > "
> If mbox=bbbb, <server>:S-bbbb:SendData: 12 getquotaroot "bbbb"[CRLF] (single
> log line).
> Is "mboxname part of <server>:S-<mboxname>: in imap log" obtaained from
> finally sent imap command(getquotaroot in this case)?
> It's set from "currently selected mbox at a cached connection", isn't it?

I'm not sure I understand what your asking here. The sequence of commands I was showing in the last attachment was dragging the folder into another folder and then dragging the folder back out. Are you asking about the final quota request in that attachment produced by me dragging the folder back out to the top level (where everything works)?
(In reply to ryan.martindale from comment #15)
> Here's the beginning of the "list" request:
> b9e69800:<server>:A:CreateNewLineFromSocket: * LIST (\HasChildren) "\\"
> bbbbbbbbbbb

Thanks. Hierarchy delimiter="\", and the "\" in quoted-string of imap is escaped by "\".

> > I can't understand reason of "two newlines" between "S-bbbbbbbbbbb\bbbb" and
> > ":SendData: 12 getquotaroot ..." in imap log.　
> > > 2015-11-11 00:00:39.664332 UTC - 1429194496[7f3a6c0d61a0]: 6f246800:<server>:S-bbbbbbbbbbb\bbbb
> > > 
> > > :SendData: 12 getquotaroot "bbbbbbbbbbb\\bbbb
> > > 
> > > "
> > If mbox=bbbb, <server>:S-bbbb:SendData: 12 getquotaroot "bbbb"[CRLF] (single
> > log line).
> > Is "mboxname part of <server>:S-<mboxname>: in imap log" obtaained from
> > finally sent imap command(getquotaroot in this case)?
> > It's set from "currently selected mbox at a cached connection", isn't it?
> 
> I'm not sure I understand what your asking here. The sequence of commands I
> was showing in the last attachment was dragging the folder into another
> folder and then dragging the folder back out.
> Are you asking about the final quota request in that attachment produced by me dragging the folder back out
> to the top level (where everything works)?

No.
My question was;
(a) 'getquotaroot "bb...bb\\bbbb"[CRLF]' is preprocessed,
    and it is changed to 'getquotaroot "bb...bb\\bbbb<0x0D0A0A>"[CRLF]',
    then :S-bb...bb\\bbbb<0x0D0A0A>[CRLF]:SendData ... is logged.
    (getquotaroot only problem.)
(b) After end of select "bb...bb\\bbbb" command
    which was generaated from ProcessCurrentURL:imap://...:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb:,
    "mbox name internally held for the cached connection" is set to "bb...bb\bbbb<0x0D0A0A>",
    then 'getquotaroot "bb...bb\\bbbb<0x0D0A0A>"[CRLF]' is issued by Tb,
    and :S-bb...bb\\bbbb<0x0D0A0A>[CRLF]:SendData ... is logged.
    (not getquotaroot only problem. it's merely that getquotaroot is issued first after select.)

Which occurred?

By the way, what is your purpose of hierarchy delimiter="\"?
Please note that "\" and "~" is omited in Set O in followinf RFC.
  http://tools.ietf.org/html/rfc2152
    Set O
    Rationale. The characters "\" and "~" are omitted because they are
    often redefined in variants of ASCII.
And, most popular hierarchy delimiter is "/" and "." is used by some imap software/servers,
And, AFAIK, even MS's imap software on Windows Server, hierarchy delimiter is not "\".
(In reply to WADA from comment #16)
> (In reply to ryan.martindale from comment #15)
...
> > Are you asking about the final quota request in that attachment produced by me dragging the folder back out
> > to the top level (where everything works)?
> 
> No.
> My question was;
> (a) 'getquotaroot "bb...bb\\bbbb"[CRLF]' is preprocessed,
>     and it is changed to 'getquotaroot "bb...bb\\bbbb<0x0D0A0A>"[CRLF]',
>     then :S-bb...bb\\bbbb<0x0D0A0A>[CRLF]:SendData ... is logged.
>     (getquotaroot only problem.)
> (b) After end of select "bb...bb\\bbbb" command
>     which was generaated from
> ProcessCurrentURL:imap://...:143/select%3E%5Cbbbbbbbbbbb%5Cbbbb:,
>     "mbox name internally held for the cached connection" is set to
> "bb...bb\bbbb<0x0D0A0A>",
>     then 'getquotaroot "bb...bb\\bbbb<0x0D0A0A>"[CRLF]' is issued by Tb,
>     and :S-bb...bb\\bbbb<0x0D0A0A>[CRLF]:SendData ... is logged.
>     (not getquotaroot only problem. it's merely that getquotaroot is issued
> first after select.)
> 
> Which occurred?
> 

Looking at the TB log file in a hex editor, there is the entry that you note:

2015-11-12 16:49:51.457797 UTC - -1281435904[7f09b9eb1ae0]: b9e62000:<server>:S-bbbbbbbbbbb\bbbb<CR><LF><LF>:SendData: 4 getquotaroot "bbbbbbbbbbb\\bbbb<CR><LF><LF>"

followed by subsequent entries:

2015-11-12 16:49:51.547813 UTC - -1281435904[7f09b9eb1ae0]: b9e62000:<server>:S-bbbbbbbbbbb\bbbb<CR><LF><LF>:CreateNewLineFromSocket: 4 NO GETQUOTAROOT Invalid IMAP command argument
2015-11-12 16:49:51.547850 UTC - -1281435904[7f09b9eb1ae0]: b9e62000:<server>:S-bbbbbbbbbbb\bbbb<CR><LF><LF>:SendData: 5 UID fetch 1:* (FLAGS)
2015-11-12 16:49:51.547881 UTC - -1281435904[7f09b9eb1ae0]: ReadNextLine [stream=ba4d1e00 nb=22 needmore=0]
2015-11-12 16:49:51.547890 UTC - -1281435904[7f09b9eb1ae0]: b9e62000:<server>:S-bbbbbbbbbbb\bbbb<CR><LF><LF>:CreateNewLineFromSocket:  BAD unknown command

followed by a logout:

2015-11-12 16:49:51.555452 UTC - -1281435904[7f09b9eb1ae0]: b9e62000:<server>:NA:SendData: 6 logout

I would think it would imply condition (b) because the name keeps printing with <CR><LF><LF> for the log message location in subsequent queries (not just in the data) until the logout occurs.

> By the way, what is your purpose of hierarchy delimiter="\"?
> Please note that "\" and "~" is omited in Set O in followinf RFC.
>   http://tools.ietf.org/html/rfc2152
>     Set O
>     Rationale. The characters "\" and "~" are omitted because they are
>     often redefined in variants of ASCII.
> And, most popular hierarchy delimiter is "/" and "." is used by some imap
> software/servers,
> And, AFAIK, even MS's imap software on Windows Server, hierarchy delimiter
> is not "\".

I'm not choosing these settings - it is a corporate server (IBM's Lotus Notes - if I'm not mistaken). I don't have any control over it.
(In reply to ryan.martindale from comment #17)
> > And, most popular hierarchy delimiter is "/" and "." is used by some imap
> > software/servers,
> > And, AFAIK, even MS's imap software on Windows Server, hierarchy delimiter
> > is not "\".
> I'm not choosing these settings - it is a corporate server (IBM's Lotus
> Notes - if I'm not mistaken). I don't have any control over it.

Oh, Lotus Domino on Intel/Win machine instead of on *nix machine...
I recommend to stop using Win and to change to *nix on Intel or Power or Z or S/38 :-)
FYI.
"\\" in this bug is merely an simplest example of pretty basic rule of "escaping back-slash and double-quote in quoted" which is defined by rfc 3501.          
http://tools.ietf.org/html/rfc3501#section-9
  quoted          = DQUOTE *QUOTED-CHAR DQUOTE
  QUOTED-CHAR     = <any TEXT-CHAR except quoted-specials> /
                    "\" quoted-specials
  quoted-specials = DQUOTE / "\"
  TEXT-CHAR       = <any CHAR except CR and LF>

String of "bb...bb\\bbbb<0x0D0A0A>" in imap getquotarot commad is apparent RFC 3501 vilatin by Thunderbird.
Blocks: folders-with-special-characters
Summary: Newlines are being inserted into IMAP request for quota → Newlines are being inserted into IMAP request for quota (when hierarchy delimiter is "\"=BackSlash, Tb appends 0x0D0A0A to mboxname in quoted text part of IMAP command)inserts
Flow like following?

> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapUrl.cpp#422
> 422 void nsImapUrl::ParseImapPart(char *imapPartOfUrl)
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapUrl.cpp#588
> 588       m_imapAction = nsImapEnsureExistsFolder;
> =>
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#7931
> 7931     case nsIImapUrl::nsImapEnsureExistsFolder:
> 7932       sourceMailbox = OnCreateServerSourceFolderPathString();
> 7933       OnEnsureExistsFolder(sourceMailbox);
> 7934       break;
> =>
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#6251
> 6251 void nsImapProtocol::OnEnsureExistsFolder(const char * aSourceMailbox)
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#6284
> 6284     bool created = CreateMailboxRespectingSubscriptions(aSourceMailbox);
> =>
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#7498
> 7498 bool nsImapProtocol::CreateMailboxRespectingSubscriptions(const char *mailboxName)
> 7508     RemoveHierarchyDelimiter(mailboxWODelim);
> =>
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#7487
> 7487 void nsImapProtocol::RemoveHierarchyDelimiter(nsCString &mailboxName)
> 7488 {
> 7489   char onlineDelimiter[2] = {0, 0};
> 7490   if (m_imapMailFolderSink)
> 7491     m_imapMailFolderSink->GetOnlineDelimiter(&onlineDelimiter[0]);
> 7492   // take the hierarchy delimiter off the end, if any.
> 7493   if (onlineDelimiter[0])
> 7494     mailboxName.Trim(onlineDelimiter, false, true);
> 7495 }

(1) Because hiearachy delimiter="\" == escape character in string processing,
    <strng>.Trim generates 0x0D0A0A at end of mailboxName.
(2) When "select" command, mailboxName is not selected at cached connection,
    and "Mmox name in select request" is used fr select command.
(3) After normal end of select command at a caached connection,
    mailboxName generated by (1) is used.
Summary: Newlines are being inserted into IMAP request for quota (when hierarchy delimiter is "\"=BackSlash, Tb appends 0x0D0A0A to mboxname in quoted text part of IMAP command)inserts → Newlines are being inserted into IMAP request for quota (when hierarchy delimiter is "\"=BackSlash, Tb appends 0x0D0A0A to mboxname in quoted text part of IMAP command)
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: