Closed Bug 1541378 Opened 6 years ago Closed 6 years ago

Right bracket ")" in keyword field causes IMAP PARSER Internal Syntax Error

Categories

(MailNews Core :: Networking: IMAP, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: ewaerpok, Unassigned)

Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

Attempt IMAP sync of folder containing message with right bracket character ")" in keyword field (Exchange 2007 Category) via local workstation DavMail Gateway 5.2.0-2961 using EWS protocol.

Actual results:

Sync process stops when FETCH-ing the message with right bracket character.
IMAP Protocol Log reports "PARSER:Internal Syntax Error on line: %s". Full text copied further below.
Removing the message or changing the Category causes sync process to complete successfully.
No issues when Category contains only left bracket character "(" - or other non-alphanumeric characters including ampersand &, plus +, front-slash /, space , full-stop ., or hyphen -.

IMAP Protocol Log record:
[1128:Unnamed thread 1B1F9AC0]: I/IMAP 1BC0E000:localhost:S-test:CreateNewLineFromSocket: * 1 FETCH (UID 3 RFC822.SIZE 3002 FLAGS (\Seen (test)) BODY[HEADER.FIELDS (FROM TO CC BCC SUBJECT DATE MESSAGE-ID PRIORITY X-PRIORITY REFERENCES NEWSGROUPS IN-REPLY-TO CONTENT-TYPE REPLY-TO)] {398}
[1128:Unnamed thread 1B1F9AC0]: I/IMAP 1BC0E000:localhost:S-test:PARSER:Internal Syntax Error on line: %s: * 1 FETCH (UID 29 RFC822.SIZE 3002 FLAGS (\Seen (test)) BODY[HEADER.FIELDS (FROM TO CC BCC SUBJECT DATE MESSAGE-ID PRIORITY X-PRIORITY REFERENCES NEWSGROUPS IN-REPLY-TO CONTENT-TYPE REPLY-TO)] {398}

Expected results:

Presumably, the expected outcome should be for this bracket to be escaped somehow to enable completion of the sync process and download all messages in the folder, with Protocol Log records similar to the following:

[1128:Unnamed thread 1B1F9AC0]: I/IMAP 1BC0E000:localhost:S-test:CreateNewLineFromSocket: * 1 FETCH (UID 3 FLAGS (\Seen (test-START-ESCAPE)END-ESCAPE))
[1128:Unnamed thread 1B1F9AC0]: D/IMAP ReadNextLine [stream=19056B00 nb=26 needmore=0]
[1128:Unnamed thread 1B1F9AC0]: I/IMAP 1BC0E000:localhost:S-test:CreateNewLineFromSocket: * 8 OK UID FETCH completed

Component: Untriaged → Networking: IMAP
Product: Thunderbird → MailNews Core

According to my reading of the IMAP spec, ')' in a keyword is not allowed. But tb does allow the user to create a keyword (tag) containing '(' or ')' but it replaces it with '_' or sometimes '-' it seems, so that the IMAP is requirements are not violated at the server. Tb still displays the keyword locally as entered so that keyword "_test_" shows as "(test)" in the UI. I haven't looked at the code or docs for any more information on rational for this.

However, it appears that the reporter's invalid imap keywords are being created by another program (DavMail, EWS protocol?). Therefore, since tb is apparently trying to obey the RFC 3501 for valid keyword chars, this is not really a bug since it is a syntax error in that more than one closing ')' and possibly more than one opening '(' is seen in the imap response line.

I'm not sure if reporter is saying that only the message with the "bad" keyword is affected or does it stop the download of all messages in the mailbox, even ones with completely valid or no keywords?

This stops the download of any and all messages in the folder, even ones with completely valid or no keywords. To clarify further, it only stops the download of messages in the folder(s) containing the affected message - not all messages in the mailbox. That is, all other folders (those containing only messages with no "bad" keywords) are downloaded in full - including sub-folders of affected folder(s).

I have now found confirmation of this as a known bug in DavMail at https://sourceforge.net/p/davmail/bugs/642/

The bug report was closed with a workaround - to manually create mappings. So I have asked the DavMail project to consider implementing automatic mapping for invalid IMAP keywords: https://sourceforge.net/p/davmail/discussion/644056/thread/a1497174cd/

Whilst there is always likely to be scope for invalid keywords to be presented from this or any other IMAP server, could there be scope for Thunderbird / MailNews Core to handle them differently?

I think having ( or ) in a keyword/tag is really bad since they are also used in the imap protocol as delimiters. It would be hard to know what they mean. So the DavMail gateway should really fix that problem. So I will mark this tb bug report as invalid.

However, tb probably shouldn't just give up on all messages in a mbox when one has bad syntax. Not sure what other "bad" tag chars or other items cause this problem, if any. Not sure how hard this would be to fix but probably not easy. I haven't seen other tb bug reports about this. If, after DavMail fixes this, and you see that other invalid tag chars cause a problem for the whole mbox, please open another tb bug report for that.

Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.