500272 - Message advanced search using "contains" fails to match substrings of email addresses (IMAP online search, Gmail IMAP doesn't support SEARCH header properly yet)

Status: NEW

(Thunderbird :: Search, defect)

Description

[Matt Carter]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11
Build Identifier: 2.0.0.21 (20090302)

When I search for all messages whose "To: " or "Cc: " header lines contain a particular string, the search fails even though that string occurs (as a proper substring) in the "To: " or "Cc: " header lines of one or more messages.

Reproducible: Always

Steps to Reproduce:
1. Start Thunderbird's main mail window front end.
2. Note that there is a message in the inbox with this header:
      To: "'Humpty Dumpty'" <[e-mail]>
2. Right-click on "Inbox" in the left pane.
3. Select "Search..."
4. Specify "To or cc" "contains" "dcmtn".
5. Click "Search"

Actual Results:  
The search dialog reports "No matches found" on the bottom status line.

Expected Results:  
The search should find the message that has "[e-mail]" as a "To: " address.

The Inbox is located on an IMAP server (Gmail, to be exact).

Comment 1

[WADA:World Anti-bad-Duping Agency]

Following is response to SEARCH command from Gmail IMAP. 
(A) Gmail IMAP looks to interpret TO "dcmt" as 'TO is dcmt' instead of 'TO contains dcmt'.
(B) Gmail IMAP looks to interpret TO "[e-mail]" as 'TO or CC is [e-mail]'.

Matt Carter, can you check with ordinal IMAP server other than Gmail IMAP?

(UID=1588, No CC: header)
  From: aaa-1 aaa-1 <aaa-1@a.a.a>
  To: "'Humpty Dumpty'" <[e-mail]>
  Subject: Test
(UID=1589, with CC: header)
  From: aaa-1 aaa-1 <aaa-1@a.a.a>
  To: [e-mail]
  CC: "'Humpty Dumpty'" <[e-mail]>
  Subject: Test-2

> SendData: 12 uid SEARCH UNDELETED TO "dcmt"
> CreateNewLineFromSocket: * SEARCH 
> CreateNewLineFromSocket: 12 OK SEARCH completed (Success) 
> SendData: 13 uid SEARCH UNDELETED TO "[e-mail]" 
> CreateNewLineFromSocket: * SEARCH 1588 1589 
> CreateNewLineFromSocket: 13 OK SEARCH completed (Success) 
> SendData: 14 uid SEARCH UNDELETED CC "dcmt" 
> CreateNewLineFromSocket: * SEARCH 
> CreateNewLineFromSocket: 14 OK SEARCH completed (Success) 
> SendData: 15 uid SEARCH UNDELETED CC "[e-mail]" 
> CreateNewLineFromSocket: * SEARCH 1589 
> CreateNewLineFromSocket: 15 OK SEARCH completed (Success) 
> SendData: 16 uid SEARCH UNDELETED OR TO "dcmt" HEADER CC "dcmt" 
> CreateNewLineFromSocket: * SEARCH 
> CreateNewLineFromSocket: 16 OK SEARCH completed (Success) 
> SendData: 17 uid SEARCH UNDELETED OR TO "[e-mail]" HEADER CC "[e-mail]" 
> CreateNewLineFromSocket: * SEARCH 1588 1589 
> CreateNewLineFromSocket: 17 OK SEARCH completed (Success) 

Definition by RFC 3501; ( http://tools.ietf.org/html/rfc3501#section-6.4.4 )
> 6.4.4. SEARCH Command
> HEADER <field-name> <string>
>         Messages that have a header with the specified field-name (as
>         defined in [RFC-2822]) and that contains the specified string
>         in the text of the header (what comes after the colon).  If the
>         string to search is zero-length, this matches all messages that
>         have a header line with the specified field-name regardless of
>         the contents.
> CC <string> Messages that contain the specified string in the envelope structure's CC field.
> TO <string> Messages that contain the specified string in the envelope structure's TO field.

By the way, you can easily view IMAP flow by your self.
> https://wiki.mozilla.org/MailNews:Logging

Comment 2

[WADA:World Anti-bad-Duping Agency]

Oh, it was already known Gmail IMAP bug.
> http://mail.google.com/support/bin/answer.py?hl=en&answer=78761
> Does Gmail support all IMAP features?
>   # Substring search. All searches are assumed to be words.
>   # Searching arbitrary headers. Only some headers are available for searches: From/CC/BCC/To/Subject.

Comment 3

[Matt Carter]

I have tried the steps-to-reproduce on an IMAP server other than Gmail.  I confirmed that the problem does not occur on the other IMAP server; it only occurs on Gmail.

Comment 4

[Matt Carter]

I have discovered a work-around for this problem:
Instruct Thunderbird to download your mail folder, then work offline.
Then, all searches are performed locally by Thunderbird rather than by the (Gmail) IMAP server.
This way, searches that involve substring matches work correctly.
After completing your search, switch Thunderbird back to online mode.

Comment 5

[WADA:World Anti-bad-Duping Agency]

If folder of offline-use=yes, Saved Search Folder(Virtual Folder) with "search online=off" can be used as a workaround too.
  Merit:   No need to enter Offline mode.
  Demerit: Editing of conditions & re-choosing target folders is annoying.

Comment 6

[Wayne Mery (:wsmwk)]

I selected "online" for a gmail account, but I didn't do a protocol log to check that the online search was actually done.

Stephen, is it behaving properly for you?

Comment 7

[Stephen Rasku]

Sorry, Wayne.  I don't use Thunderbird on Windows any more so I am not in a position to answer your question.

Comment 8

[sanarturo]

I have this problem as well, that with Gmail IMAP, searches do not work, at least when searching for senders.

E.g., I have many emails from the domain 'g2.com', but while nothing is returned with a sender from that domain, I do get hundreds of other emails, and a spot search in the message source found not even 'g2', much less 'g2.com'.

Interestingly, a search for 'g2' returns NOTHING.