513247 - Search Messages (advanced search) message body contains is slow. Search could be faster by checking other criteria first, like date

Status: RESOLVED DUPLICATE of bug 365727

(MailNews Core :: Search, defect)

Description

[fuggi]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)
Build Identifier: 2.0.0.23 (20090812)

Assuming, that a search of a certain e-mail by a message content condition in a local message folder (using the search window ([Ctrl]+[Shift]+[F])) takes x seconds, the addition of a date constraint (e.g. date after 01.01.2009) should decrease the required time for a new search - but it does not. The new search still takes x seconds. Hence, it looks as if the new search still processes all messages of that folder anyway. Hence, searching in message bodies is always extremely slow.

Reproducible: Always

Steps to Reproduce:
1. Press [Ctrl]+[Shift]+[F].
2. Select a local folder, enter a message text search condition, and start the search.
3. Add a date search condition and start the search again.
Actual Results:  
slow search operation

Expected Results:  
Limit processed messages to that of the specified date range and, hence, speed up search time drastically.

Comment 1

[Kent James (:rkent)]

I'm in the middle of looking at some issues in combined searches. I think the search proceeds in order, with the first terms evaluated first, and searching stops when it is should. So try your search entering date first, then body, and see if it makes a difference.

Not that we should demand that of people. It would probably be possible to automatically reorder search terms, so that body search is always at the end of any AND or OR group.

Comment 2

[fuggi]

@Kent:
Yes, you are right. If the date condition is the first one, the search operation is processed as expected. But as you have already pointed out, it's not the user who should care about the order of search conditions.

Comment 3

[Thomas D. (:thomas8)]

This blocks related bug 383895:
Thunderbird Message Body/fulltext search or filter is painfully slow compared to Outlook on folders with lots of messages (both "quick search" and "search messages")

This bug is NOT a duplicate of bug 383895:
- This bug is about multiple search criteria (including body), where we are searching in the wrong places (messages that are excluded by other criteria) and in the wrong order (which slows us down)
- Bug 383895 is about the general slowness of body/fulltext search as such

Comment 4

[Magnus Melin [:mkmelin]]

Just a quick note about a possible fix. I'd imagine the terms should be reordered before doing nsMsgSearchSession::BeginSearching. http://mxr.mozilla.org/comm-central/source/mailnews/base/search/src/nsMsgSearchSession.cpp#434

Comment 5

[Wayne Mery (:wsmwk)]

slow <> major
https://bugzilla.mozilla.org/page.cgi?id=fields.html#importance

Comment 6

[Wayne Mery (:wsmwk)]

fixing this bug does not improve bug 383895 unless it is done via gloda (which isn't necessary for the usecase described), so moving 383895 from depends to related

Comment 7

[Felix Pahl]

I agree the search conditions should be reordered. There have been a number of related bugs (bug 166502, bug 295088), which are marked as fixed, but the search conditions are currently not being reordered.

Comment 8

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.