1352145 - Problem with global search containing percent sign (%)

Status: UNCONFIRMED

(Thunderbird :: Search, defect)

Description

[compunologist]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20170323105023

Steps to reproduce:

Execute global search for emails with body or subject text containing the percent sign.


Actual results:

Messages with a percent sign "%" in the subject or body text are not displayed in the global search result.

The concerning emails however CAN be found by right clicking a folder-->Search messages-->Match all of the following-->Subject (or Body)-->..%...-->Search




Expected results:

Concerning emails should have been found by global search.

I have tried rebuilding the Global Database (Gloda): exit Thunderbird, deleting global-messages-db.sqlite and restart Thunderbird but to no avail. It seems the percent sign handling should be improved.

Comment 1

[Jorg K (CEST = GMT+2)]

Global search (Gloda) and "normal" search are completely different. The former indexes messages into an SQLite database and then performs searches - using SQL - on that database, the latter searches the messages themselves.

It's well possible that SQL searches in Gloda using a % character don't work since that character is a wildcard character in SQL. Have you tried escaping that character? So search for \% or "%". Sorry, I'm not very familiar with Gloda and the original author is usually busy elsewhere.

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Global search is built on a tokenizer. "%" is currently treated as punctuation that delimits tokens and so never can be used to search directly.  If you search for "100%" the token "100" is what is searched for.  That can match again "100% fun" or "100 hats".  There's also the issue that there's a minimum token size, so entering "10%" produces no token because a token needs to be at least 3 characters long (unless dealing with CJK characters).

There's a variety of things that could be done here from specializing the tokenizer to recognize "%" as significant or being smarter about saying "global search can't actually search for that, do you want to try a cross-folder search."  There are probably dupes available for at least the latter; I'd at least check some of Thomas D.'s excellent search bugs and meta bugs, he's done excellent work at characterizing potential enhancements for gloda.

Note that Jorg is right about my involvement level here; I haven't been involved for quite some time and am now stepping away entirely.