Closed Bug 1531317 Opened 2 years ago Closed 1 year ago

Add a way to search for email (browser.messages.query)

Categories

(Thunderbird :: Add-Ons: Extensions API, enhancement)

enhancement
Not set
normal

Tracking

(thunderbird_esr6870+ fixed, thunderbird69 fixed)

RESOLVED FIXED
Thunderbird 69.0
Tracking Status
thunderbird_esr68 70+ fixed
thunderbird69 --- fixed

People

(Reporter: Fallen, Assigned: hermar05)

References

Details

Attachments

(7 files, 4 obsolete files)

In bug 1488176 comment 27 et al Geoff and I talked about a way to query messages. It looks like we never filed a followup for this.

The search API should be modeled in a way that we can use gloda to speed up searches, but at the same time also make it work when gloda is disabled. an index: false option could allow manually disabling gloda, or true to explicitly request it.

Conceptually, it could be similar to the browser.tabs.query() method, but suitable for mail. The responses should use message lists.

Hermar was looking into writing a gloda based experiment for this, I'd love for us to work with him to make what he is working on suitable for adding to Thunderbird.

Any progress?

Flags: needinfo?(hermar05)

logic is done for searching with gloda through an API.
I will be adding promises today and then upload a version...
"at the same time also make it work when gloda is disabled" still needs to be made though...

Facing a problem with Gloda parameters:
"Cannot query on attribute from because its canQuery parameter hasn't been set to true."
otherwise complete

Flags: needinfo?(hermar05) → needinfo?(geoff)

Please attach what you've done so far. I can't see what's going on (or not) without it.

Flags: needinfo?(geoff)

part 1/4

part 2/4

Attached file manifest of the gloda API (obsolete) β€”

part 3/4

Attached file schema of the Gloda API (obsolete) β€”

some are not working and therefore needs to be checked!
part 4/4

My understanding of GloDa is very limited, but I don't think you can just pass a string to most of the constraining functions. I got "involves" working for example, but only because I took an existing identity object from an earlier query.

You were looking for a list of things you could query, this might help: https://hg.mozilla.org/comm-central/file/tip/mailnews/db/gloda/modules/fundattr.js#l59. I think you should be able to do most of the ones with objectNoun set to NOUN_FULLTEXT, NOUN_NUMBER, and NOUN_STRING. It looks like authorMatches, subjectMatches, etc. are going to be the most useful ones.

Depends on: 1546840

Well I guess I am done...
I finished the schema based on those: https://hg.mozilla.org/comm-central/file/tip/mailnews/db/gloda/modules/fundattr.js#l59 and
tested the possible options within the Developers Toolbox with Geoffs help.
He confirmed that the "read" option is not working so I removed it from the schema (should it be possible in the near future it would only be adding a couple of rows to the schema so that others may know its possible).

I would like some feedback regarding the "implementation" and the status of "read" though...
(@Geoff: I found that it was slow because I made log entries whenever I added something to the collection, after I removed it I was, in my opinion, working ok)

Flags: needinfo?(geoff)
Attached file schema.json β€”

I think your implementation's good in theory but practically it has some problems. I think it's probably best if you unroll that forEach loop and handle each query constraint individually. Some names and types in the schema aren't going to line up with what Gloda expects.

Here's a schema that's roughly what I'd do. I've used some existing types from other built-in schema. Thoughts:

  • folder: I think Gloda expects a single nsIMsgFolder, which you can get from FolderManager.get. It would be nice if more than one folder could be used but if it doesn't work let's not attempt that now.
  • fromDate and toDate: the type I used should give you a Date object, but I've not used it before so don't quote me on that. If one date is specified but not the other, fill in the other with something sensible.
  • author, recipients, subject, and body can use their "*Matches" constraints, though they seem to be case-sensitive which is annoying
  • flagged: Gloda calls this starred but we're using flagged for WebExtensions

I've left out read, tags, and attachments for now. Read still seems to be broken, and the others are complicated. Let's get the rest going first.

Flags: needinfo?(geoff)
Attachment #9058895 - Attachment is obsolete: true
Attachment #9058896 - Attachment is obsolete: true
Attachment #9058898 - Attachment is obsolete: true
Attachment #9058899 - Attachment is obsolete: true

Here's Marvin's API (with some minor changes) in the form of a patch. Requesting review from someone else because I've been involved a lot here.

Attachment #9073161 - Flags: review?(mkmelin+mozilla)

And some tests. I had a major fight with GloDa over this, but I won in the end.

Attachment #9073162 - Flags: review?(mkmelin+mozilla)
Attachment #9073161 - Flags: review?(mkmelin+mozilla) → review+
Attachment #9073162 - Flags: review?(mkmelin+mozilla) → review+

Well done Marvin. We'll get this into the tree then we can follow up any remaining issues (just read = true/false, I think) in a new bug.

Keywords: checkin-needed

Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/14390389783d
Message search WebExtension API. r=mkmelin
https://hg.mozilla.org/comm-central/rev/450a66aeeaa8
Message search WebExtension API - tests. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Keywords: checkin-needed
Resolution: --- → FIXED

You want this on TB 68 beta and later ESR?

Target Milestone: --- → Thunderbird 69.0

Ultimately yes, I'm hopeful of finishing some remaining things quickly then it can all go at once.

Attachment #9099798 - Flags: approval-comm-esr68?
Attachment #9099799 - Flags: approval-comm-esr68?
Attachment #9099798 - Flags: approval-comm-esr68? → approval-comm-esr68+
Attachment #9099799 - Flags: approval-comm-esr68? → approval-comm-esr68+

author, recipients, subject, and body can use their "*Matches" constraints, though they seem to be case-sensitive which is annoying

Do these support wildcards?

Flags: needinfo?(hermar05)

I am sorry I kind of forgot to answer.

First: I just checked myself... author, reciepients, subject and body do not care for case-sensitivity you may write mArViN and he still finds the mails I send, recieved or those in which my name is in the subject or the body.

Regarding support for wildcards: they do not the reason is the four query options which you mentioned expect a value of string type. "*" would only search for said star in the option which you chose... you may work around that with {"subject":""} or {"author":""} wich will lead to a whole lot of mails but I can not guarantee that those are all available ones...

I hope that this answer somehow helps you.. otherwise ask away I will try to answer timely...

Flags: needinfo?(hermar05)

Thanks Marvin.

You need to log in before you can comment on or make changes to this bug.