741723 - Global search isn't returning results prior to 2010

Status: RESOLVED DUPLICATE of bug 537856

(Thunderbird :: Search, defect)

Description

[Ludovic Hirlimann [:Usul]]

I have this nice email from Davida in one of my mailboxes. It dates from 2008 and it's subject is contract.

If I search using gloda for contract - I don't find it even if I refine with the sender or/and with the folder it's in. The timeline above only shows 2010 and 2011 it doesn't show any 2009 nor 2008 dates. It's not the only email which produces this issue.
 
I've installed Rkent's glodaquilla and the email in question has a gloda id of 402b and a dirty column set to 0.

My gloda database is quite big :
ls -lh global-messages-db.sqlite
-rw-r--r--  1 ludo  staff   1.2G Apr  3 10:55 global-messages-db.sqlite

There is nothing in the error console.

Where do I go on from here to debug this issue ?

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Bug 537856.  This is just our hard-coded LIMIT of 400 results on fulltext searches that is misleading because the deduplication tends to make the number look less big and round.  Like 387 instead, which looks much less like an artificial limit.

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

and by "just", I don't mean that this is a good thing.  I mean that it's a very annoying-to-users thing, but it's not a new regression or data corruption or anything.

Comment 3

[Ludovic Hirlimann [:Usul]]

(In reply to Andrew Sutherland (:asuth) from comment #1)
> Bug 537856.  This is just our hard-coded LIMIT of 400 results on fulltext
> searches that is misleading because the deduplication tends to make the
> number look less big and round.  Like 387 instead, which looks much less
> like an artificial limit.

In my case it displays : 10 of 223 so it only finds 223 right (or because I have gmail accounts that could be more but with duplicates removed). I find 223 really far off the 400 limit.

And arg it's hard coded so I can't change this to see more messages for people that have big Dbs.

Comment 4

[Jim Porter (:squib)]

This should be ameliorated by the landing of bug 742236, although it might be nice to have an optional "search everything with no LIMIT" button.

Comment 5

[Wayne Mery (:wsmwk)]

(In reply to Jim Porter (:squib) from comment #4)
> This should be ameliorated by the landing of bug 742236, although it might
> be nice to have an optional "search everything with no LIMIT" button.

ludo, si this better now?

Comment 6

[Ludovic Hirlimann [:Usul]]

(In reply to Wayne Mery (:wsmwk) from comment #5)
> (In reply to Jim Porter (:squib) from comment #4)
> > This should be ameliorated by the landing of bug 742236, although it might
> > be nice to have an optional "search everything with no LIMIT" button.
> 
> ludo, si this better now?

If the limit as been removed yes. I've disabled gloda since I've joined IT , I had way too huge mailboxes that gloda could not deal with.

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to Ludovic Hirlimann [:Usul] from comment #6)
> (In reply to Wayne Mery (:wsmwk) from comment #5)
> > (In reply to Jim Porter (:squib) from comment #4)
> > > This should be ameliorated by the landing of bug 742236, although it might
> > > be nice to have an optional "search everything with no LIMIT" button.
> > 
> > ludo, si this better now?
> 
> If the limit as been removed yes. 
bug 742236 provides mailnews.database.global.search.msg.limit, so it allows user override.
But the remaining experience is Bug 537856, so I'm duping to there

> I've disabled gloda since I've joined IT ,
> I had way too huge mailboxes that gloda could not deal with.

glodaquilla doesn't help? it allows fine grained exclusions. if you set it then blow away the existing sqlite you should be good