Closed Bug 516902 Opened 15 years ago Closed 15 years ago

Gloda cannot search everything - it fails to find a specific message

Categories

(Thunderbird :: Search, defect)

x86
macOS
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 512917

People

(Reporter: cmtalbert, Assigned: asuth)

Details

(Keywords: testcase, Whiteboard: [no l10n impact])

Ok, I have a message that has managed to evade Gloda.  This was really annoying.  Just spent about 30 minutes trying to use gloda for searching.  Here are the things I tried.

1. Search for messages containing person Tim Riley (did not work, for obvious reasons)
2. Search for messages containing person adam goucher, which should have worked
3. Search for messages from list bt@goucher.ca
4. Search for messages with bt
5. Search for messages with pdf
6. Search for attachments of type pdf
7. Search for messages with QC1
8. Search for messages with beautiful testing
9. Search for messages with figures
10. Search for messages with review

In cases 2-10 this message did not come up, and it should have.

Here is the message:
Return-Path: bt-bounces@goucher.ca
Received: from mail.mozilla.com (LHLO mail.mozilla.com) (10.2.72.15) by
 mail.mozilla.com with LMTP; Fri, 4 Sep 2009 14:54:58 -0700 (PDT)
Received: from dm-mail01.mozilla.org (dm-mail01.mozilla.org [10.2.74.105])
	by mail.mozilla.com (Postfix) with ESMTP id 3C40821007C
	for <ctalbert@mozilla.com>; Fri,  4 Sep 2009 14:54:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at mozilla.org
X-Spam-Flag: NO
X-Spam-Score: -1.858
X-Spam-Level: 
X-Spam-Status: No, score=-1.858 required=6.31 tests=[AWL=0.454,
	BAYES_00=-2.312] autolearn=unavailable
Received: from psmtp.com (exprod5mx219.postini.com [64.18.0.78])
	by dm-mail01.mozilla.org (Postfix) with ESMTP id A0FFFB8186
	for <ctalbert@mozilla.com>; Fri,  4 Sep 2009 14:54:27 -0700 (PDT)
Received: from source ([64.34.111.254]) by exprod5mx219.postini.com ([64.18.4.10]) with SMTP;
	Fri, 04 Sep 2009 14:54:34 PDT
Received: from cwh5.canadianwebhosting.com (cwh5.canadianwebhosting.com [76.74.187.100])
	by barmail1.idig.net (Spam & Virus Firewall) with ESMTP
	id A09A12F625D8; Fri,  4 Sep 2009 14:54:05 -0700 (PDT)
Received: from cwh5.canadianwebhosting.com (cwh5.canadianwebhosting.com [76.74.187.100]) by barmail1.idig.net with ESMTP id jyiEjqLWg5AeHqXl; Fri, 04 Sep 2009 14:54:05 -0700 (PDT)
Received: from localhost ([127.0.0.1] helo=cwh5.canadianwebhosting.com)
	by cwh5.canadianwebhosting.com with esmtp (Exim 4.69)
	(envelope-from <bt-bounces@goucher.ca>)
	id 1MjgjJ-0002bO-SE; Fri, 04 Sep 2009 14:54:03 -0700
Received: from localhost ([127.0.0.1] helo=cwh5.canadianwebhosting.com)
	by cwh5.canadianwebhosting.com with esmtpa (Exim 4.69)
	(envelope-from <adam@goucher.ca>) id 1MjgeZ-0001uy-MH
	for bt@goucher.ca; Fri, 04 Sep 2009 14:50:09 -0700
Received: from adam-2.local ([127.0.0.2] helo=adam-2.local) with IPv4:26 by
	cwh5.canadianwebhosting.com; 4 Sep 2009 14:49:04 -0700
Message-ID: <4AA18B4D.9050002@goucher.ca>
Date: Fri, 04 Sep 2009 17:49:01 -0400
From: Adam Goucher <adam@goucher.ca>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
MIME-Version: 1.0
To: "bt@goucher.ca" <bt@goucher.ca>
Content-Type: multipart/mixed; boundary="------------060108090102090704090905"
X-Mailman-Approved-At: Fri, 04 Sep 2009 14:53:43 -0700
Subject: [Bt] [Fwd: Beautiful Testing QC1]
X-BeenThere: bt@goucher.ca
X-Mailman-Version: 2.1.11.cp3
Precedence: list
List-Id: Discussions around Beautiful Testing <bt_goucher.ca.goucher.ca>
List-Unsubscribe: <http://goucher.ca/mailman/options/bt_goucher.ca>,
	<mailto:bt-request@goucher.ca?subject=unsubscribe>
List-Archive: <http://goucher.ca/pipermail/bt_goucher.ca>
List-Post: <mailto:bt@goucher.ca>
List-Help: <mailto:bt-request@goucher.ca?subject=help>
List-Subscribe: <http://goucher.ca/mailman/listinfo/bt_goucher.ca>,
	<mailto:bt-request@goucher.ca?subject=subscribe>
Sender: bt-bounces@goucher.ca
Errors-To: bt-bounces@goucher.ca
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - cwh5.canadianwebhosting.com
X-AntiAbuse: Original Domain - mozilla.com
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - goucher.ca
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels:     (S:99.90000/99.90000 CV:99.9000 FC:95.5390 LC:95.5390 R:95.9108 P:95.9108 M:97.0282 C:98.6951 )
X-pstn-settings: 5 (2.0000:2.0000) s cv gt3 gt2 gt1 r p m c 
X-pstn-addresses: from <adam@goucher.ca> [32/2] 

This is a multi-part message in MIME format.
--------------060108090102090704090905
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit


Okay gang, its getting /really/ close now. Here is what just arrived in 
my mailbox. Please check over your chapters and keep the following stuff 
in mind.

-adam

Hi Tim and Adam,

The QC1 is attached for review. Please forward this PDF to the rest of the authors, and let them know they can use Adobe Reader's commenting tools to make their edits.

About the figures: the illustrator needs some time to draw them, so the images in the QC1 are still the drafts. (Not pretty, and not properly sized, but they serve their purpose as placeholders.) The drawings will be finished by September 11. At that time, I'll send you a new PDF so that the authors who have figures in their chapters can review them.

Here are some guidelines for the QC1:

* Make sure code indentation and alignment are correct.
* Make sure fonts are used consistently.
* Mark any technical, content, or formatting errors.
* Make sure figures are correctly numbered.
* For instances where long code lines flow into the margin, please mark a good breaking point.
* Make sure the bios in the Contributors chapter are up-to-date.

All reviewed chapters are due to me by September 18. If you or any of your coauthors have questions, don't hesitate to get in touch.


--------------060108090102090704090905
Content-Type: application/pdf;
 name="beautiful_testing_qc1.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="beautiful_testing_qc1.pdf"

<binary pdf document here>
Flags: blocking-thunderbird3?
Keywords: testcase
I am sorry to hear that you had such a frustrating experience :(

The problem is likely that gloda never actually indexed the message.  There are a number of bugs we are fixing for rc1 that involve gloda giving up when it encounters 'bad' messages.

The GlodaQuilla extension adds custom columns that let you know whether gloda has indexed a message (via the "gloda id" and "gloda dirty" columns):
https://addons.mozilla.org/en-US/thunderbird/addon/9873

Gloda has two discrete modes of indexing; sweep and event-driven.  If you toggle the read state of a message, star/unstar, tag/untag or otherwise do something to the message, the event-driven indexer will be triggered and then probably make the message visible to gloda.
Ah don't worry about my frustration. I'm getting less and less frustrated with shredder as time goes on.  It's getting better, but it definitely still has some warts. :)

So, I think then a normal user will not understand why their message doesn't show up when they search for it.  What is it that makes this a "bad message"?  You can't expect a user to do something to every message.  I think a very common use case for search is the following:

1. User gets email, reads and moves on
2. Two weeks later, user realizes that email was important
3. User goes looking for email.

FWIW this messsage was read.  Of course, I'm on nightlies so every day I have a new build.  It's entirely possible that the build when I toggled the read state on this message had a bug so that the event completely missed the event driven indexer.

I would recommend to the QA team that they try to get a set of people to stay on beta 4 for a little while to test the indexer so that you can verify that all the issues like this are fixed.  Perhaps running some kind of "two week trial" event in conjunction with beta 4 or some such.
Sorry, I forgot to actually say the main thing I meant to say.  Gloda probably gave up on *another* message that was bad, not this message.  This affects the sweep pass and, to a lesser extent, the event-driven pass as well.

I agree that we want to get a lot of exposure for the indexer.  The good news is we also have a good unit test policy for gloda so as I fix these problems I expect them to stay fixed.

Regrettably we did have some 'recovery' logic but it was intentionally handicapped to avoid the indexer going rogue; as such it was never unit tested and its best effort became rather pitiful at some point.  The good news is that the new error-handling logic will be unit tested under various injected failures that should be reasonably comprehensive.
Whiteboard: [no l10n impact]
asuth, is this likely a dupe of an existing existing blocker?
Assignee: nobody → bugmail
Flags: blocking-thunderbird3? → blocking-thunderbird3+
It is sad to dupe such a well-reported bug, but 512917 covers this.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.