1326443 - Gloda indexing will hang if profile folder does not have Logs folder.

Status: RESOLVED FIXED

(Thunderbird :: Instant Messaging, defect)

Description

[cfeilen]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36

Steps to reproduce:

Remove logs folder from profile folder.
Start TB.
Mail slow to index, if at all.


Actual results:

Gloada indexer job for logsFolderSweep fails to run/return control. 
Indexing continues to spin, but does nothing.


Expected results:

Indexing should have skipped the folder since it was not there.

Comment 1

[cfeilen]

Not sure how to contribute yet, but fix is in C:\mozilla-source\comm-aurora\mail\components\im\modules\index_im.js, line 645.  Should yield GlodaIndexer.kWorkDone, rather than return.

Comment 2

[Wayne Mery (:wsmwk)]

what log folder?  do you mean an imap folder?  or the global-messages-db journal file?

Comment 3

[cfeilen]

The ProfD folder.
\Users\FOO\AppData\Roaming\Thunderbird\Profiles\RANDOM.default\logs\

Comment 4

[Wayne Mery (:wsmwk)]

The logs directory is only used for IRC. afaict if you have an IRC account and the directory is deleted, on startup it is recreated.
 
I was not able to reproduce using version 45.6.0.  Do you see something specifically in the Thunderbird, and where, that indicates indexer is stuck?

Comment 5

[cfeilen]

Repro steps:
1) Completely close ThunderBird
2) Delete global-messages-db.sqlite
3) Delete C:\Users\cfeilen\AppData\Roaming\Thunderbird\Profiles\7perphv0.default\logs
4) Start ThunderBird

Observe:
1) global-messages-db.sqlite is recreated, but a small size (mine is ~1.18mb)
2) Click Thunderbird -> Toolbar -> Tools -> Activity Manager
3) Note the message in Activity Manager is: "Determining which messages to index" (Will hang here indefinitely)

Fix:
1) Close Thunderbird completely
2) Create empty folder C:\Users\cfeilen\AppData\Roaming\Thunderbird\Profiles\7perphv0.default\logs
3) Open Thunderbird
4) Click Thunderbird -> Toolbar -> Tools -> Activity Manager
5) Note messages which list folders and message counts
6) global-messages-db.sqlite will increase in size (and data inside will contain all messages indexed)

Comment 6

[cfeilen]

Attachment: 1326443_1.patch

Proposed fix.

Comment 7

[Jorg K (CEST = GMT+2)]

Thanks, looks reasonable, I'll have to test this before approving it.

Comment 8

[Wayne Mery (:wsmwk)]

Excellent. Thanks!

Comment 9

[aleth [:aleth]]

Ouch, that's a nasty bug, especially as log removal will get an UI in 52. Thanks for spotting this!

Comment 10

[aleth [:aleth]]

While we're at it, can you check that after this fix, search hits inside IRC logs no longer appear after the corresponding logs are deleted?

Comment 11

[cfeilen]

@aleth, I don't run IRC with TB, so can't test that, nor any regression paths.

Comment 12

[Jorg K (CEST = GMT+2)]

Comment on attachment 8823555 [details] [diff] [review]
1326443_1.patch

Thank you for your contribution. The patch is correct.

We converted legacy generators in bug 1292569 which was a massive job. Sadly, we missed some, so we fixed some more in bug 1294960. And there we obviously missed what you're repairing now, see:
https://hg.mozilla.org/comm-central/rev/37f7bb383f98baadaf6d3e8ccf36608f46e0dc90
You can even see the return statement which should have been converted.

In the future, please supply patches with the correct format. They need a header. We will fix it this time when landing the patch (to land in the context means, committing it to the code base). We will also need to uplift his to the current beta TB 51 and Aurora/Earlybird TB 52.

Here's the header I'd suggest for the patch:
# HG changeset patch
# User C Feilen <[e-mail]>
# Parent  a74c3349d3649f47082a0ef8ecb09dc63c082a03
Bug 1326443 - Bug 1292569/bug 1294960 follow-up: Change return to yield in generator. r=jorgk,aleth

What's your name or would you like to stay anonymous?

Thanks again, we appreciate your contribution ... true to the spirit:
It's open source, it ain't working, let's go and fix it ;-)

Comment 13

[Jorg K (CEST = GMT+2)]

(In reply to aleth [:aleth] from comment #10)
> While we're at it, can you check that after this fix, search hits inside IRC
> logs no longer appear after the corresponding logs are deleted?
Nice try, but let's not overburden this bug. Here we're fixing an oversight from bug 1294960.

Comment 14

[cfeilen]

(In reply to jorgk from comment 12)
Attribution will be fine.  
Let me see if I can add those lines and update the patch.

Comment 15

[cfeilen]

Attachment: 1326443_2.patch

Adds comment header as requested by [e-mail].

Comment 16

[Jorg K (CEST = GMT+2)]

Comment on attachment 8823948 [details] [diff] [review]
1326443_2.patch

Thanks. Somehow Aleth wanted to look at this as well, so let's add r? for him again.

Comment 17

[Jorg K (CEST = GMT+2)]

C-C (TB 53): https://hg.mozilla.org/comm-central/rev/8e89d3c05ce45753b470301396c01185de56930a
C-A (TB 52): https://hg.mozilla.org/releases/comm-aurora/rev/fc8949513c6be12860698bb98f1ea57417f6cf17
C-B (TB 51): https://hg.mozilla.org/releases/comm-beta/rev/7342ce88c370df977483bff96dd06f085387d381

Comment 18

[Jorg K (CEST = GMT+2)]

I forgot to say: I needed this for beta.

Comment 19

[aleth [:aleth]]

I need to check this and comment 10 after bug 1322473 lands.

Comment 20

[aleth [:aleth]]

Thanks cfeilen for this patch! There are indeed some remaining gloda issues when logs are deleted, filing a separate bug for these. Nothing as bad as a hang though...

Comment 21

[aleth [:aleth]]

(In reply to aleth [:aleth] from comment #10)
> While we're at it, can you check that after this fix, search hits inside IRC
> logs no longer appear after the corresponding logs are deleted?

-> Bug 1333603 (not just for IRC)