Closed Bug 1574863 Opened 2 years ago Closed 2 years ago

Cannot open conversation from Gloda search result in some cases


(Thunderbird :: Instant Messaging, defect)

Not set


(thunderbird_esr6869+ fixed, thunderbird69 fixed, thunderbird70 fixed)

Thunderbird 70.0
Tracking Status
thunderbird_esr68 69+ fixed
thunderbird69 --- fixed
thunderbird70 --- fixed


(Reporter: jorgk-bmo, Assigned: jorgk-bmo)



(1 file, 1 obsolete file)

One of the nice things of Gloda is that it indexes chat conversations and can find and open them. However, I'm seeing this weird issue now for the first time.

I was looking for "pike" in my chat history and found 42 hits in IRC. Some can be opened, some can't. For example one from #developers on 09 Jan 2018 opens fine, but one from #maildev on 5 February 2019 doesn't open, in fact, all hits from #maildev don't seem to open.

There's an error on the console:
TypeError: logTime is undefined - logger.js:742:11
which is here:
I'm not sure how it can be undefined since it was set a line above:
let [logTime] = getDateFromFilename(OS.Path.basename(path));

I checked on disk and the file is there:

D:\MAIL-THUNDERBIRD\jorgk.default\logs\irc\\>dir 2019-02-05*
 Volume in drive D has no label.
 Volume Serial Number is 7E0E-9CA4

 Directory of D:\MAIL-THUNDERBIRD\jorgk.default\logs\irc\\

[stuff deleted]
06 Feb 2019  01:05             8,769 2019-02-05.230651+0100.json
              11 File(s)         28,967 bytes
               0 Dir(s)  158,148,448,256 bytes free

D:\MAIL-THUNDERBIRD\jorgk.default\logs\irc\\>grep -i pike 2019-02-05*
2019-02-05.230651+0100.json:{"date":"2019-02-05T22:24:39.000Z","who":"Pike","text":"of all the places","flags":["incoming"]}

I've checked in the Gloda database, and it knows the log file:

select path from imConversations where path like "%maildev%2019-02-05%"
[mote hits]

So where is this going wrong?

Doesn't work in TB 60 either, so not a recent regression.

OK, found it. I had dropped a file into the logs directory which turned up in the loop here:
but didn't match the wildcard here:

Effect, the entire thing falls apart. Maybe we should be a little more accepting.

Attached patch 1574863-chat-logs.patch (obsolete) — Splinter Review

OK, no one should mess with those logs, but I did some searching and sorting in there and left some files behind. There's no harm in skipping unrecognised files.

I don't know who wants to review this, I'll try Florian first, this only needs visual inspection.

Assignee: nobody → jorgk
Attachment #9086355 - Flags: review?(florian)
Comment on attachment 9086355 [details] [diff] [review]

Review of attachment 9086355 [details] [diff] [review]:

Thanks for fixing this edge case. Reading this makes me wonder if we should log the name of unrecognized files to the console, but I don't care enough to request a change to do this.
Attachment #9086355 - Flags: review?(florian) → review+

Like this? Console says:
Found log file with name not maching YYYY-MM-DD.HHmmSS+ZZzz.format: all-chat.txt

Attachment #9086376 - Flags: review?(florian)
Comment on attachment 9086376 [details] [diff] [review]

Review of attachment 9086376 [details] [diff] [review]:

Seems reasonable, thanks. I hope we don't have massive amounts of unknown files that will spam the console.
Attachment #9086376 - Flags: review?(florian) → review+

There shouldn't be any, otherwise this bug would have been filed before. I'm likely the only persons processing those log files and leaving undesired files behind in the directory. BTW, I strung all the small log files together to one big file with cat since grepping through a large file is much faster then grepping through 6000+ small files which the file system has to open individually. Also for Gloda to open the log in question takes a few seconds since it's processing many log files in the directory.

Are there any plans to switch form one log file per session to something more compact? Also for backup it's not convenient to have all those small log files.

As for the patch: I considered switching to Services.console.logStringMessage() but that doesn't stand out in the log.

Attachment #9086355 - Attachment is obsolete: true

Pushed by
Handle 'foreign' files in logs directory gracefully. r=florian

Closed: 2 years ago
Resolution: --- → FIXED
Comment on attachment 9086376 [details] [diff] [review]

No harm in backporting this.
Attachment #9086376 - Flags: approval-comm-esr68?
Attachment #9086376 - Flags: approval-comm-beta+
Target Milestone: --- → Thunderbird 70.0
Version: unspecified → Trunk

(In reply to Jorg K (GMT+2) from comment #7)

Are there any plans to switch form one log file per session to something more compact? Also for backup it's not convenient to have all those small log files.

I used to want to automatically compress the old log files into one gzip file per month. But I don't expect to work on this any time soon, if ever.

The fact that you need to grep shows that something is broken with the search. And I happen to sometimes grep my log folder too, so I would probably want search to be fixed (which could involve replacing gloda with something simpler and more efficient) before we compress logs on disk.

Pushed by
Follow-up: remove trailing spaces. rs=white-space-only DONTBUILD

Sorry about the trailing spaces, they're barely visible in splinter now, filed bug 1575001.

The fact that you need to grep shows that something is broken with the search.

Well, the other day I was looking for something I thought should have been there. Gloda didn't find it, and neither did grep, so it was just in my head :-(

Attachment #9086376 - Flags: approval-comm-esr68? → approval-comm-esr68+
You need to log in before you can comment on or make changes to this bug.