1249056 - Thunderbird freezes periodically when Windows search integration is enabled

Status: RESOLVED INCOMPLETE

(Thunderbird :: Search, defect)

Description

[Andreas]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20160210153822

Steps to reproduce:

Enable Windows 10 search integration in Thunderbird preferences. Move messages between IMAP folders.


Actual results:

In approx. 10% of the moves, Thunderbird freezes for about 1 minute. After that the move is complete. Process monitor shows that Thunderbird is trying to access a wdseml file in a mozmsgs folder at the time it is frozen. I suspect that a deadlock with a concurrent access to that file by the windows search indexer occurs. After disabling the Windows 10 search integration, the freeze problem disappears.


Expected results:

Moves between IMAP folders should not cause Thunderbird to freeze even if Windows 10 search integration is activated.

Comment 1

[Wayne Mery (:wsmwk)]

Thunderbird has no reason to access to file after it is created or after a deleted (related to a move) has occurred.  And afaik no other software should be accessing the file synchronously or exclusively.  So, the interference is likely some other software. For example antivirus SW, which should not be accessing anything the the Thunderbird profile directory hierarchy. 

Which AV software are you using? What happens if you disable it or define an exclusion for Thunderbird profile directory?

Comment 2

[Andreas]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #1)
> Thunderbird has no reason to access to file after it is created or after a
> deleted (related to a move) has occurred.  And afaik no other software
> should be accessing the file synchronously or exclusively.  So, the
> interference is likely some other software. For example antivirus SW, which
> should not be accessing anything the the Thunderbird profile directory
> hierarchy. 

I can see from process monitor (the tool from SysInternals) that Thunderbird freezes as it tries to move the wdseml file. I can see a system function called and a delay of approx. 1 minute before the "SUCCESS" result is displayed.

> 
> Which AV software are you using? What happens if you disable it or define an
> exclusion for Thunderbird profile directory?

None other than the Windows Defender which comes with Windows 10. As far as I know, exclusions cannot be defined in Windows Defender.

Comment 3

[Wayne Mery (:wsmwk)]

> I can see a system function called and a delay of approx. 
In process monitor you need to dig deeper to see which system function is being used. 
iirc you want to look at process tree

> As far as I know, exclusions cannot be defined in Windows Defender.
http://windows.microsoft.com/en-us/windows-10/add-an-exclusion-to-windows-defender

Comment 4

[Andreas]

> In process monitor you need to dig deeper to see which system function is being used. 

After some further investigation with Process Monitor I got the following protocol:

Time             Process                Operation                Path
20:46:53,4704783 thunderbird.exe        SetRenameInformationFile source.mozmsgs\somefilename.wdseml
20:48:21,6206965 SearchProtocolHost.exe CloseFile                source.mozmsgs\somefilename.wdseml
20:48:21,6225034 thunderbird.exe        CloseFile                target.mozmsgs

Seems to me that thunderbird.exe is waiting for SearchProtocolHost.exe to release the file somefilename.wdseml. In between 46:53 and 48:21 Thunderbird is frozen.

Comment 5

[Kent James (:rkent)]

What is this folder source.mozmsgs? Where is it located, and what is it used for?

Comment 6

[Andreas]

(In reply to Kent James (:rkent) from comment #5)
> What is this folder source.mozmsgs? Where is it located, and what is it used
> for?

"source" is the IMAP folder where the message was before I moved it. "target" then is the folder where I moved it.
It's located here:
...\AppData\Roaming\Thunderbird\Profiles\xyz.default\ImapMail\mailservername\source.mozmsgs
It's used to store the *.wdseml files that Thunderbird creates for Windows Desktop Search.

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to Andreas from comment #6)
> (In reply to Kent James (:rkent) from comment #5)
> > What is this folder source.mozmsgs? Where is it located, and what is it used
> > for?
> 
> "source" is the IMAP folder where the message was before I moved it.
> "target" then is the folder where I moved it.
> It's located here:
> ...\AppData\Roaming\Thunderbird\Profiles\xyz.
> default\ImapMail\mailservername\source.mozmsgs
> It's used to store the *.wdseml files that Thunderbird creates for Windows
> Desktop Search.

Correct. When "Allow Windows Search to search messages" is enabled, for every thunderbird <folder> there is an equivalent <folder>.mozmsgs folder which holds every email as a separate .wdseml file. 

.mozmsgs folders have the file attribute "Allow files in this folder to have contents indexed"
.wdseml files have the file attribute "Allow this file to have contents indexed"
And .wdseml extension type is identified to WSS (Windows Search Service) to be indexed by contents and file attributes. http://windows.microsoft.com/en-us/windows7/change-advanced-indexing-options

Comment 8

[Wayne Mery (:wsmwk)]

BTW, your data is very detailed, which is a great thing to have.

I don't know what's causing the delay, and I don't know the internals of Thunderbird in this area. Given that it's an imap folder, it seems to me the process is Thunderbird 1. copies the message to the imap folder, 2. deletes the message from the source imap folder 3. wdseml file gets deleted. And somewhere in there the new wdseml file is created - but I think that step should have no impact.

Can you reproduce the delay if you copy a message from imap to local folder, or from local folder to local folder?

Comment 9

[Andreas]

Some further experiments gave these results:

imap or pop3 inbox immediately after new message received -> any imap or local folder => delay
any other move (e. g. local to local, imap to local, etc.) => no delay

where -> means message moved to
      => means results in

Comment 10

[Wayne Mery (:wsmwk)]

I'm hoping sid0 will be able to offer insight on comment 4 and related items

Comment 11

[Wayne Mery (:wsmwk)]

I wonder if windows indexing service has a bug, or is less performant, in Windows 10

Comment 12

[Alexander Buchner]

As I already described in bug 1283908 I can see the same behavior even when "Allow Windows Search to search messages" is deactivated.

The only workaround that worked for me was disabling the Windows service "Windows Search".

Comment 13

[Alexander Buchner]

Update:
I now noticed that I also encounter this problem when Thunderbird is trying to save a draft to the drafts directory.
The Windows service "Windows Search" was NOT running.

Comment 14

[Wayne Mery (:wsmwk)]

Alexander,

What is size of windows.edb in C:\ProgramData\Microsoft\Search\Data\Applications\Windows ?
Is your disk SSD, slow laptop disk, or fast non-SSD?
And how much free space is available on your disk?

Comment 15

[Wayne Mery (:wsmwk)]

Same question for Andreas

Comment 16

[Andreas]

File size is approx. 80 MB. 
Disk type is SSD. 
Free disk space is approx. 65 GB.

Comment 17

[Alexander Buchner]

File size is approx. 752 MB. 
Disk type is SSD. 
Free disk space is approx. 28 GB.

Comment 20

[Wayne Mery (:wsmwk)]

Does anyone better understand what is happening here?   (Now that we're a couple years into this)

Why are some users affected and not others?
Why is this worse in win10 than win7?  
And is there anything we can do to mitigate this?

ditto bug 1308783

Comment 21

[Wayne Mery (:wsmwk)]

Andreas,

Do you still see this when using version 102? If you do, please create a performance profile using the procedure at https://support.mozilla.org/en-US/kb/profiling-thunderbird-performance

Comment 22

[Phoenix]

Resolved per whiteboard