375292 - Stray tmprules.dat files created when getting new messages

Status: RESOLVED FIXED

(MailNews Core :: Filters, defect)

Description

[Pawel Worach]

User-Agent:       Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.4pre) Gecko/20070325 Firefox/2.0.0.4pre
Build Identifier: version 2.0pre (20070325)

Using a GMail POP3 account with some filter rules leaves a lot of files around in /tmp.
-rw-------   1 username  wheel      25 Mar 25 18:05 tmprules-1.dat
-rw-------   1 username  wheel      25 Mar 25 18:10 tmprules-2.dat
-rw-------   1 username  wheel      25 Mar 25 18:15 tmprules-3.dat
-rw-------   1 username  wheel      25 Mar 25 18:20 tmprules-4.dat
-rw-------   1 username  wheel      25 Mar 25 18:25 tmprules-5.dat
-rw-------   1 username  wheel      25 Mar 25 18:00 tmprules.dat

They all contain:
version="8"
logging="no"

Reproducible: Always

Comment 1

[Jo Hermans]

That file is created in nsMsgFilterService::SaveFilterList(), and there's must have been something gone wrong, because the tmpFiltersFile->Delete() is never reached. Can you check if there is a file in your profile called tmprules.dat or msgFilterRules.dat, which is read only ?

Comment 2

[Pawel Worach]

The are all rw.
> find . -name msgFilterRules.dat -ls
4263055        4 -rw-------    1 username          staff                  25 Jan  7 02:55 ./Mail/Local Folders/msgFilterRules.dat
966652        4 -rw-------    1 username          staff                  25 Jan  7 02:55 ./Mail/News & Blogs/msgFilterRules.dat
4263270        8 -rw-------    1 username          staff                3353 Mar 25 18:15 ./Mail/pop.gmail.com/msgFilterRules.dat

Comment 3

[Simon Bünzli]

Happening under WinXP as well (TB2 build 20070326).

Shouldn't a temporary file be deleted anyway, no matter what errors occur in the function creating it?

Comment 4

[Sam Steingold]

I observe this on fc5 (2.6.18.8) with version 2.0.0.0 (20070326)

Comment 5

[Dmitry Bilunov]

Bug confirmed under Thunderbird 2.0.0.0 (20070419) (Gentoo build, amd64).

Comment 6

[Peter Reynolds]

I've been getting these folders appearing in my News and Blogs for some time, and just in the last day or two in the Local Folders section too.  I'd assumed they were created because we use two different versions of Thunderbird (on two computers) to access our files across a network, but presumably this is not in fact the cause?

Comment 7

[Peter Reynolds]

PS I'm using Thunderbird Trunk Alpha (nightly), and my wife uses the release version (not Alpha or Beta).

Comment 8

[bugmenot]

Bug confirmed under Thunderbird 2.0.0.0 (20070518) (Gentoo build, x86).

Comment 9

[wiebe]

I have it too.

Thunderbird: version 2.0.0.0 (20070519)
Gentoo x68

I use thunderbird to connect to an IMAP server, and I get these files in /tmp.

Comment 10

[Torleiv Ringer]

Ubuntu Feisty Thunderbird 2.0.0.0 20070326 also getting this problem.

Comment 11

[Frank Winkler]

Also on Solaris/SPARC with IMAP account(s) - I've been seeing this for a while now and finally decided to search the bugs DB.

As a w/a, I'm using a cron job deleting the files.

Comment 12

[Peter Reynolds]

I discovered that, despite these not being visible in the Folders list in Thunderbird, it is also creating loads of these files for each of my 10 or so different email accounts!

I've never heard of a cron job, but Googling it looks like it's a Unix thing.  Some of us still use a Microsoft operating system even if our other software is Open Source ;-)

Comment 13

[Jonathan Billings]

I can confirm this bug under Thunderbird 2.0.0.0 (Fedora 7 package: thunderbird-2.0.0.0-1.fc7).

I see a new file in /tmp/ created for each time thunderbird checks for new messages.  First file is /tmp/tmprules.dat, then /tmp/tmprules-1.dat, then /tmp/tmprules-2.dat, and so on.

If I remove all message filters the problem remains.

I believe the problem is part of the Junk Mail system.  I can cause the tmprules file to be created when:

1.) I have (at least) 2 accounts
2.) One account has Junk Settings->Trust junk mail headers set by: [Spamassassin]
3.) The other account has Junk Settings->Enable adaptive junk mail controls for this account or Junk Settings->Do not mark mail as junk if the sender is in [Personal Address Book].

If I only have one account, enabling the junk mail settings doesn't cause new tmprules.dat files to be created.  

Comment 14

[Jonathan Billings]

(In reply to comment #13)
> I can confirm this bug under Thunderbird 2.0.0.0 (Fedora 7 package:
> thunderbird-2.0.0.0-1.fc7).

Just upgraded to Thunderbird version 2.0.0.4 (20070615), it still has the same incorrect behavior.

Comment 15

[bugmenot]

(In reply to comment #8)
> Bug confirmed under Thunderbird 2.0.0.0 (20070518) (Gentoo build, x86).
> 

I've now upgraded to Thunderbird 2.0.0.4 (20070618) (Gentoo build, x86) and the issue is still here. I'm checking exclusively IMAP accounts (2 with SSL, 3 without), and these files appear in /tmp/ (also seeing left-over files in my profile as per Jo Hermans' post and Pawel Worach confirmation).

Comment 16

[Alexey Gladkov]

Attachment: Remove tmprules

I think tmpFiltersFile should be always removed before SaveFilterList() finished.

Comment 17

[David :Bienvenu]

Alexey, have you been able to reproduce the problem? I think your fix is probably the right thing, but I'd really like to know what's making things go wrong in the first place.

Comment 18

[Alexey Gladkov]

I have reproduced this problem with 2.0.0.0 and 2.0.0.4 versions. I will debug SaveFilterList() to answer your question.

Comment 19

[David :Bienvenu]

oh, thank you! getnewmail shouldn't cause SaveFilterList to get called in the first place - so a stack trace would be interesting.

Comment 20

[Kyle Guinn]

I can also confirm this bug on seamonkey 1.1.2.

Comment 21

[Alexey Gladkov]

Sorry for the long silence. 
I put into this SaveFilterList() function some debug messages (ZZZ prefix):

...
ZZZ nsMsgFilterService::SaveFilterList() InFunction()
ZZZ nsMsgFilterService::SaveFilterList(): filterFile /usr/lib/thunderbird/isp/SpamAssassin.sfd
ZZZ nsMsgFilterService::SaveFilterList(): ret = filterList->SaveToFile(tmpFileStream): NS_SUCCEEDED
ZZZ nsMsgFilterService::SaveFilterList(): ret = tmpFiltersFile->CopyToDir(parentDir): NS_FIALED
###!!! ASSERTION: error opening/saving filter list: 'NS_SUCCEEDED(ret)', file nsMsgFilterService.cpp, line 215
Break: at file nsMsgFilterService.cpp, line 215
Begin mail message delivery.
Abort mail message delivery.
Opening file SpamAssassin.sfd failed
...

I think, it's clear enough. /usr/lib/thunderbird/isp/SpamAssassin.sfd is not writable by user in Linux.

Comment 22

[Alexey Gladkov]

Attachment: Remove tmprules

Bug assigned to Nobody.
How can review this patch ?

Comment 23

[David :Bienvenu]

Comment on attachment 272146 [details] [diff] [review]
Remove tmprules

thx, Alexy, that makes a lot of sense, as to why only some people would have this issue. And your fix looks fine - I'm not sure you need to check if the file exists before calling delete but it shouldn't hurt.

Can you give me a stack track of when we get into this scenario? We shouldn't be trying to save over the .sfd file at all...

Comment 24

[Peter Niemayer]

I use an executable built with that latest patch above. The spurious tmprules-*.dat files are gone, but the Junk Settings->Trust junk mail headers set by: [Spamassassin] option still works unrealiable - for no obvious reasons, about half of the SPAM-mails clearly marked as SPAM by Spamassassin are not treated as SPAM by thunderbird.

Comment 25

[Thomas]

(In reply to comment #24)
> I use an executable built with that latest patch above. The spurious
> tmprules-*.dat files are gone, but the Junk Settings->Trust junk mail headers
> set by: [Spamassassin] option still works unrealiable - for no obvious reasons,
> about half of the SPAM-mails clearly marked as SPAM by Spamassassin are not
> treated as SPAM by thunderbird.
> 

I have the same problem that half of the emails marked as SPAM by Spamassassin are not detected. This problem (and the tmprules-*.dat files, of which on my system thousands are generated each day due to my high mail volume) has been there since I switched to 2.0

Comment 26

[Y Giridhar Appaji Nag]

Users of the icedove Debian package also <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440698#5">reported this bug</a>.

Comment 27

[Hungerburg]

As an icedove user, turning off the "trust spamassassin" option, that was on on one of the accounts, fixed it for me immediately. /tmp was littered with thousands of those files (its only cleared on reboot I guess).

peter

Comment 28

[Manel Rodero]

Hello,

This bug happens as confirmed in Thunderbird 2.0.0.0 to 2.0.0.6 under Windows XP. We have detected the creation of tmp files because the computer is VERY SLOW while using Thunderbird.

First of all, our users told us that "when creating an email, Thunderbird doesn't responds to the keyboard" and after some seconds "the letters and words appears suddenly". Thunderbird is slow (seems like is hanging for some seconds) while using menus, dialog boxes or simply trying to close it. When this happens the use of CPU of Thunderbird increases up to 99% (as showing in Task Manager).

We've used FileMon/RegMon for watching what's Thunderbird doing while is "not responsive" and we've seen that Thunderbird is creating a lot of "tmprules*.dat" files in %TEMP% folder of the user (i.e. C:\Documents and Settings\<user>\Temp).   I've seen the creation of thousands of files in just some 4-5 seconds. The temporary folder of the users have tmprules-1.dat up to tmprules-9999.dat files.

I suppose that the CPU hung of 99% is due to the fact that some Thunderbird routine is creating these files and doesn't let Windows react to user inputs (mouse, keyboard, screen, etc.).

We suppose too that these problem appears every time that Thunderbird checks the inbox for new messages and this problem appears if you have message filters but too if you don't have these filters.

We have SOLVED this problem (the slowdown of the PCs NOT the creation of tmp files) doing the following:

- Closing Thunderbird
- Deleting msgfilterrules.dat of every account in the user profile
- Deleting tmprules*.dat in the temp folder of the user

Then, Thunderbird is VERY responsive. It doesn't hangs but the temporary files continue appearing in the temp folder (at a 2-3 files per session rythm not thousands of them). We don't know if the problem will appear in the futur (for example when the file number will be 9999) so we're watching for a solution in the code of Thunderbird.

We will look for the settings related to SpamAssassin but I think (I need to check this in an user computer) we have them disabled.

I hope this information will point the route to a solution.

Thank you.

PS: Sorry for my poor english :-(

Comment 29

[Manel Rodero]

Hi again,

I've forget saying that we've started using FileMon/RegMon for monitoring Thunderbird when an user asked us "why their message filter rules dissapear after closing TH". We monitored their profile directory to see if the msgFilterRules.dat was updated and we saw the creation of thousands of tmp files.

PS: Remember this after reading https://bugzilla.mozilla.org/show_bug.cgi?id=390992 ;-)

Comment 30

[Manel Rodero]

Here are our Account Settings > user@account > Junk Settings

[ ] Enable adaptative junk mail controls for this account
[X] Trust junk mail headers set by: SpamAssassin

So, the trust junk is checked but the adaptative junk is not. Is Thunderbird doing something with this configuration? Are these two independent options? Or the trust option only works when the first one is enabled?

Thanks.

Comment 31

[David :Bienvenu]

If you manually change /usr/lib/thunderbird/isp/SpamAssassin.sfd  so that it's writeable by the user, does that fix the problem?

I vaguely remember there being an issue about where we look for the .sfd files as well, but I can't find that bug.

Comment 32

[Peter Reynolds]

I'm still getting this problem even after I turned off the Trust Spamassassin thing.  I don't know when the files are generated, but it is conceivable that they appear when my wife is using Thunderbird 2.0.0.0 (20070326)through the network, or it could be when I am using Thunderbird 3.0a1pre (trunk nightlies)

Comment 33

[David :Bienvenu]

looking again at this patch, it's not right for the trunk because of the conversion from nsIFileSpec to nsIFile. I can try to come up with a trunk version.

I think the patch would be ok for the 2.0 branch, however.

Comment 34

[Manel Rodero]

We don't know why this problem starts.

We only know that if this problem appears (slow pc caused by thunderbird blocked while creating tmp files) we can solve it by:

- closing thunderbird
- deleting tmprules*.dat and msgfilterrules.dat
- starting thunderbird

After taht, thunderbird works ok, but it starts creating again temporary files (this time, only a couple of files). We have seen that, when the users says to us "my thunderbird is slow", the problem is caused by the temporary files problem. We see in the %TEMP% folder up to 9999 temporary dat files.

We can't reproduce the problem after the solve it (by the way, we don't know if the problem will appear again if the count for the temporary files will be again 9999).

File SpamAssassin.sfd under Windows is under the program files directory so no writable by a "user".

Comment 35

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Will all these files be created at once while running one getNewMail or step by step? If it's on one step do we have a problem in creating the file or why such a huge amount of files are created? The attached patch only handles the deleting of such temporary files.

Comment 36

[Manel Rodero]

Hi,

As we've seen while running FileMon, these files are created at once. Every check mail time thousands of these files are created. I don't know when/why the 9999 limit is reached. When we have detected the problem we saw this amount of temporary files.

At this moment we don't have any client with the slow problem after we did the deletion of the temporary files but ... we don't know if the problem will appear again when some magic number of temporary files is reached ;-)

We can't test the patch because we don't compile the Windows code by ourselves. We install/upgrade using the standard binaries.

Comment 37

[Manel Rodero]

Hello again,

The "slow problem" has appeared again.

Now we have debugged it and we have the FileMon (FileMonitor) log files where we can see what's doing Thunderbird when it works ok and when it works bad.

When working bad (fragments):
=============================

16	11:26:12	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Options: Open  Access: All	
17	11:26:12	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Attributes: A	
18	11:26:12	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		
19	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
20	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
21	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
22	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	NAME COLLISION	Options: Create  Access: All	
23	11:26:12	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules-1.dat	SUCCESS	Options: Open  Access: All	
24	11:26:12	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules-1.dat	SUCCESS	Attributes: A	
25	11:26:12	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules-1.dat	SUCCESS		
26	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
27	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
28	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
29	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules-1.dat	NAME COLLISION	Options: Create  Access: All	
30	11:26:12	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules-2.dat	SUCCESS	Options: Open  Access: All	
31	11:26:12	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules-2.dat	SUCCESS	Attributes: A	
32	11:26:12	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules-2.dat	SUCCESS		
33	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
34	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
35	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
36	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules-2.dat	NAME COLLISION	Options: Create  Access: All	
37	11:26:12	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules-3.dat	SUCCESS	Options: Open  Access: All	
38	11:26:12	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules-3.dat	SUCCESS	Attributes: A	
39	11:26:12	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules-3.dat	SUCCESS		
40	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
41	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
42	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
43	11:26:12	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules-3.dat	NAME COLLISION	Options: Create  Access: All	
44	11:26:12	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules-4.dat	SUCCESS	Options: Open  Access: All	
45	11:26:12	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules-4.dat	SUCCESS	Attributes: A	
[...]
59342	11:26:28	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules-2689.dat	SUCCESS		
59343	11:26:28	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
59344	11:26:28	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
59345	11:26:28	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
59346	11:26:28	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules-2689.dat	NAME COLLISION	Options: Create  Access: All	
59347	11:26:28	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules-2690.dat	SUCCESS	Options: Open  Access: All	
59348	11:26:28	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules-2690.dat	SUCCESS	Attributes: A	
59349	11:26:28	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules-2690.dat	SUCCESS		
60790	11:26:31	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Options: Open  Access: All	
60791	11:26:31	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Attributes: 	
60792	11:26:31	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS		
60793	11:26:31	thunderbird.exe:2372	CREATE	D:\USUARIS	NAME COLLISION	Options: Create Directory  Access: All	
60794	11:26:31	thunderbird.exe:2372	CREATE	D:\USUARIS\castell	NAME COLLISION	Options: Create Directory  Access: All	
60795	11:26:31	thunderbird.exe:2372	CREATE	D:\USUARIS\castell\THUNDERBIRD	NAME COLLISION	Options: Create Directory  Access: All	
60796	11:26:31	thunderbird.exe:2372	CREATE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	NAME COLLISION	Options: Create  Access: All	
60797	11:26:31	thunderbird.exe:2372	CREATE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Options: OverwriteIf  Access: All	
60798	11:26:31	thunderbird.exe:2372	WRITE 	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Offset: 0 Length: 4096	
60799	11:26:31	thunderbird.exe:2372	WRITE 	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Offset: 4096 Length: 4096	
60800	11:26:31	thunderbird.exe:2372	WRITE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Offset: 8192 Length: 4096	
60801	11:26:31	thunderbird.exe:2372	WRITE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Offset: 12288 Length: 4096	
60802	11:26:31	thunderbird.exe:2372	WRITE 	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS	Offset: 16384 Length: 3223	
60803	11:26:31	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\localstore.rdf	SUCCESS		

When working OK:
================

16	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	NOT FOUND	Options: Open  Access: All	
17	11:31:55	thunderbird.exe:2372	CREATE	C:\DOCUME~1	NAME COLLISION	Options: Create Directory  Access: All	
18	11:31:55	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL	NAME COLLISION	Options: Create Directory  Access: All	
19	11:31:55	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP	NAME COLLISION	Options: Create Directory  Access: All	
20	11:31:55	thunderbird.exe:2372	CREATE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Options: Create  Access: All	
21	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		
22	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Options: OpenIf  Access: All	
23	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Length: 0	
24	11:31:55	thunderbird.exe:2372	WRITE 	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Offset: 0 Length: 27	
25	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		
26	11:31:55	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\ImapMail\	SUCCESS	Options: Open Directory  Access: All	
27	11:31:55	thunderbird.exe:2372	DIRECTORY	D:\USUARIS\castell\THUNDERBIRD\ImapMail\	SUCCESS	FileBothDirectoryInformation: bustia.fib.upc.es	
28	11:31:55	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\	SUCCESS		
29	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS	Options: Open Directory  Access: All	
30	11:31:55	thunderbird.exe:2372	DIRECTORY	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS	FileBothDirectoryInformation: tmprules.dat	
31	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS		
32	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Options: Open Sequential  Access: All	
33	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	FileAttributeTagInformation	
34	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Length: 27	
35	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Attributes: A	
36	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	FileStreamInformation	
37	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Attributes: A	
38	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	FileEaInformation	
39	11:31:55	thunderbird.exe:2372	CREATE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Options: Create Sequential  Access: All	
40	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	FileFsAttributeInformation	
41	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Attributes: A	
42	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	FileFsAttributeInformation	
43	11:31:55	thunderbird.exe:2372	SET INFORMATION 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Length: 27	
44	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Length: 27	
45	11:31:55	thunderbird.exe:2372	WRITE 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Offset: 0 Length: 27	
46	11:31:55	thunderbird.exe:2372	READ 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Offset: 0 Length: 4096	
47	11:31:55	thunderbird.exe:2372	SET INFORMATION 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	FileBasicInformation	
48	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		
49	11:31:55	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS		
67	11:31:55	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\	SUCCESS	Options: Open Directory  Access: All	
68	11:31:55	thunderbird.exe:2372	DIRECTORY	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\	SUCCESS	FileBothDirectoryInformation: msgFilterRules.dat	
69	11:31:55	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\	SUCCESS		
70	11:31:55	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS	Options: Open  Access: All	
71	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS	FileAttributeTagInformation	
72	11:31:55	thunderbird.exe:2372	DELETE 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS		
73	11:31:55	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS		
74	11:31:55	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Options: Open  Access: All	
75	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	FileAttributeTagInformation	
76	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	Attributes: A	
77	11:31:55	thunderbird.exe:2372	OPEN	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS	Options: Open  Access: All	
78	11:31:55	thunderbird.exe:2372	SET INFORMATION 	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\tmprules.dat	SUCCESS	FileRenameInformation	
79	11:31:55	thunderbird.exe:2372	CLOSE	D:\USUARIS\castell\THUNDERBIRD\ImapMail\bustia.fib.upc.es\msgFilterRules.dat	SUCCESS		
80	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS	Options: Open Directory  Access: All	
81	11:31:55	thunderbird.exe:2372	DIRECTORY	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS	FileBothDirectoryInformation: tmprules.dat	
82	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\	SUCCESS		
83	11:31:55	thunderbird.exe:2372	OPEN	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	Options: Open  Access: All	
84	11:31:55	thunderbird.exe:2372	QUERY INFORMATION	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS	FileAttributeTagInformation	
85	11:31:55	thunderbird.exe:2372	DELETE 	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		
86	11:31:55	thunderbird.exe:2372	CLOSE	C:\DOCUME~1\CASTELL\TEMP\tmprules.dat	SUCCESS		

I'll attach the log files so you can review (load) it using FileMon.

As we can see, when Thunderbird works properly (i.e. after deleting all tmprules*.dat in the Temp folder of the user), it creates, uses and DELETES tmprules.dat (see lines 85&86).

But, when the problem happens (we don't why it starts) we can see that Thunderbird tries to create tmprules-x.dat files always starting by the first one until it can create a tmprules-x.dat file that doesn't exist. So, as the user works with Thunderbird, more and more files are created/checked, etc. This checking/creation process consumes a lot of CPU (100%) and Thunderbird doesn't respond to keyboard inputs of the user.

Please, can someone check and solve this problem?

Thank you very much.

PS: Note that I can "mask" it by having the PC delete all these temporary files before the user logon but it's better to solve the problem in its root, isn'it?

Comment 38

[Manel Rodero]

Attachment: Thunderbird use of tmprules.dat when is working properly

You can browse the log in a more efficient way using FileMon (from SysInternals/Microsoft).

Comment 39

[Manel Rodero]

Attachment: Thunderbird use of tmprules.dat when is working bad

You can browse the log in a more efficient way using FileMon (from SysInternals/Microsoft).

Comment 40

[Kyle Guinn]

I have to ask:  What is the purpose of creating tmprules.dat if we immediately go and delete it?  I'm assuming that it is used to transfer data from one part of the program to another.  Is there a better way of doing this that doesn't involve writing a temporary file?

Comment 41

[Sander Goudswaard]

Adding myself to Cc list. Reproduced on 2.0.0.6 on XP SP2 with an IMAP account. Filters disappear and new filters are not saved (only in the session), which is observed when ultimately 9999 files have been created in the users's temp dir.
For what it's worth, it only happens for users with non-admin rights. For admin users the issue does not happen. Could this be related to how the file is supposed to be deleted?

Comment 42

[Magnus Melin [:mkmelin]]

So, looking at comment 23 and comment 33. David, should we just land this on branch? 

On trunk, move is used instead of copy, so there should be no need to delete the tmp file. 

http://lxr.mozilla.org/mozilla1.8/source/mailnews/base/search/src/nsMsgFilterService.cpp#183 
http://lxr.mozilla.org/seamonkey/source/mailnews/base/search/src/nsMsgFilterService.cpp#189

Comment 43

[David :Bienvenu]

Comment on attachment 272146 [details] [diff] [review]
Remove tmprules

requesting 2.0.x approval

Comment 44

[Jean-Michel Reghem]

the fix of this bug on branch 2.0 will also fix bug 403907

Comment 45

[Magnus Melin [:mkmelin]]

I think I was wrong in comment 42, trunk too should probably need to delete the tmp file when/if move fails. (But the patch should still land on branch.)

Can someone give me steps to reproduce this? Enable trust spam assassin and make sure program installation dir is not writable, then what? Would like to make sure what I had in mind works...

Comment 46

[Geoff Oakham]

So *that's* where all those files are coming from and *that's* why filters are broken.  I think this is a serious bug.. voting..


I can confirm it's still happening with 2.0.0.9.

Comment 47

[Daniel Veditz [:dveditz]]

Comment on attachment 272146 [details] [diff] [review]
Remove tmprules

approved for 1.8.1.12, a=dveditz for release-drivers

Comment 48

[Reed Loden [:reed]]

MOZILLA_1_8_BRANCH:

Checking in mailnews/base/search/src/nsMsgFilterService.cpp;
/cvsroot/mozilla/mailnews/base/search/src/nsMsgFilterService.cpp,v  <--  nsMsgFilterService.cpp
new revision: 1.48.2.5; previous revision: 1.48.2.4
done

Still open for trunk patch.

Comment 49

[Alexey Gladkov]

I will port this patch on trunk in few days.

Comment 50

[Peter Niemayer]

Alas, while the applied patch does prevent the stray ".tmp" files from being created, it does not fix the problem that SpamAssassin headers are ignored, even if configured otherwise. This is a major problem for all those terrorized by lots of "inline-image-SPAM" which seems not to be detected by the Mozilla-internal junk mail code.

Does anybody have an idea how these two symptoms relate - as they occured first (seemingly) at the same revision level?

Comment 51

[Jean-Michel Reghem]

@Peter: For tje problem with SpamAssassin maybe you could open a new bug (or there is maybe already a dupe ...)
As if the stray tmp file is fixed, this bug will be RESOLVED-FIXED after verification of fix on trunk ...

Have a look to this bug: Bug 381589  "Trust Spamassassin option is not working in Thunderbird 2.0"

Comment 52

[Jean-Michel Reghem]

Bug fix verified on build Thunderbird/2.0.0.12pre ID:2008011303

Comment 53

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As per comment 52 => verfied1.8.1.12

Comment 54

[Alexey Gladkov]

Attachment: Remove tmprules (patch for trunk)

Sorry for a delay.

Comment 55

[Reed Loden [:reed]]

Checking in mailnews/base/search/src/nsMsgFilterService.cpp;
/cvsroot/mozilla/mailnews/base/search/src/nsMsgFilterService.cpp,v  <--  nsMsgFilterService.cpp
new revision: 1.63; previous revision: 1.62
done

Comment 56

[Sander Goudswaard]

Since yesterday's trunk build, a user who had this problem originally, msgFilterRules.dat has been deleted. After restore, the file has been deleted again.
Using IMAP, non-admin XP SP2.

Comment 57

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #56)
> Since yesterday's trunk build, (snip) msgFilterRules.dat has been deleted.
It's Bug 413680.

Comment 58

[Robin Gaster]

Either this bug is now not fixed or something very similar is happening. My message filters have suddenly stopped working. New filters cannot be saved and are eliminated when TB closes.

I have looked hard for the tmprules-xx.dat files, but have not been able to find more than a few stray ones 

I am using 2.0.0.16, WinXP

Comment 59

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #58)
> I have looked hard for the tmprules-xx.dat files, but have not been able to
> find more than a few stray ones 

To Robin Gaster:  
Did you search for tmprules.dat too? ( for xx=0, tmprules.dat instead of tmprules-0.dat ) Did you search for tmprules-xx.dat/tmprules.dat at where? %temp% directory only? Did you search mail directory in which msgFilterRules.dat is held?
In a forum in Japan, following phenomena was reported.
 If garbage of ...\tmprules.dat exists("...\" is path of ...\msgFilterRules.dat)  
 (a)msgFilterRules.dat can't be updated.(Exception is reported to Error Console)
    So all added message filters disappears after restart.
 (b)If msgFilterRules.dat doesn't exist, null msgFilterRules.dat is created.
    After null msgFilterRules.dat creation, problem of (a) occurs.
    So all created message filters disappears after restart.

Comment 60

[Robin Gaster]

yes, I searched for all versions of tmprules*.dta