1497819 - Session.json info is lost in via network file shared folder, losing customizations

Status: RESOLVED FIXED

(Thunderbird :: General, defect)

Description

[hartnegg]

Thunderbird 60 introduces an incompatibility with Shared Folders of VirtualBox.

If the profile directory is stored in such a folder, several session files get the suffix "tmp" appended to their names. For example "session.json.tmp" instead of "session.json". Other affected files are extensions, sessionCheckpoints, xulstore and addons.

This causes Thunderbid to forget various settings each time it gets restarted.

The settings are preserved if I manually rename the files before starting Thunderbird:
ren extensions.json.tmp extensions.json
ren session.json.tmp session.json
ren sessionCheckpoints.json.tmp sessionCheckpoints.json
ren xulstore.json.tmp xulstore.json
ren addons.json.tmp addons.json

This is new in Thunderbird 60.x. Older versions worked fine with session directory in a VirtualBox shared folder.

The problem does not occur on a normal network shared folder, only with the folders from the host passed through VirtalBox to the guest OS. Eventhough they look to Windows exactly the same. Somehow Thunderbird now detects a difference, and give the session files incorrect names.

Comment 1

[Jorg K (CEST = GMT+2)]

None of what you're doing is supported. That said, we use Mozilla platform code for the handling of all those files. It the same problem exists in Firefox, I'll move the bug to the Firefox queue so it will get some attention.

Comment 2

[AndresPena]

I am having a similar issue, but without VirtualBox involved. We have all the profiles stored at network drives (Win7, Thunderbird 60.3.0), and since version 60.x we have this .tmp files situation. If we remove the .tmp extension, Thunderbird deletes the file a creates a brand new .json file with default contents, so every time we run Thunderbird we lose all customizations.

Comment 3

[hartnegg]

The Browser-Console reports "Win error 32 during operation rename on file F:\Thunderbird\default\extensions.json". The process cannot access the file because it is already used by another process. 
resource://gre/modules/DeferredTask.jsm:313:7
self-hosted:1229:9

Comment 4

[most]

Same problem here.
Windows 10, Thunderbird 60.4, the Thunderbird profile is stored on a network drive.
We have also this .tmp files problem as described above. It came with our latest update of Thunderbird - I don't know the prvious version, now it is 60.4. Thunderbird worked with the profiles on a network drive for many years.
I see also this rename error in the Browser-Console. Example for one of the files:
Win error 32 during operation rename on file \Server\Mail-Most\session.json (Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.

) DeferredTask.jsm:313
_runTask resource://gre/modules/DeferredTask.jsm:313:7
throw self-hosted:1229:9

It seems that Thunderbird saves the settings the following way:

1. save the latest settings in .tmp files
2. delete the original files
3. rename the .tmp files to the original files

And it seems that the last step does not work. Maybe because the network drive still performes step 2 at this moment?
Maybe there is some retry and/or some waiting time necessary between step 2 and step 3 ?

Comment 5

[Jorg K (CEST = GMT+2)]

OK, revisiting this, finally the penny dropped. session.json is a file administered by the Mozilla platform code which Thunderbird uses for 90% of it's operation.

Does this problem exist in Firefox, that is, do a similar operation using FF. If so, the bug needs to go to the Firefox queue.

(Oh, I already said that in comment #1)

Comment 6

[most]

I don't know if this problem exists in Firefox - we only have Thunderbird profiles on the network drive.

And I want to underline that in my case the Thunderbird profile is NOT on VirtualBox shared folder but just on a share on a server in our network. So I wonder if my post is at the right place here...

Comment 7

[Jorg K (CEST = GMT+2)]

Well, in this case your post isn't in the right place. Please try this with Firefox, and if it fails, file a bug for Firefox.

Comment 8

[Wayne Mery (:wsmwk)]

Not important enough for someone to have tested Firefox or filed followup bugs?

Anyway, yes, there are definitely integrity issues with file shares. https://mzl.la/3cVSSPe is only a partial list - should be easy to riff on that to connect this to some worthy actionable bug(s).

Comment 9

[alta88]

Attachment: Bug1497819-backupTo.patch

backupTo should be added to session store, and will help with a lost session file here; the file lock problem may be made obsolete though not necessarily fixed.

Comment 10

[Ben Campbell]

Comment on attachment 9218039 [details] [diff] [review]
Bug1497819-backupTo.patch

Review of attachment 9218039 [details] [diff] [review]:
-----------------------------------------------------------------

r+ because I think it'll help here.
But I'd guess there are a whole bunch of other places with similar issues.

Comment 11

[Ben Campbell]

Just a few more notes:

JSONFile always uses writeAtomic() with a tmpFile.

writeAtomic() implementation appears to be here:
https://searchfox.org/mozilla-central/source/toolkit/components/osfile/NativeOSFileInternals.cpp#931

The steps in comment #4 aren't quite right - it doesn't delete the file before renaming the tmpFile - it merely renames tmpFile to the real file (which overwrites file if it already exists). This is the pretty standard way of doing "atomic" file writes.
I don't know enough about network shares or virtualbox to know what extra things might go wrong there. I do know that atomic write saves to network shares is an historically thorny issue with no real good solution.

I don't quite see why the .json file would be locked - as per comment #3 and comment #4. I could imagine that the user might have closed (or crashed) TB then reopened it, and the network filesystem still thinks the json file is locked for a while?
I'm assuming that we're not talking about multiple instances of thunderbird sharing the same profile dir, right? That would seem to be a recipe for disaster...

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/263efd97c221
Add backupTo to session store JSONFile. r=benc