1780549 - Repairing an IMAP folder while in offline mode deletes mbox file

Status: RESOLVED FIXED

(MailNews Core :: Database, defect, P2)

Description

[1248]

Steps to reproduce:

1. set thunderbird to offline mode
2. right click on a folder (in my case, IMAP)
3. --> properties
4. --> repair

Actual results:

The folder is emptied. All messages are gone. On the hard disk, the msf file is updated, and the corresponding mbox file is deleted

Expected results:

the msf file shoud be rebuild, but no messages should be deleted!)

My thunderbird version:

Allgemeine Informationen

Name: Thunderbird
Version: 102.0.3
Build-ID: 20220718182443
Distributions-ID:

Update-Kanal: release
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.0.3
Betriebssystem: Windows_NT 10.0 19044
Betriebssystem-Theme:

Starter-Prozess: Aktiviert
Fenster mit mehreren Prozessen: 0/0
Fission-Fenster: 0/0
          Standardmäßig aktiviert
Externe Prozesse: 2
Unternehmensrichtlinien: Inaktiv
Google-Location-Service-Schlüssel: Fehlt
Google-Safebrowsing-Schlüssel: Fehlt
Mozilla-Location-Service-Schlüssel: Fehlt
Abgesicherter Modus: false
Speichergröße (RAM): 4,0 GB
Speicherplatz verfügbar: 30,3 GB

Comment 1

[Andrei Hajdukewycz [:sancus]]

This isn't a regression, it happens in 91 as well and probably earlier versions. I wasn't able to find a copy of this filed beforehand, however.

IMAP repair folder doesn't just rebuild the msf file, it redownloads all messages as well. So if you attempt this offline, it can't do that, which results in deletion. If you go online and repair folder it fixes it.

Either Repair Folder should be greyed out for IMAP folders while offline, or maybe the code should check if you're offline and then only rebuild the msf file.

I don't see how repairing IMAP folders while offline is a useful operation, so I would accept the grey out solution if it's simpler.

Comment 2

[Wayne Mery (:wsmwk)]

any matches in https://mzl.la/3aTzlDs ?

Comment 3

[Andrei Hajdukewycz [:sancus]]

(In reply to Wayne Mery (:wsmwk) from comment #2)

any matches in https://mzl.la/3aTzlDs ?

I didn't see anything based on the summaries.

Comment 4

[1248]

First of all, thanks for the quick investigation. Indeed it seems not to be critical after all.

If I may comment on the options for solution:
There is a case for wanting to requild msf offline, namely if you have many messages or big messages in the folder and/or a slow internet connection. In this case it is much more convenient to just rebuild the msf file. From my experience it seems to me anyway that in the mjority of cases problems occur in the msf files, not so much in the mbox file with the downloaded messages.

If you choose anyway to solve this bug with the "grey out" option, then it would be highly appreciated if you the at least tell the user somehow why the button is greyed out. Otherwise I see users asking for support because they will not understand why the button is not working. As an alternative I could imagine that you do not diable the button, but insert a check for online state in the click event. If the state is offline, then you could pop up a message box and tell the user that he needs to go online for repairing.

Just my 2 ct...

Comment 5

[Andrei Hajdukewycz [:sancus]]

(In reply to 1248 from comment #4)

There is a case for wanting to requild msf offline, namely if you have many messages or big messages in the folder and/or a slow internet connection. In this case it is much more convenient to just rebuild the msf file.

Perhaps so, the problem is it may be difficult to guarantee that the message data is actually complete especially with IMAP. And having a button with the potential to cause destructive results is worse than not having one.

But honestly, I'm not an expert on this part of the code. And we also need to consider the POP3 case. I'm not sure what happens there. Repair works fine on Local Folders when offline, at least.

@BenC:
Should we fix Repair Folder to use the local message store when offline, is that a significant amount of work? Is it possible to make sure it won't be a destructive operation before allowing it?

If we go with a UI-based solution instead, we can check with Alex to see what is the best way to convey that Repair Folder is disabled due to being in offline mode.

Comment 6

[Ben Campbell]

(In reply to Andrei Hajdukewycz [:sancus] from comment #5)

@BenC:
Should we fix Repair Folder to use the local message store when offline, is that a significant amount of work? Is it possible to make sure it won't be a destructive operation before allowing it?

In theory it should be possible to leverage the local folder repair to rebuild an IMAP msgDB, but in practice I know there would be a whole heap of really sharp edges that would make it a nightmare - both to get right in the first place, and in terms of future maintenance. So it'd be a big job.

I think it might be a worthwhile goal, but I'd want to see a lot of cleanup and documentation before then - I'd want to see it essentially "fall out for free" from a refactored and reusable local folder repair and from having a solid concept on the IMAP side of having "provisional" messages which were robustly mapped to server-side messages when the network becomes available (there is a concept already of queued-up, offline IMAP operations, but I'm not familiar enough with that to know how such provisional messages are represented in the msgDB).

Really, I think the better goal would be to have .msf files that didn't break and require rebuilding in the first place :-)

Comment 7

[Andrei Hajdukewycz [:sancus]]

(In reply to Ben Campbell from comment #6)

In theory it should be possible to leverage the local folder repair to rebuild an IMAP msgDB, but in practice I know there would be a whole heap of really sharp edges that would make it a nightmare - both to get right in the first place, and in terms of future maintenance. So it'd be a big job.
[...]
Really, I think the better goal would be to have .msf files that didn't break and require rebuilding in the first place :-)

Agree, this isn't a good use of our time currently. Maybe there is a simpler patch to grey out the repair buttons so that people don't shoot themselves in the foot at least. But honestly, I suspect this bug has been around for many years and I don't think this is urgent.

Comment 8

[Andrei Hajdukewycz [:sancus]]

Here's what we should do:

1. I haven't checked what happens to a POP3 account if you do this. Maybe it still works. That should be determined.

2. Disable the Repair Folder button for at least IMAP accounts when offline, maybe also POP3.

3. If #2 is awkward/difficult, then we can just disable the Repair Folder button when in offline mode in all cases. That is better than allowing people to blow up their own data in some cases.

I'm probably not going to assign someone to this immediately, in any case.

Comment 9

[Magnus Melin [:mkmelin]]

For POP3 there is no problem.

In theory we should be able to just prevent the deletion of the offline store in offline mode for imap. We could, but turns out after repair the folder content disappears from view and doesn't show up until you go online anyway so I think it's best to just disable Repair for IMAP while in offline mode.

Comment 10

[Magnus Melin [:mkmelin]]

Attachment: Bug 1780549 - Disable Repair Folder for IMAP folders when in offline mode. r=benc

Comment 11

[Ryan Lee Sipes]

Thank you Magnus, let's make sure this is closed when the patch is landed.

Comment 12

[Magnus Melin [:mkmelin]]

Let's get this landed.
It does seem we should warn the user before we blow nuke their potentially tens of gigabytes of offline store. Will file a new bug for that.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/aa494a498a31
Disable Repair Folder for IMAP folders when in offline mode. r=benc

Comment 14

[Magnus Melin [:mkmelin]]

Comment on attachment 9287362 [details]
Bug 1780549 - Disable Repair Folder for IMAP folders when in offline mode. r=benc

[Approval Request Comment]
Regression caused by (bug #): not a regression
User impact if declined: bad experience with repair folder if offline
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): safe

Comment 15

[Wayne Mery (:wsmwk)]

Comment on attachment 9287362 [details]
Bug 1780549 - Disable Repair Folder for IMAP folders when in offline mode. r=benc

[Triage Comment]
Approved for beta

Comment 16

[Rob Lemley [:rjl]]

Thunderbird 104.0b3:
https://hg.mozilla.org/releases/comm-beta/rev/6a67a6b5faa7

Comment 17

[Wayne Mery (:wsmwk)]

Comment on attachment 9287362 [details]
Bug 1780549 - Disable Repair Folder for IMAP folders when in offline mode. r=benc

[Triage Comment]
Approved for esr102

Comment 18

[Rob Lemley [:rjl]]

Thunderbird 102.2.0:
https://hg.mozilla.org/releases/comm-esr102/rev/fc399ff31912