Exchange: concurrent offline downloads into a folder link message headers to another message's body (wrong message displayed)
Categories
(MailNews Core :: Networking: Exchange, defect)
Tracking
(Not tracked)
People
(Reporter: btirmarche, Assigned: btirmarche)
Details
Attachments
(2 files, 2 external files)
|
Part 1: maildir/mbox stores - don't let a pre-empted write commit or discard another write's message
11.40 KB,
patch
|
Details | Diff | Splinter Review | |
|
11.92 KB,
patch
|
Details | Diff | Splinter Review |
Thunderbird 158, Exchange (EWS) account on on-prem Exchange 2013, maildir store, offline download on.
Symptom: clicking message K shows message D. On disk, K's header points to a cur/ file that holds D's body (byte-identical), while K's offlineMessageSize is K's real size. D's body was stored 3 times in the same second, once as an orphan file. At that same moment, stderr logged GetItem: Protocol(XpCom(NS_BASE_STREAM_CLOSED)) twice. It happened at startup: autosync and message display were downloading into the same folder concurrently. After a full Inbox re-download, 43 of 67 headers pointed to another message's body.
Cause:
- The maildir store tracks one ongoing write per folder, and
GetNewMsgOutputStreampre-empts it.FinishNewMessageandDiscardNewMessagelook the write up by folder and onlyMOZ_ASSERTthat the stream matches. In release builds, a pre-empted writer commits (or deletes) the current writer's file and gets its name as storeToken. - Exchange's
MsgFetcheropens the store stream inOnFetchStart, before the GetItem round trip. Any concurrent fetch into the folder (autosync, display, junk classification) therefore pre-empts it.
Patches (stack of 2):
- Stores: when the stream does not match,
FinishNewMessagefails withNS_ERROR_ABORT, andDiscardNewMessageleaves the other writer alone (maildir; same explicit check for mbox's Finish). Test:test_maildirPreemptedWrite.js. - Exchange: open the store stream when the data arrives. The data and
OnFetchStoprun in the same main-thread task, so no other write can interleave. Test:test_concurrentOfflineFetch.js.
Built and tested locally (macOS, mozilla-central of 2026-10-16 + comm-central d438ab08ff66): the new tests pass, and both fail without the fixes (the second one with the NS_BASE_STREAM_CLOSED seen in the field). Exchange unit tests 30/30, local unit tests 64/64, lint clean on the changed lines.
Possibly related: bug 2001966, bug 2002241.
| Assignee | ||
Comment 1•1 month ago
|
||
Part 1 of 2 (store side, includes test_maildirPreemptedWrite.js). This touches mailnews/local, so it probably needs a back-end reviewer as well.
| Assignee | ||
Comment 2•1 month ago
|
||
Part 2 of 2 (Exchange side, includes test_concurrentOfflineFetch.js). Applies on top of part 1.
| Assignee | ||
Comment 3•20 days ago
|
||
Both stores track one ongoing write per folder, and GetNewMsgOutputStream()
pre-empts it. FinishNewMessage() and DiscardNewMessage() only asserted (in debug
builds) that the stream was the ongoing one. In release builds, with maildir, a
pre-empted writer committed or deleted the current writer's file, and got its
name as storeToken: its header then pointed to another message's body.
When the stream doesn't match, FinishNewMessage() now fails with
NS_ERROR_ABORT and DiscardNewMessage() leaves the other write alone (maildir;
same explicit check for mbox's FinishNewMessage()).
Updated•20 days ago
|
| Assignee | ||
Comment 4•20 days ago
|
||
MsgFetcher opened the store's output stream in OnFetchStart(), before the
GetItem request went over the network, so any other download into the same
folder in the meantime (autosync, message display, junk classification)
pre-empted it. That caused NS_BASE_STREAM_CLOSED failures and, before part 1,
headers linked to another message's body.
Open the stream when the data arrives instead. The data and OnFetchStop() are
delivered in the same main-thread task, so no other write can interleave.
| Assignee | ||
Comment 5•19 days ago
|
||
I've put the two patches up for review on Phabricator: D337987 (stores) and D337988 (Exchange).
Updated•15 days ago
|
| Assignee | ||
Comment 6•15 days ago
|
||
Part 1 (D337987) is accepted and ready to land on its own; part 2 (D337988) is still under review, hence leave-open. A try run is welcome if needed before landing.
Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/127546b3070a
Part 1: Don't let a pre-empted write commit or discard another write's message in the maildir and mbox stores. r=mkmelin
Updated•13 days ago
|
| Assignee | ||
Comment 8•13 days ago
|
||
Part 2 (D337988) is accepted, with a good try run; ready to land. Part 1 already landed.
Pushed by daniel@thunderbird.net:
https://hg.mozilla.org/comm-central/rev/2b0c32918a49
Part 2: Open the store's output stream only when Exchange message data arrives. r=jtracey
Description
•