Closed Bug 2083044 Opened 1 month ago Closed 13 days ago

Exchange: concurrent offline downloads into a folder link message headers to another message's body (wrong message displayed)

Categories

(MailNews Core :: Networking: Exchange, defect)

Thunderbird 158
defect

Tracking

(Not tracked)

RESOLVED FIXED
161 Branch

People

(Reporter: btirmarche, Assigned: btirmarche)

Details

Attachments

(2 files, 2 external files)

Thunderbird 158, Exchange (EWS) account on on-prem Exchange 2013, maildir store, offline download on.

Symptom: clicking message K shows message D. On disk, K's header points to a cur/ file that holds D's body (byte-identical), while K's offlineMessageSize is K's real size. D's body was stored 3 times in the same second, once as an orphan file. At that same moment, stderr logged GetItem: Protocol(XpCom(NS_BASE_STREAM_CLOSED)) twice. It happened at startup: autosync and message display were downloading into the same folder concurrently. After a full Inbox re-download, 43 of 67 headers pointed to another message's body.

Cause:

  • The maildir store tracks one ongoing write per folder, and GetNewMsgOutputStream pre-empts it. FinishNewMessage and DiscardNewMessage look the write up by folder and only MOZ_ASSERT that the stream matches. In release builds, a pre-empted writer commits (or deletes) the current writer's file and gets its name as storeToken.
  • Exchange's MsgFetcher opens the store stream in OnFetchStart, before the GetItem round trip. Any concurrent fetch into the folder (autosync, display, junk classification) therefore pre-empts it.

Patches (stack of 2):

  1. Stores: when the stream does not match, FinishNewMessage fails with NS_ERROR_ABORT, and DiscardNewMessage leaves the other writer alone (maildir; same explicit check for mbox's Finish). Test: test_maildirPreemptedWrite.js.
  2. Exchange: open the store stream when the data arrives. The data and OnFetchStop run in the same main-thread task, so no other write can interleave. Test: test_concurrentOfflineFetch.js.

Built and tested locally (macOS, mozilla-central of 2026-10-16 + comm-central d438ab08ff66): the new tests pass, and both fail without the fixes (the second one with the NS_BASE_STREAM_CLOSED seen in the field). Exchange unit tests 30/30, local unit tests 64/64, lint clean on the changed lines.

Possibly related: bug 2001966, bug 2002241.

Part 1 of 2 (store side, includes test_maildirPreemptedWrite.js). This touches mailnews/local, so it probably needs a back-end reviewer as well.

Part 2 of 2 (Exchange side, includes test_concurrentOfflineFetch.js). Applies on top of part 1.

Both stores track one ongoing write per folder, and GetNewMsgOutputStream()
pre-empts it. FinishNewMessage() and DiscardNewMessage() only asserted (in debug
builds) that the stream was the ongoing one. In release builds, with maildir, a
pre-empted writer committed or deleted the current writer's file, and got its
name as storeToken: its header then pointed to another message's body.

When the stream doesn't match, FinishNewMessage() now fails with
NS_ERROR_ABORT and DiscardNewMessage() leaves the other write alone (maildir;
same explicit check for mbox's FinishNewMessage()).

Assignee: nobody → benoit
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

MsgFetcher opened the store's output stream in OnFetchStart(), before the
GetItem request went over the network, so any other download into the same
folder in the meantime (autosync, message display, junk classification)
pre-empted it. That caused NS_BASE_STREAM_CLOSED failures and, before part 1,
headers linked to another message's body.

Open the stream when the data arrives instead. The data and OnFetchStop() are
delivered in the same main-thread task, so no other write can interleave.

I've put the two patches up for review on Phabricator: D337987 (stores) and D337988 (Exchange).

Attachment #9671126 - Attachment description: Bug 2083044 - Part 1: Don't let a pre-empted write commit or discard another write's message in the maildir and mbox stores. r?#thunderbird-back-end-reviewers → Bug 2083044 - Part 1: Don't let a pre-empted write commit or discard another write's message in the maildir and mbox stores. r=mkmelin

Part 1 (D337987) is accepted and ready to land on its own; part 2 (D337988) is still under review, hence leave-open. A try run is welcome if needed before landing.

Target Milestone: --- → 161 Branch

Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/127546b3070a
Part 1: Don't let a pre-empted write commit or discard another write's message in the maildir and mbox stores. r=mkmelin

Attachment #9671127 - Attachment description: Bug 2083044 - Part 2: Open the store's output stream only when Exchange message data arrives. r?jtracey → Bug 2083044 - Part 2: Open the store's output stream only when Exchange message data arrives. r=jtracey

Part 2 (D337988) is accepted, with a good try run; ready to land. Part 1 already landed.

Pushed by daniel@thunderbird.net:
https://hg.mozilla.org/comm-central/rev/2b0c32918a49
Part 2: Open the store's output stream only when Exchange message data arrives. r=jtracey

Status: ASSIGNED → RESOLVED
Closed: 13 days ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: