1952503 - Intermittent comm/mailnews/local/test/unit/test_over4GBMailboxes.js | xpcshell return code: 0 (Unexpected exception 2152857614)

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: brendan [at] thunderbird.net
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=498040971&repo=comm-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CKdyNoxBQ8eNYD6qCv6pNQ/runs/0/artifacts/public/logs/live_backing.log

---

Comment 1

[Intermittent Failures Robot]

Summary

2 failures in 4009 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 2

Platform and build breakdown:

• windows11-64-2009-ccov-qr: 2
  • opt: 2

Table

	no_variant
windows11-64-2009-ccov-qr--/opt	2

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-03-03&endday=2025-03-09&tree=all

Comment 2

[Intermittent Failures Robot]

Summary

2 failures in 4826 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 2

Platform and build breakdown:

• linux1804-64-ccov-qr: 2
  • opt: 2

Table

	no_variant
linux1804-64-ccov-qr--/opt	2

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-03-10&endday=2025-03-16&tree=all

Comment 3

[Magnus Melin [:mkmelin]]

https://treeherder.mozilla.org/logviewer?job_id=500208516&repo=comm-central&lineNumber=5133-5134

[task 2025-03-21T03:12:22.116Z] 03:12:22     INFO -  Unexpected exception Component returned failure code: 0x80550005 [nsIMsgLocalMailFolder.getDatabaseWOReparse]
[task 2025-03-21T03:12:22.116Z] 03:12:22     INFO -  compactUnder4GiB@/builds/worker/workspace/build/tests/xpcshell/tests/comm/mailnews/local/test/unit/test_over4GBMailboxes.js:450:10

Comment 4

[Magnus Melin [:mkmelin]]

Tentatively marking this a regression from bug 1925117, which is a bit worrying.
Ben, any ideas? The 0x80550005 is NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE.

Comment 5

[Ben Campbell]

My guess is that it's failing on this line, immediately after the compact:

Assert.ok(gInbox.msgDatabase.summaryValid);

https://searchfox.org/comm-central/rev/bcacae34b10c1bc8ade08a3eecbb943af0209a07/mailnews/local/test/unit/test_over4GBMailboxes.js#477

My guess would be that it's failing in nsMsgLocalMailFolder::GetMsgDatabase(),
which calls nsMsgLocalMailFolder::GetDatabaseWOReparse(),
which calls nsMsgLocalMailFolder::OpenDatabase(),
which calls nsMsgDBService::OpenFolderDB()...

There's a potentially relevant note I left here about OpenFolderDB() returning NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE...
https://searchfox.org/comm-central/source/mailnews/base/src/FolderCompactor.cpp#683

The nsMsgDBService has a ropey-looking db cache. Databases remove themselves from that cache when they are destroyed, but maybe in this case the test is holding the old database open (maybe via a car containing a nsIMsgDBHdr?), meaning the compaction doesn't destroy the old database, and GetMsgDatabase() call retrieves the old cached one?

Comment 6

[Magnus Melin [:mkmelin]]

Well, the failure from comment 3 is earlier, at https://searchfox.org/comm-central/rev/bcacae34b10c1bc8ade08a3eecbb943af0209a07/mailnews/local/test/unit/test_over4GBMailboxes.js#450

Comment 7

[Hartmut Welpmann [:welpy-cw]]

This already happened when part 2 of bug 1925117 was checked in, with error 0x8052000E (2152857614), that's NS_ERROR_FILE_IS_LOCKED, see https://treeherder.mozilla.org/logviewer?job_id=497569832&repo=comm-central&lineNumber=4245 from Windows 11 x64 22H2 CCov WebRender opt X7.

Comment 8

[Intermittent Failures Robot]

Summary

5 failures in 5035 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 5

Platform and build breakdown:

• linux1804-64-ccov-qr: 3
  • opt: 3
• linux1804-64-tsan-qr: 1
  • opt: 1
• windows11-64-2009-ccov-qr: 1
  • opt: 1

Table

	no_variant
linux1804-64-ccov-qr--/opt	3
linux1804-64-tsan-qr--/opt	1
windows11-64-2009-ccov-qr--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-03-17&endday=2025-03-23&tree=all

Comment 9

[Ben Campbell]

Attachment: Bug 1952503 - Add trace output to test_over4GBMailboxes.js to help find intermittent failures. r=#thunderbird-reviewers

Comment 10

[Ben Campbell]

Doh. My comment 5 analysis was looking at the NS_ERROR_FILE_IS_LOCKED crash (although I got confused and was looking for places in that path that threw NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE).

I think the NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE is a different crash signature?

All the crash signatures so far seem to be in TSAN and CCOV builds, with optimisations on, for windows and linux.

In any case, it's pretty intermittent, and I haven't been able to repeat it locally.
I think the thing is to just keep adding instrumentation to drill down using the treeherder logs, and help us figure out what's happening.

First step: figuring out exactly which call in the test is failing.

Comment 11

[Ben Campbell]

Just had a failure which was definitely GetDatabaseWOReparse() returning NS_MSG_ERROR_FOLDER_SUMMARY_OUT_OF_DATE, more-or-less immediately after a compaction (in the previous subtest).
https://firefoxci.taskcluster-artifacts.net/MZtfaYF5Rz2TtUlLSiCTUQ/0/public/logs/live_backing.log

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/806cb411d480
Add trace output to test_over4GBMailboxes.js to help find intermittent failures. r=tobyp

Comment 13

[Intermittent Failures Robot]

Repository breakdown:

• comm-central: 7

Table

	no_variant
linux1804-64-ccov-qr--/opt	4
windows11-64-2009-ccov-qr--/opt	3

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-03-24&endday=2025-03-30&tree=all

Comment 14

[Intermittent Failures Robot]

10 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 10

Table

	no_variant
linux1804-64-ccov-qr--/opt	3
linux1804-64-tsan-qr--/opt	5
windows11-64-2009-ccov-qr--/opt	2

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-03-31&endday=2025-04-06&tree=all

Comment 15

[Intermittent Failures Robot]

6 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 6

Table

	no_variant
linux1804-64-ccov-qr--/opt	2
linux1804-64-tsan-qr--/opt	3
windows11-64-2009-ccov-qr--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-04-07&endday=2025-04-13&tree=all

Comment 16

[Intermittent Failures Robot]

3 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 3

Table

	no_variant
linux1804-64-ccov-qr--/opt	1
linux1804-64-tsan-qr--/opt	1
windows11-64-24h2-ccov--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-04-14&endday=2025-04-20&tree=all

Comment 17

[Intermittent Failures Robot]

3 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 3

Table

	no_variant
linux1804-64-ccov-qr--/opt	1
linux1804-64-tsan-qr--/opt	1
windows11-64-24h2-ccov--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-04-21&endday=2025-04-27&tree=all

Comment 18

[Intermittent Failures Robot]

4 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 4

Table

	no_variant
linux1804-64-ccov-qr--/opt	3
linux1804-64-tsan-qr--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-04-28&endday=2025-05-04&tree=all

Comment 19

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 1

Table

	no_variant
linux1804-x86_64/ccov	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-05-05&endday=2025-05-11&tree=all

Comment 20

[Intermittent Failures Robot]

2 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 2

Table

	no_variant
linux1804-x86_64/ccov	1
windows11-x86_64/ccov	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-05-12&endday=2025-05-18&tree=all

Comment 21

[Intermittent Failures Robot]

3 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 3

Table

	no_variant
linux1804-x86_64/ccov	1
windows11-x86_64/ccov	2

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-05-19&endday=2025-05-25&tree=all

Comment 22

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 1

Table

	no_variant
windows11-x86_64/ccov	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1952503&startday=2025-06-09&endday=2025-06-15&tree=all

Comment 23

[Hartmut Welpmann [:welpy-cw]]

Might be fixed by bug 1959858.