1534119 - Crash in nsImapProtocol::HandleMessageDownLoadLine via nsIMAPBodypart, accessing freed memory

Reporter

Description

•

5 years ago

+++ This bug was initially created as a clone of Bug #1444389 (fixed in 60.3.0) +++ -- which was speculated to perhaps related to bug 1264302 (reported for TB38) / bug 1216951 (fixed in TB60) and friends.

bug 1444389 adds a check for null line but the crashing continues. In fact the crash rate increases as more users adopt version 60.x per graph https://crash-stats.mozilla.com/signature/?product=Thunderbird&signature=nsImapProtocol%3A%3AHandleMessageDownLoadLine&date=%3E%3D2018-11-10T11%3A29%3A00.000Z&date=%3C2019-03-10T11%3A29%3A00.000Z#graphs

bp-f3a9a798-86fa-4c1b-979b-d95ff0190308 66 beta

bp-76aa5b6f-9176-412f-b1ba-54ad10190203 60.5.0 A per earlier cresh report there is one email with 18 attachments (17 photos and one rtf file which in aggregate is about 12Mb in size) which is is attempting to download from Yahoo mail and it ceashed every time. Is there a hard limit in the number of attachments or file size in Mozilla Thunderbird as to why it stuggles with this email?

0 xul.dll nsImapProtocol::HandleMessageDownLoadLine(char const*, bool, char*) comm/mailnews/imap/src/nsImapProtocol.cpp:3997
1 xul.dll nsIMAPBodypart::GenerateBoundary(nsIMAPBodyShell*, bool, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:472
2 xul.dll nsIMAPBodypartMultipart::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:983
3 xul.dll nsIMAPBodypartMessage::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:842
4 xul.dll nsIMAPBodyShell::Generate(char*) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:263
5 xul.dll nsImapProtocol::ProcessSelectedStateURL() comm/mailnews/imap/src/nsImapProtocol.cpp:2759
6 xul.dll nsImapProtocol::ProcessCurrentURL() comm/mailnews/imap/src/nsImapProtocol.cpp:1784
7 xul.dll nsImapProtocol::ImapThreadMainLoop() comm/mailnews/imap/src/nsImapProtocol.cpp:1403
8 xul.dll nsImapProtocol::Run() comm/mailnews/imap/src/nsImapProtocol.cpp:1062

bp-76aa5b6f-9176-412f-b1ba-54ad10190203 60.5.3
0 xul.dll nsImapProtocol::HandleMessageDownLoadLine(char const*, bool, char*) comm/mailnews/imap/src/nsImapProtocol.cpp:3997
1 xul.dll nsIMAPBodypart::GenerateEmptyFilling(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:509
2 xul.dll nsIMAPBodypartLeaf::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:591
3 xul.dll nsIMAPBodypartMultipart::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:985
4 xul.dll nsIMAPBodypartMessage::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:842
5 xul.dll nsIMAPBodyShell::Generate(char*) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:263
6 xul.dll nsImapProtocol::ProcessSelectedStateURL() comm/mailnews/imap/src/nsImapProtocol.cpp:2687
7 xul.dll nsImapProtocol::ProcessCurrentURL() comm/mailnews/imap/src/nsImapProtocol.cpp:1784
8 xul.dll nsImapProtocol::ImapThreadMainLoop() comm/mailnews/imap/src/nsImapProtocol.cpp:1403
9 xul.dll nsImapProtocol::Run() comm/mailnews/imap/src/nsImapProtocol.cpp:1062

Jorg K (CEST = GMT+2)

Comment 1

•

5 years ago

Wayne, the last one you quoted is for 60.5.0, but here is one for 60.5.3: bp-c5a965fa-ce5c-462e-9a40-52d400190310.

It crashes at comm/mailnews/imap/src/nsImapProtocol.cpp:3997 which is:

  // if this line is for a different message, or the incoming line is too big
  if (((m_downloadLineCache->CurrentUID() != GetServerStateParser().CurrentResponseUID())  <== 3997
       && !m_downloadLineCache->CacheEmpty()) ||                                     <= also 3997, I wrapped it.
      (m_downloadLineCache->SpaceAvailable() < lineLength + 1) )
    FlushDownloadCache();

So we're dealing with a case with lots of data, like one reporter said, 12 MB with various attachments. So it looks like m_downloadLineCache got free'ed somehow since the crash shows invalid memory: 0xe5e5e5e9.

So it would be good to reproduce it. Meanwhile I spotted something fishy, patch coming.

Wayne Mery (:wsmwk)

Reporter

Comment 2

•

5 years ago

(In reply to Jorg K (GMT+1) from comment #1)

Wayne, the last one you quoted is for 60.5.0, but here is one for 60.5.3: bp-c5a965fa-ce5c-462e-9a40-52d400190310.

Yes - I clearly label them as such and offer them as data for comparison. :) Whether it helps, is related, or not related, is something which can be determined later.

Comment 3

•

5 years ago

Richard's from bug 1505660 ... bp-9d9da253-2cbf-4896-8864-4542e0190115 bug 1534119 [@ nsImapProtocol::HandleMessageDownLoadLine ] - not actively using Thunderbird

Richard, do you have more for us?

Flags: needinfo?(richard.leger)

Richard Leger

Comment 4

•

5 years ago

Since 15/01/2019 I had only three crashes...

bp-8a14849d-581b-435b-97b6-6f4400190318 18/03/2019, 10:29
bp-830c1844-cf6a-48c4-b84b-2cb650190302 02/03/2019, 16:58
bp-ce49864b-66cb-4f0b-8626-f66580190128 28/01/2019, 11:11

I would not recall what caused them... but should be indicated in the submission report for the two last (in the list)... the 18/08 was pending submission so I submitted it just now... do not recall what caused it...

Is that what you wanted?

Flags: needinfo?(richard.leger)

Wayne Mery (:wsmwk)

Reporter

Comment 5

•

5 years ago

(In reply to Richard Leger from comment #4)

Since 15/01/2019 I had only three crashes...

bp-8a14849d-581b-435b-97b6-6f4400190318 [@ SkBitmapCache::PrivateDeleteRec ]
bp-830c1844-cf6a-48c4-b84b-2cb650190302 [@ mozilla::net::nsLoadGroup::AddRequest ]
bug 1508649 - bp-ce49864b-66cb-4f0b-8626-f66580190128 [@ shutdownhang | _PR_MD_WAIT_CV | PR_Wait | PR_CWait | nsMsgCopy::DoCopy ]

Wayne Mery (:wsmwk)

Reporter

Comment 6

•

5 years ago

(In reply to Jorg K (GMT+2) (reduced availability 14-19 of Sept.) from comment #1)

... I spotted something fishy, patch coming.

Did this (patch) get spun off to another bug?

Flags: needinfo?(jorgk)

Jorg K (CEST = GMT+2)

Comment 7

•

5 years ago

Yes, it did, bug 1534124. I should have noted it here, sorry.

Flags: needinfo?(jorgk)

Wayne Mery (:wsmwk)

Reporter

Comment 8

•

3 years ago

Attached image Screen Shot 2021-01-23 at 11.58.05 PM.png — Details

bp-fe0dd441-85d1-4e2e-b28f-533d50210123 0xe5e5e5e9
bp-9c176cb0-d1c9-452f-bbb2-369ad0210123 0xffffffffffffffff

version 78 crash rate is significantly lower compared to version 68

Wayne Mery (:wsmwk)

Reporter

Comment 9

•

3 years ago

(In reply to Wayne Mery (:wsmwk) from comment #8)

Created attachment 9198861 [details]
Screen Shot 2021-01-23 at 11.58.05 PM.png

bp-fe0dd441-85d1-4e2e-b28f-533d50210123 0xe5e5e5e9
bp-9c176cb0-d1c9-452f-bbb2-369ad0210123 0xffffffffffffffff

version 78 crash rate is significantly lower compared to version 68

And a further 10% decrease over the past 6 months.

Depends on: 1175168

Comment 10

•

3 years ago

Anything notable about Antony's crash bp-7adb27d4-5100-42c5-b6a5-676140211011 with 94.0b1 and 94.0b2 ?

Flags: needinfo?(mkmelin+mozilla)

Magnus Melin [:mkmelin]

Comment 11

•

3 years ago

Unfortunately not.

Flags: needinfo?(mkmelin+mozilla)

Antony

Comment 12

•

3 years ago

•

Edited

(In reply to Wayne Mery (:wsmwk) from comment #10)

Anything notable about Antony's crash bp-7adb27d4-5100-42c5-b6a5-676140211011 with 94.0b1 and 94.0b2 ?

Wayne

One thing might help, I updated 93.0b5 to 94.0b1 via the "ABout" window, and did the same for b2 build 1, or, to be more precise, was advised of new update beta2 by popup window and used that prompt.

Regressing to 93.0b5 resolved freeze/crash, (and the button high-light back to the very pale green).

I updated this version to 94.0b2 build 2 by using the FTP file!!!! Since then TB has been stable. So could there be some inherant issue with the patch update version that is NOT apparent with the full file from the FTP?

Antony

Comment 13

•

3 years ago

94.0 b3 - starting to have freezes again and crashes - seems to be when downloading an email, either one I selected or one in background.

Whether it is the one I select that is the problem or always the same one but in the background and TB is attempting to download tht one at the same time as I select one - need to check that.

These are the two logs from this evening BST

bp-36ca62e1-4c2b-4131-84e1-b535f0211020.txt
bp-2360d486-90fc-4998-bc3e-dcf3b0211020.txt

Wayne Mery (:wsmwk)

Reporter

Comment 14

•

3 years ago

The crash rate for beta went up on Oct 8/9 https://crash-stats.mozilla.org/signature/?release_channel=beta&signature=nsImapProtocol%3A%3AHandleMessageDownLoadLine&date=%3E%3D2021-09-22T12%3A50%3A00.000Z&date=%3C2021-10-22T12%3A50%3A00.000Z#graphs to rank it at #15, up through buildid 20211014184052 94.0b3.

So far there are no crashes of 94.0b4 in the 12 hours since it has been released. At the crash rate of 6 per day we should know in 24 hours if it is gone.

Wayne Mery (:wsmwk)

Reporter

Comment 15

•

3 years ago

bp-36ca62e1-4c2b-4131-84e1-b535f0211020 @ pthread_mutex_lock | PR_EnterMonitor | nsImapProtocol::GetPseudoInterrupted Bug 847356 - crash in PR_EnterMonitor via nsImapProtocol::GetPseudoInterrupted | nsIMAPBodypartMultipart::Generate. m_protocolConnection of nsIMAPBodyShell is invalid/destroyed
bp-2360d486-90fc-4998-bc3e-dcf3b0211020 @ nsImapProtocol::HandleMessageDownLoadLine this bug

Imap has got some serious flaws https://mzl.la/3GfGdWM ... but we are making progress https://mzl.la/3Bbg5bI

Antony

Comment 16

•

3 years ago

been running 94.0b5 for days now and seems to be stable so far... will also rely to the beta testing email

Wayne Mery (:wsmwk)

Reporter

Comment 17

•

3 years ago

The last crashing buildid is 20211026214349 .... which is 94.0b5 ... So there are crashes for the version you are running. But no crashes for beta 95.

https://crash-stats.mozilla.org/signature/?release_channel=%21release&signature=nsImapProtocol%3A%3AHandleMessageDownLoadLine&date=%3E%3D2021-05-13T13%3A06%3A00.000Z&date=%3C2021-11-13T13%3A06%3A00.000Z#graphs

Flags: needinfo?(acdp)

Updated

•

2 years ago

Depends on: 1742991

Antony

Comment 18

•

2 years ago

(In reply to Wayne Mery (:wsmwk) from comment #17)

The last crashing buildid is 20211026214349 .... which is 94.0b5 ... So there are crashes for the version you are running. But no crashes for beta 95.

https://crash-stats.mozilla.org/signature/?release_channel=%21release&signature=nsImapProtocol%3A%3AHandleMessageDownLoadLine&date=%3E%3D2021-05-13T13%3A06%3A00.000Z&date=%3C2021-11-13T13%3A06%3A00.000Z#graphs

Still getting random crashes, or app just freezes, even on the latest, 96.0b2

Flags: needinfo?(acdp)

Wayne Mery (:wsmwk)

Reporter

Comment 19

•

2 years ago

(In reply to Antony from comment #18)

Still getting random crashes, or app just freezes, even on the latest, 96.0b2

Antony,
What are your crash IDs from the past two months?
And, are you also crashing with 96.0b3?

Flags: needinfo?(acdp)

Antony

Comment 20

•

2 years ago

(In reply to Wayne Mery (:wsmwk) from comment #19)

(In reply to Antony from comment #18)

Still getting random crashes, or app just freezes, even on the latest, 96.0b2

Antony,
What are your crash IDs from the past two months?
And, are you also crashing with 96.0b3?

Wayne

Now on 97.0b3 - get random TB shutdowns (in that not reported as a Crash), just closes on me. No specific action either.. living with it for the mo, but is rather a bore
This was the last "official" crash - bp-791528ab-390f-4350-8968-796431220125

Flags: needinfo?(acdp)

gene smith

Comment 21

•

2 years ago

From comment 20:

This was the last "official" crash - bp-791528ab-390f-4350-8968-796431220125

Says "not found". Are these deleted after a certain time?

BMO Automation

Updated

•

2 years ago

Severity: critical → S2

Wayne Mery (:wsmwk)

Reporter

Comment 22

•

1 year ago

Signature no longer exists in any version 102. But can't pinpoint when the crash rate changed because the version 91 crashes have aged off. I doubt bug 1742991 helped (it landed in 91.5.1).

(In reply to gene smith from comment #21)

From comment 20:

This was the last "official" crash - bp-791528ab-390f-4350-8968-796431220125

Says "not found".

It was not found because that bugzilla link goes to crash-stats.mozilla.org. But the crash was not submitted to mozilla.org, it was on the short lived https://crash-stats.thunderbird.net/ (may it never return).

Are these deleted after a certain time?

Crash reports older than 6 months are deleted from crash-stats.mozilla.org. strict cutoff.

Summary: Crash in nsImapProtocol::HandleMessageDownLoadLine, accessing freed memory → Crash in nsImapProtocol::HandleMessageDownLoadLine via nsIMAPBodypart, accessing freed memory

Wayne Mery (:wsmwk)

Reporter

Comment 23

•

1 year ago

•

Edited

Signature no longer exists in any version 102.

Like Bug 1627442 .

Perhaps bug 847356 helped? It was fixed in beta 98 but never uplifted to 91. The patch "Ensure nsImapProtocol is kept in existence for duration of nsImapBodyShell::Generate"

Status: NEW → RESOLVED

Closed: 1 year ago

Resolution: --- → WORKSFORME