1490637 - Crash in msmpeg2vdec.dll@0x25a3fb correlated to Trend Micro

Status: RESOLVED WORKSFORME

(External Software Affecting Firefox :: Other, defect)

Description

[[:philipp]]

This bug was filed from the Socorro interface and is
report bp-2c5302fa-efbd-49dc-a7a6-77e3a0180907.
=============================================================

Top 2 frames of crashing thread:

0 msmpeg2vdec.dll msmpeg2vdec.dll@0x25a3fb 
1 ntdll.dll RtlpLookupDynamicFunctionEntry 

=============================================================

this crash signature is spiking up since 2018-09-06 with 64bit builds of firefox on windows 7. user comments are generally indicating that this is a repeat crash hitting every minute or so, so it's having a particularly bad impact on affected users.  

correlations show that this is fully correlated to modules from Trend Micro Security software, so likely a recent signature update of them is causing this instability:
(100.0% in signature vs 00.99% overall) Module "TmUmEvt64.dll" = true
(100.0% in signature vs 00.99% overall) Module "tmmon64.dll" = true

the crashing trend with a low on weekends is indicating that it's primarily affecting users in an enterprise environment and we also got this report on sumo about it: https://support.mozilla.org/en-US/questions/1233453

Comment 1

[Julien Cristau [:jcristau]]

Marco, would you mind reaching out?

Comment 2

[Marco Castelluccio [:marco]]

I've reached out to the only contact I have. I see Makoto and Ryan were in contact with Trend Micro in bug 1196899, maybe they can help reaching out again?

Comment 3

[Ryan VanderMeulen [:RyanVM]]

I've emailed my contacts and CCed you.

Comment 4

[Pascal Chevrel:pascalc]

The number of crashes on beta were divided by 3 on both beta and release over the last 3 weeks and the last crash with Nightly 64 was also 3 weeks ago. I suppose that an update to Trend Micro fixed the issue upstream and that the numbers will keep on improving as people update their security software. Untracking for 63 as a consequence but leaving the 63 status as affected to keep an eye on the 63 crashes in the coming weeks.

Comment 5

[[:philipp]]

unfortunately the recent numbers don't look like the issue has received a fix from trend micro yet. should we poke them about the current state again?

Comment 6

[Marco Castelluccio [:marco]]

(In reply to [:philipp] from comment #5)
> unfortunately the recent numbers don't look like the issue has received a
> fix from trend micro yet. should we poke them about the current state again?

Trend Micro was able to get logs to investigate the crash and is working on a hotfix that should be released in November.

Comment 7

[Marco Castelluccio [:marco]]

Trend Micro has released the hotfix: https://esupport.trendmicro.com/en-us/home/pages/technical-support/1121641.aspx.
More details about the issue in the hotfix readme at https://esupport.trendmicro.com/media/14013962/Ti_150_win_en_amsp_hfb8.10.1054_readme.txt:
> The "TmUmEvt.dll" library should stop stack tracing once the
> "msmpeg2vdec.dll" library loads. However, the function that checks
> if the library has loaded will not work before AMSP starts, which
> may cause which may cause Mozilla(R) Firefox(R) to crash if it
> starts before Trend Micro Security Software gets ready.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.