Closed Bug 536974 Opened 15 years ago Closed 8 years ago

[AcrobatReader] sharp rise in Firefox Crashes on [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]

Categories

(Plugins Graveyard :: PDF (Adobe), defect)

Product:
Plugins Graveyard
Component:
PDF (Adobe)
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: chofmann, Unassigned)

References

Details

(Keywords: crash, Whiteboard: [crashkill][crashkill-outreach][explosive?])

Crash Data

there is a jump in acrobat reader crashes in the past 3 days on 3.6b5 and maybe other releases. up 41 slots, to current rank as the #15 top crash.

stacks look like 
http://crash-stats.mozilla.com/report/index/050b43d9-2ac8-452f-9ff9-b89c82091228

Frame  	Module  	Signature [Expand]  	Source
0 	Multimedia.api 	Multimedia.api@0x42f8b 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x262d7 	
4 	EScript.api 	EScript.api@0x55f20 	
5 	AcroRd32.dll 	AcroRd32.dll@0x130814 	
6 	user32.dll 	InternalCallWinProc 	
7 	user32.dll 	UserCallWinProcCheckWow 	
8 	user32.dll 	DispatchMessageWorker 	
9 	user32.dll 	DispatchMessageW 	
10 	AcroRd32.dll 	AcroRd32.dll@0xf2c93 	
11 	AcroRd32.dll 	AcroRd32.dll@0xf2698 	
12 	AcroRd32.dll 	AcroRd32.dll@0x38ce 	
13 	AcroRd32.dll 	AcroRd32.dll@0x1922d2

most look like AcroRd32.dll  	9.2.0.124 is involved

more reports at http://crash-stats.mozilla.com/report/list?range_value=2&range_unit=weeks&signature=Multimedia.api%400x42f8b&version=Firefox%3A3.6b5

hopefully this increase in crashes is not related to attempts at exploiting the current unpatched zero-day

https://bugzilla.mozilla.org/show_bug.cgi?id=524460#c3

we should watch for continued crash growth, and consider blocking 9.2.0.124 as soon as the new update is released.  plugin update page should also help get users updated of that version as soon as possible.
Flags: wanted1.9.2?
also related is a similar stack which as jumped #171 slots to rank #22


http://crash-stats.mozilla.com/report/index/5c4d88d9-5758-4bcb-b22d-7e49f2091225

Frame  	Module  	Signature [Expand]  	Source
0 		@0xc0c0c0c 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x262d7 	
4 	EScript.api 	EScript.api@0x55f20 	
5 	AcroRd32.dll 	AcroRd32.dll@0x130814 	
6 	user32.dll 	InternalCallWinProc 	
7 	user32.dll 	UserCallWinProcCheckWow 	
8 	user32.dll 	DispatchMessageWorker 	
9 	user32.dll 	DispatchMessageW 	
10 	AcroRd32.dll 	AcroRd32.dll@0xf2c93 	
11 	AcroRd32.dll 	AcroRd32.dll@0xf2698 	
12 	AcroRd32.dll 	AcroRd32.dll@0x38ce 	
13 	AcroRd32.dll 	AcroRd32.dll@0x1922d2
Summary: Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] → Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ]
Summary: Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] → sharp rise in Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ]
also  [@ @0x0 | EScript.api@0x27429 ]  up 122 slots to rank #54

http://crash-stats.mozilla.com/report/index/3ea8fd61-3518-4eaf-b21c-779e52091228

Frame  	Module  	Signature [Expand]  	Source
0 		@0x0 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x81d6 	
4 	EScript.api 	EScript.api@0x816e 	
5 	EScript.api 	EScript.api@0x23475 	
6 	EScript.api 	EScript.api@0x1c439 	
7 		@0x18cdcb8f


combined these three signatures would be inside the top 5 crashes for the past 3 days
Summary: sharp rise in Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] → sharp rise in Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]
no crashes for these signatures in the first half of this month, then dramatic growth in the last few days

    EScript.api@0x27429  Multimedia.api@0x42f8b

20091214-crashdata   0    0
20091215-crashdata   0    0
20091216-crashdata  19    0
20091217-crashdata   8    0
20091218-crashdata  20    4
20091219-crashdata   4    1
20091220-crashdata  26   10
20091221-crashdata  96   21
20091222-crashdata 199   64
20091223-crashdata 229   76
20091224-crashdata 275   72
20091225-crashdata 335  128
20091226-crashdata 747  362
20091227-crashdata 764  601
these are affecting all firefox releases

signature list
 484 EScript.api@0x27429
 280 @0x0 | EScript.api@0x27429

release total-crashes
              EScript.api@0x27429 crashes
                         pct.
all     207108  764     0.0036889
3.0.15  2940    5       0.00170068
3.0.16  32075   76      0.00236945
3.5.5   8122    21      0.00258557
3.5.6   110490  494     0.00447099
3.6b5   20527   129     0.00628441
3.6b4   2074    9       0.00433944
3.6b3   628             0
3.6b2   634     3       0.00473186
3.6b1   1937    5       0.00258131

checking --- 20091227-crashdata.csv Multimedia.api@0x42f8b
release total-crashes
              Multimedia.api@0x42f8b crashes
                         pct.
all     207108  601     0.00290187
3.0.15  2940    4       0.00136054
3.0.16  32075   47      0.00146532
3.5.5   8122    10      0.00123122
3.5.6   110490  377     0.00341207
3.6b5   20527   142     0.00691772
3.6b4   2074    6       0.00289296
3.6b3   628             0
3.6b2   634     1       0.00157729
3.6b1   1937            0
Summary: sharp rise in Firefox 3.6b5 Crash [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ] → sharp rise in Firefox Crashes on [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]
interesting modules with versions for 3.6b5 2009 12 27 shows 100% correlation to the latest version -> 9.2.0.124

  EScript.api@0x27429|EXCEPTION_ACCESS_VIOLATION (104 crashes)
    100% (104/104) vs.   2% (322/17963) Multimedia.api
          0% (0/104) vs.   0% (1/17963) 9.0.0.332
          0% (0/104) vs.   0% (71/17963) 9.1.0.163
        100% (104/104) vs.   1% (250/17963) 9.2.0.124

  Multimedia.api@0x42f8b|EXCEPTION_ACCESS_VIOLATION (97 crashes)
    100% (97/97) vs.   2% (322/17963) Multimedia.api
          0% (0/97) vs.   0% (1/17963) 9.0.0.332
          0% (0/97) vs.   0% (71/17963) 9.1.0.163
        100% (97/97) vs.   1% (250/17963) 9.2.0.124
Keywords: user-doc-needed
> 0     @0xc0c0c0c     
> 1     EScript.api     EScript.api@0x27429     

I believe 0xC0 can be used as a NOP sled so this might be actual evidence of a failed heap-spray exploit beyond merely guessing based on crash volume.
Whiteboard: [crashkill][crashkill-outreach][explosive?]
.api = Adobe Reader.

I can't remember which @adobe person covers reader and it's 1:30am and I'm on vacation.
Summary: sharp rise in Firefox Crashes on [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ] → [AcrobatReader] sharp rise in Firefox Crashes on [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]
This is a recently-discovered issue with Acrobat and Reader and will be fixed with the releases on the next Patch Tuesday (Jan 12).  It is considered a vulnerability and is now believed to be being exploited in the wild (hence the spike).  See the following for more details including how to mitigate it until Jan 12:

http://www.adobe.com/support/security/advisories/apsa09-07.html
fyi, i have been unable to reproduce crashes with our crash automation using urls with these signatures from 12/24-27.
Hi Rudi,

Will the change to Acrobat and Reader be identifiable with a new version update so we can do the right version checking for our plugin updater checks over in bug  524460.  we will also try and get plugin update checks live on Jan 12 as soon as you release.
Unfortunately, no, my bad. I dropped the ball on this and didn't follow up here at Adobe.  I've now made sure that this version-string (for plugin checks) is on everyone's radar now but it won't be there for the Jan 12 release.
If anyone has hit these crashes or is reading this bug and thinks they have spotted a pdf that attempts to exploit there is a call out by the internet storm center to forward files for analysis.   http://isc.sans.org/diary.html?storyid=7903  

looks like they have spotted a few instances of pdf's containing exploit code.
(In reply to comment #12)
Hi Rudi,
Will the version string be in either the "plugin name" or "plugin description"?

These two location are used with Firefox 3.5 and will be used when we do cross-browser web based detection (except for IE which doesn't have navigator.plugins and will use another mechanism).
The version number will be in the plugin description.  When it comes out I'll update this bug with the necessary info.
Component: Plug-ins → PDF (Adobe)
Flags: wanted1.9.2?
Product: Core → Plugins
QA Contact: plugins → adobe-reader
Version: Trunk → unspecified
The plugins for 9.3.2 and 8.2.2 now have the version in plugin description; you can assume that any plug-in without a version number is out-of-date.

The URL to go to for non-current versions is:

http://www.adobe.com/go/acrobat_reader_updates
this is the most annoying bug ever, and creating trouble / disrupting work for what feels like infinite time (may be a year or so)
@yeren: what version of Adobe Acrobat/Reader do you have? I believe the fix for this has been shipping for over four months. I don't have the specific version info available right now but o do 't want to lose this thread.
I've got the latest version of both reader and acrobat. I beleive the problem actually lies in Adobe's plugin(latest version again 9.3.163), because I see crashes in Internet explorer (8) and Google Chrome (latest) as well, at least when the "display pdf in browser" optipon is checked in Reader or Acrobat. But in IE8 and Chrome it's at least only killing a tab and not the complete browser. 
I know there's the workaround to display .pdf's in a separate window, but that's quite annoying, as websites like ep.espacenet.com rely heavily on this feature.

Maybe it's imnportant to tell that the pdf (or part of it) is already displayed when everything crashes. 
I'm sure this is an Adobe problem, but they don't seem to have any kind of useful support at all. (Reminds me of Apple's talk about flash).
Might as well be an interaction with the combination of Acrobat and Reader, as I don't have this problem on my computer at home (only Reader installed, but otherwise much the same configuration (i.e. windows 7 with latest versions of everything)).
yeren: please file a new bug: https://bugzilla.mozilla.org/enter_bug.cgi?product=Plugins&component=PDF%20%28Adobe%29

don't worry, rudi will be able to see it and work with you, but this bug is really not supposed to be dedicated to your problem.
Crash Signature: [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]
Crash Signature: [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ] → [@ multimedia.api@0x42f8b ] [@ escript.api@0x27429 ] [@ @0x0 | escript.api@0x27429 ]
Keywords: topcrash, user-doc-needed
Closing old bugs in the Plugins component. We aren't going to track issues in 3rd-party plugins in the Mozilla bug tracker. In addition, support for NPAPI plugins will be removed at the end of this year; for more details see the post at https://blog.mozilla.org/futurereleases/2015/10/08/npapi-plugins-in-firefox/

If there is a serious bug in Firefox, it needs to be filed in the "Core" product, "Plug-Ins" component.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → INCOMPLETE
Product: Plugins → Plugins Graveyard
You need to log in before you can comment on or make changes to this bug.