536974 - [AcrobatReader] sharp rise in Firefox Crashes on [@ Multimedia.api@0x42f8b ] [@ EScript.api@0x27429 ] [@ @0x0 | EScript.api@0x27429 ]

Status: RESOLVED INCOMPLETE

(Plugins Graveyard :: PDF (Adobe), defect)

Description

[chris hofmann]

there is a jump in acrobat reader crashes in the past 3 days on 3.6b5 and maybe other releases. up 41 slots, to current rank as the #15 top crash.

stacks look like 
http://crash-stats.mozilla.com/report/index/050b43d9-2ac8-452f-9ff9-b89c82091228

Frame  	Module  	Signature [Expand]  	Source
0 	Multimedia.api 	Multimedia.api@0x42f8b 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x262d7 	
4 	EScript.api 	EScript.api@0x55f20 	
5 	AcroRd32.dll 	AcroRd32.dll@0x130814 	
6 	user32.dll 	InternalCallWinProc 	
7 	user32.dll 	UserCallWinProcCheckWow 	
8 	user32.dll 	DispatchMessageWorker 	
9 	user32.dll 	DispatchMessageW 	
10 	AcroRd32.dll 	AcroRd32.dll@0xf2c93 	
11 	AcroRd32.dll 	AcroRd32.dll@0xf2698 	
12 	AcroRd32.dll 	AcroRd32.dll@0x38ce 	
13 	AcroRd32.dll 	AcroRd32.dll@0x1922d2

most look like AcroRd32.dll  	9.2.0.124 is involved

more reports at http://crash-stats.mozilla.com/report/list?range_value=2&range_unit=weeks&signature=Multimedia.api%400x42f8b&version=Firefox%3A3.6b5

hopefully this increase in crashes is not related to attempts at exploiting the current unpatched zero-day

https://bugzilla.mozilla.org/show_bug.cgi?id=524460#c3

we should watch for continued crash growth, and consider blocking 9.2.0.124 as soon as the new update is released.  plugin update page should also help get users updated of that version as soon as possible.

Comment 1

[chris hofmann]

also related is a similar stack which as jumped #171 slots to rank #22


http://crash-stats.mozilla.com/report/index/5c4d88d9-5758-4bcb-b22d-7e49f2091225

Frame  	Module  	Signature [Expand]  	Source
0 		@0xc0c0c0c 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x262d7 	
4 	EScript.api 	EScript.api@0x55f20 	
5 	AcroRd32.dll 	AcroRd32.dll@0x130814 	
6 	user32.dll 	InternalCallWinProc 	
7 	user32.dll 	UserCallWinProcCheckWow 	
8 	user32.dll 	DispatchMessageWorker 	
9 	user32.dll 	DispatchMessageW 	
10 	AcroRd32.dll 	AcroRd32.dll@0xf2c93 	
11 	AcroRd32.dll 	AcroRd32.dll@0xf2698 	
12 	AcroRd32.dll 	AcroRd32.dll@0x38ce 	
13 	AcroRd32.dll 	AcroRd32.dll@0x1922d2

Comment 2

[chris hofmann]

also  [@ @0x0 | EScript.api@0x27429 ]  up 122 slots to rank #54

http://crash-stats.mozilla.com/report/index/3ea8fd61-3518-4eaf-b21c-779e52091228

Frame  	Module  	Signature [Expand]  	Source
0 		@0x0 	
1 	EScript.api 	EScript.api@0x27429 	
2 	EScript.api 	EScript.api@0x26881 	
3 	EScript.api 	EScript.api@0x81d6 	
4 	EScript.api 	EScript.api@0x816e 	
5 	EScript.api 	EScript.api@0x23475 	
6 	EScript.api 	EScript.api@0x1c439 	
7 		@0x18cdcb8f


combined these three signatures would be inside the top 5 crashes for the past 3 days

Comment 3

[chris hofmann]

no crashes for these signatures in the first half of this month, then dramatic growth in the last few days

    EScript.api@0x27429  Multimedia.api@0x42f8b

20091214-crashdata   0    0
20091215-crashdata   0    0
20091216-crashdata  19    0
20091217-crashdata   8    0
20091218-crashdata  20    4
20091219-crashdata   4    1
20091220-crashdata  26   10
20091221-crashdata  96   21
20091222-crashdata 199   64
20091223-crashdata 229   76
20091224-crashdata 275   72
20091225-crashdata 335  128
20091226-crashdata 747  362
20091227-crashdata 764  601

Comment 4

[chris hofmann]

these are affecting all firefox releases

signature list
 484 EScript.api@0x27429
 280 @0x0 | EScript.api@0x27429

release total-crashes
              EScript.api@0x27429 crashes
                         pct.
all     207108  764     0.0036889
3.0.15  2940    5       0.00170068
3.0.16  32075   76      0.00236945
3.5.5   8122    21      0.00258557
3.5.6   110490  494     0.00447099
3.6b5   20527   129     0.00628441
3.6b4   2074    9       0.00433944
3.6b3   628             0
3.6b2   634     3       0.00473186
3.6b1   1937    5       0.00258131

checking --- 20091227-crashdata.csv Multimedia.api@0x42f8b
release total-crashes
              Multimedia.api@0x42f8b crashes
                         pct.
all     207108  601     0.00290187
3.0.15  2940    4       0.00136054
3.0.16  32075   47      0.00146532
3.5.5   8122    10      0.00123122
3.5.6   110490  377     0.00341207
3.6b5   20527   142     0.00691772
3.6b4   2074    6       0.00289296
3.6b3   628             0
3.6b2   634     1       0.00157729
3.6b1   1937            0

Comment 5

[chris hofmann]

seems to be hitting a lot of users on myspace across a wide variety of pages



domains of sites Multimedia.api@0x42f8b
  82 http://viewmorepics.myspace.com
  46 http://messaging.myspace.com
  32 \N//
  31 http://www.tagged.com
  13 http://profile.myspace.com
  13 http://comment.myspace.com
   9 http://bulletins.myspace.com
   8 http://www.mediatakeout.com
   7 http://home.myspace.com

also several comments about hitting it on torrent finder

  1 http://isohunt.com/torrents/alvin+and+the+chipmunks?ihs1=13&iho1=&iht=-1&ihp=1
   1 http://isohunt.com/torrents/?ihq=video+phone
   1 http://isohunt.com/torrent_details/91394521/Boz+Scaggs?tab=summary
   1 http://isohunt.com/torrent_details/142304293/the+hangover+unrated?tab=comments


domains of sites EScript.api@0x27429
  67 http://viewmorepics.myspace.com
  53 \N//
  45 http://messaging.myspace.com
  27 http://www.tagged.com
  21 http://profile.myspace.com
  11 http://bitmusic.eu
   9 http://comment.myspace.com
   8 http://www.myspace.com
   8 http://isohunt.com
   7 http://home.myspace.com
   6 http://www.facebook.com
   6 http://www.animecrazy.net
   6 http://torrent-finder.com

Comment 6

[chris hofmann]

interesting modules with versions for 3.6b5 2009 12 27 shows 100% correlation to the latest version -> 9.2.0.124

  EScript.api@0x27429|EXCEPTION_ACCESS_VIOLATION (104 crashes)
    100% (104/104) vs.   2% (322/17963) Multimedia.api
          0% (0/104) vs.   0% (1/17963) 9.0.0.332
          0% (0/104) vs.   0% (71/17963) 9.1.0.163
        100% (104/104) vs.   1% (250/17963) 9.2.0.124

  Multimedia.api@0x42f8b|EXCEPTION_ACCESS_VIOLATION (97 crashes)
    100% (97/97) vs.   2% (322/17963) Multimedia.api
          0% (0/97) vs.   0% (1/17963) 9.0.0.332
          0% (0/97) vs.   0% (71/17963) 9.1.0.163
        100% (97/97) vs.   1% (250/17963) 9.2.0.124

Comment 7

[Daniel Veditz [:dveditz]]

> 0     @0xc0c0c0c     
> 1     EScript.api     EScript.api@0x27429     

I believe 0xC0 can be used as a NOP sled so this might be actual evidence of a failed heap-spray exploit beyond merely guessing based on crash volume.

Comment 8

[timeless]

.api = Adobe Reader.

I can't remember which @adobe person covers reader and it's 1:30am and I'm on vacation.

Comment 9

[Rudi Sherry]

This is a recently-discovered issue with Acrobat and Reader and will be fixed with the releases on the next Patch Tuesday (Jan 12).  It is considered a vulnerability and is now believed to be being exploited in the wild (hence the spike).  See the following for more details including how to mitigate it until Jan 12:

http://www.adobe.com/support/security/advisories/apsa09-07.html

Comment 10

[Bob Clary [:bc] (inactive)]

fyi, i have been unable to reproduce crashes with our crash automation using urls with these signatures from 12/24-27.

Comment 11

[chris hofmann]

Hi Rudi,

Will the change to Acrobat and Reader be identifiable with a new version update so we can do the right version checking for our plugin updater checks over in bug  524460.  we will also try and get plugin update checks live on Jan 12 as soon as you release.

Comment 12

[Rudi Sherry]

Unfortunately, no, my bad. I dropped the ball on this and didn't follow up here at Adobe.  I've now made sure that this version-string (for plugin checks) is on everyone's radar now but it won't be there for the Jan 12 release.

Comment 13

[chris hofmann]

If anyone has hit these crashes or is reading this bug and thinks they have spotted a pdf that attempts to exploit there is a call out by the internet storm center to forward files for analysis.   http://isc.sans.org/diary.html?storyid=7903  

looks like they have spotted a few instances of pdf's containing exploit code.

Comment 14

[Austin King [:ozten]]

(In reply to comment #12)
Hi Rudi,
Will the version string be in either the "plugin name" or "plugin description"?

These two location are used with Firefox 3.5 and will be used when we do cross-browser web based detection (except for IE which doesn't have navigator.plugins and will use another mechanism).

Comment 15

[Rudi Sherry]

The version number will be in the plugin description.  When it comes out I'll update this bug with the necessary info.

Comment 16

[Rudi Sherry]

The plugins for 9.3.2 and 8.2.2 now have the version in plugin description; you can assume that any plug-in without a version number is out-of-date.

The URL to go to for non-current versions is:

http://www.adobe.com/go/acrobat_reader_updates

Comment 17

[yeren]

this is the most annoying bug ever, and creating trouble / disrupting work for what feels like infinite time (may be a year or so)

Comment 18

[Rudi Sherry]

@yeren: what version of Adobe Acrobat/Reader do you have? I believe the fix for this has been shipping for over four months. I don't have the specific version info available right now but o do 't want to lose this thread.

Comment 19

[yeren]

I've got the latest version of both reader and acrobat. I beleive the problem actually lies in Adobe's plugin(latest version again 9.3.163), because I see crashes in Internet explorer (8) and Google Chrome (latest) as well, at least when the "display pdf in browser" optipon is checked in Reader or Acrobat. But in IE8 and Chrome it's at least only killing a tab and not the complete browser. 
I know there's the workaround to display .pdf's in a separate window, but that's quite annoying, as websites like ep.espacenet.com rely heavily on this feature.

Maybe it's imnportant to tell that the pdf (or part of it) is already displayed when everything crashes. 
I'm sure this is an Adobe problem, but they don't seem to have any kind of useful support at all. (Reminds me of Apple's talk about flash).
Might as well be an interaction with the combination of Acrobat and Reader, as I don't have this problem on my computer at home (only Reader installed, but otherwise much the same configuration (i.e. windows 7 with latest versions of everything)).

Comment 20

[timeless]

yeren: please file a new bug: https://bugzilla.mozilla.org/enter_bug.cgi?product=Plugins&component=PDF%20%28Adobe%29

don't worry, rudi will be able to see it and work with you, but this bug is really not supposed to be dedicated to your problem.

Comment 21

[Benjamin Smedberg]

Closing old bugs in the Plugins component. We aren't going to track issues in 3rd-party plugins in the Mozilla bug tracker. In addition, support for NPAPI plugins will be removed at the end of this year; for more details see the post at https://blog.mozilla.org/futurereleases/2015/10/08/npapi-plugins-in-firefox/

If there is a serious bug in Firefox, it needs to be filed in the "Core" product, "Plug-Ins" component.