521752 - threadsafety problems in BitDefender Antiphishing Toolbar extension causing Firefox 3.5.* crashes

Status: RESOLVED WORKSFORME

(Firefox :: Extension Compatibility, defect, P2)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

The BitDefender Antiphishing toolbar extension appears to be responsible for a portion of the Firefox 3.5.* crashes at the following signatures:

  nsGlobalWindow::cycleCollection::UnmarkPurple(nsISupports*)
  nsJSContext::Release()
  nsTextNode::Release()

crashes at these signatures are likely due to a bug in the code in that extension:  they show up when code does reference-counting of main-thread-only objects on other threads.  I haven't actually tested the extension (it seems to require money to get), but this seems likely to be the case.

This is easy for the developers of the extension to detect if they compile a debug build of Firefox and test the extension in that debug build:  then they will see assertion dialogs whenever they violate the threadsafety invariants (with an option to drop into the debugger).  We should contact them somehow so they can fix the extension; we may also want to consider blocklisting.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #0)
> This is easy for the developers of the extension to detect if they compile a
> debug build of Firefox and test the extension in that debug build:  then they
> will see assertion dialogs whenever they violate the threadsafety invariants
> (with an option to drop into the debugger).  We should contact them somehow so
> they can fix the extension; we may also want to consider blocklisting.

Instructions for doing this are at:
https://developer.mozilla.org/En/Simple_Firefox_build
https://developer.mozilla.org/en/Windows_Build_Prerequisites
They probably want to do a build from the 1.9.1 release branch (which corresponds to Firefox 3.5); that's pulled from the hg repository at http://hg.mozilla.org/releases/mozilla-1.9.1/

The key is that they want to do a build with --enable-debug and --disable-optimize, which means adding these two lines:
ac_add_options --disable-optimize
ac_add_options --enable-debug
when creating the "mozconfig" file as instructed in the first link above.


To be clear, the toolbar that seems to be causing the problems is the one with extension ID "[e-mail]" (version 2.0, which seems to be the only version that any of our users have).  (This should be helpful both for the bitdefender folks to know which code I'm talking about, and if we end up needing to block the extension or some versions of it.)

Comment 2

[Damon Sicore (:damons)]

From Carmen at BitDefender:

Hi Damon,

I wanted to let you know that we identified the cause of the issues  
and we will deliver a fix via product update on November 3, 2009. It  
usually takes up to 2 weeks for all customers to install the updates  
(even if it does not require an action from their part). However,  
you should be able to see a decrease in number of crashes starting  
Nov 3, 2009.

I will keep you posted on the progress.

Thanks,
Carmen.

Comment 3

[Damon Sicore (:damons)]

dbaron, can you please verify that we see a drop in crashes after those dates?

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Crashes related to bitdefender actually seem to have increased slightly over the past few weeks; we had 726 reported (usual caveats about small percentage of crashes that actually get reported) on 2009-10-26, and today's report had 822 that seem likely to have been caused by bitdefender.  So it's now crept up above 1% of all our crashes.

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

In the 2009-11-17 Firefox 3.5.5. crash report, I'm seeing around 475 crashes that are caused by bitdefender.  That's still a pretty low uptake rate.

Do we know, from BitDefender, the name of a dll that's present in both broken and fixed versions, but with different file versions in each (and what file versions of that dll are expected to be crashy vs. fixed)?  We could use that to see whether the fix is actually working.

Comment 6

[Adrian Lupei]

Hi, please tell us the number of crashes caused by Bitdefender Firefox toolbar. we had released a new version of our toolbar and the number of crashes should be lowered.

Comment 7

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I'm seeing about 145 crashes in the report for our crash sample on 2009-11-25.

But all our users who have your extension still seem to be running version "2.0" of that extension.  Did you release a new version without changing the version number?  And could you answer the questions in comment 5?


It looks like most of the users who are crashing have bdGUICtl.dll version 12.0.12.30, BTCommon.dll version 12.0.12.32, BTCProxy.dll version 12.0.12.14, BDUtils.dll version 12.0.11.1, and BTCommon.ui version 12.0.12.32, though some have older versions of those libraries.  A very small number of users seem to have newer versions of BTCommon.dll (12.0.12.33) and BTCProxy.dll (12.0.12.15); either the crash is fixed for users with those versions or very few users have those versions.

Comment 8

[Adrian Lupei]

here are the versions of binaries that should have fixed the problem in Bitdefender 2009:
BTCommon.dll      12.0.12.33
BTCProxy.dll      12.0.12.15
Ffcomm2.dll       12.3.12.18
Ffcomm3.dll       12.3.12.18
with this files the problems that we are aware of should occur. this files were updated on 4th of November 2009. how can we get more info on crashes with this files? the crashes are reported in a specific area (like Us, Brazil, China, etc) or are widely reported?

The files you have specified are old and we have updated all our Bitdefender 2009 and 2010 products.

Comment 9

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

The data I'm quoting from are available at http://people.mozilla.com/crash_analysis/ .  It's much easier to look at the "addons" analysis, but to get the versions you'd need the "modules" analysis.

Comment 10

[Adrian Lupei]

thanks ... i'll folow up this info on my team.

Comment 11

[Sheila Mooney]

Dbaron, is this still a current problem? Is it a top crash? No edits on this bug for a couple of years.

Comment 12

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

In Firefox 4 we changed (see bug 549743) the situation that crashed intermittently due to threadsafety problems to crash 100% of the time (with a runtime threading check in release builds) so that extensions would not release software with this problem.  So this is probably now fixed; if it weren't we'd be seeing a lot more crashes.