startup crash in nsPrefBranch::GetIntPref (Websense Endpoint)

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: sylvestre, Assigned: sylvestre)

Tracking

(Depends on: 1 bug, Blocks: 1 bug, {crash, topcrash})

unspecified
x86
Windows 7
crash, topcrash
Dependency tree / graph
Bug Flags:
qe-verify +

Firefox Tracking Flags

(firefox48blocking fixed, firefox49 fixed, relnote-firefox 48+, firefox50 fixed, firefox51 fixed)

Details

(Whiteboard: [startupcrash], crash signature)

Attachments

(1 obsolete attachment)

(Assignee)

Description

3 years ago
+++ This bug was initially created as a clone of Bug #828184 +++

It's #5 top browser crasher in the first hours of 18.0.
It's a startup crash but with not so many duplicates.

The last time, it was caused by Websense Endpoint (https://www.websense.com/content/Home.aspx).
(Assignee)

Updated

3 years ago
status-firefox48: --- → affected
(Assignee)

Comment 1

3 years ago
Tracking as it is a startup crash and a top crash
tracking-firefox48: --- → blocking
Keywords: topcrash
(Assignee)

Updated

3 years ago
Crash Signature: [@ nsPrefBranch::GetIntPref(char const*, int*)] [@ PREF_GetIntPref ] → [@ PREF_GetIntPref ]
(Assignee)

Updated

3 years ago
status-firefox49: --- → affected
(Assignee)

Comment 2

3 years ago
I sent an email to Websense.

Benjamin, when it happened in bug #828184, we blocked them, should we do the same here?
Flags: needinfo?(benjamin)
The first few crash reports I clicked on all had QIPCAP.dll 7.6.818.1 in the module section, which appears to be the Websense DLL. Need to get the full correlation report to see the full scope.

Will also check for Mac crashes as in the past they have come up with a signature containing qipcap.dll.
(Assignee)

Updated

3 years ago
Component: General → Other
Product: Core → External Software Affecting Firefox
Version: 18 Branch → unspecified

Updated

3 years ago
Duplicate of this bug: 1291736

Comment 5

3 years ago
Need a product manager (pdol?) to make a decision here: we typically try to contact the 3rd-party software author. Determine whether to block paticular versions or all versions. It would be useful to understand what they are doing, because if it's binary XPCOM that needs to die quickly.
Flags: needinfo?(benjamin) → needinfo?(pdolanjski)

Comment 6

3 years ago
A user on SUMO reported today crash on startup with the same crash signature:
https://support.mozilla.org/en-US/questions/1133501
(Assignee)

Updated

3 years ago
See Also: → bug 1181091
(Assignee)

Comment 7

3 years ago
Sander, I tried to contact them on support@websense.com  but I received an error. Do you have access to their platform? I don't see where I can contact them.

If they only support ESR as it has been said in https://bugzilla.mozilla.org/show_bug.cgi?id=1181091#c57, we should maybe blocklist them from every firefox release
Flags: needinfo?(sander+bugzilla)

Comment 9

3 years ago
Comment on attachment 8778181 [details]
Bug 1291738 - Block the version of websense crashing 48 & 49. Followup, we already had this dll

https://reviewboard.mozilla.org/r/69542/#review66728
Attachment #8778181 - Flags: review?(benjamin) → review+
(Assignee)

Comment 10

3 years ago
Comment on attachment 8778181 [details]
Bug 1291738 - Block the version of websense crashing 48 & 49. Followup, we already had this dll

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/69542/diff/1-2/

Comment 11

3 years ago
Pushed by sledru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8a08c6285367
Block the version of websense crashing 48 & 49 r=bsmedberg
(Assignee)

Updated

3 years ago
Assignee: nobody → sledru
status-firefox50: --- → affected
status-firefox51: --- → affected
(Assignee)

Comment 12

3 years ago
Comment on attachment 8778181 [details]
Bug 1291738 - Block the version of websense crashing 48 & 49. Followup, we already had this dll

Approval Request Comment
[Feature/regressing bug #]: Outside change
[User impact if declined]: Startup crash
[Describe test coverage new/current, TreeHerder]: I don't have a windows to test, the patch is trivial
[Risks and why]: We are dealing with startup crashes, we cannot make it worst...
[String/UUID change made/needed]: None
Attachment #8778181 - Flags: approval-mozilla-release?
Attachment #8778181 - Flags: approval-mozilla-beta?
Attachment #8778181 - Flags: approval-mozilla-aurora?
Comment on attachment 8778181 [details]
Bug 1291738 - Block the version of websense crashing 48 & 49. Followup, we already had this dll

DLL blocklist for a startup crash, Release48+, Beta49+, Aurora50+
Attachment #8778181 - Flags: approval-mozilla-release?
Attachment #8778181 - Flags: approval-mozilla-release+
Attachment #8778181 - Flags: approval-mozilla-beta?
Attachment #8778181 - Flags: approval-mozilla-beta+
Attachment #8778181 - Flags: approval-mozilla-aurora?
Attachment #8778181 - Flags: approval-mozilla-aurora+

Comment 14

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/8a08c6285367
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox51: affected → fixed
Resolution: --- → FIXED

Comment 15

3 years ago
Still crashing for me.
https://crash-stats.mozilla.com/report/index/bp-095279ae-a7b5-4f22-9ead-c1aef2160808

Should I do something to benefit from the dll blocking ?

Comment 16

3 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/8a2e696c8bd1
status-firefox50: affected → fixed

Comment 17

3 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-beta/rev/0c78321d9c7d
status-firefox49: affected → fixed
(Assignee)

Comment 18

3 years ago
(In reply to jcleval from comment #15)
> Should I do something to benefit from the dll blocking ?
We, Mozilla, maintain 4 versions of Firefox in parallel (nightly, aurora/dev edition, beta and release). Here, we took the fix only for the first one. Carsten is currently applying the change to other branches. 
Once it is done, we will start a new build including the change. This week or the next.

You can check if the crash still occurs for you on the latest nightly:
http://nightly.mozilla.org/
(Assignee)

Comment 19

3 years ago
Clearing the need info to Sander, he replied to me by email.
Flags: needinfo?(sander+bugzilla)
(In reply to Benjamin Smedberg [:bsmedberg] from comment #5)
> Need a product manager (pdol?) to make a decision here: we typically try to
> contact the 3rd-party software author. Determine whether to block paticular
> versions or all versions. It would be useful to understand what they are
> doing, because if it's binary XPCOM that needs to die quickly.

Sorry for the delay (was on PTO).  Looks like the decision was made, which seems to be the right one from my perspective.
Flags: needinfo?(pdolanjski)
Flags: qe-verify+

Comment 22

3 years ago
the patch has gone into 49.0b2, but it doesn't seem to be effective as those crashes continue and reports like https://crash-stats.mozilla.com/report/index/5cd1d208-a134-426f-bb0d-486e22160810 show that the qipcap.dll module is still hooking into the process.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
It may help to ship a qipcap.dll in the same directory - a dll that does nothing but overrules the normal one. Not sure if this is (legally) possible but I tested this and seems to work :)
(Assignee)

Updated

3 years ago
status-firefox48: fixed → affected
status-firefox49: fixed → affected
status-firefox50: fixed → affected
status-firefox51: fixed → affected
Could the issue be that there are two entries in the DLL blocklist with the same file name?
(In reply to Marco Castelluccio [:marco] from comment #24)
> Could the issue be that there are two entries in the DLL blocklist with the
> same file name?

I think the loop in patched_LdrLoadDll uses the first matching entry from sWindowsDllBlocklist:

  info = &sWindowsDllBlocklist[0];
  while (info->name) {
    if (strcmp(info->name, dllName) == 0)
      break;

    info++;
  }

So in this case it will use {"qipcap.dll", MAKE_VERSION(7, 6, 815, 1)} instead of { "qipcap.dll", MAKE_VERSION(7, 6, 818, 1) }, or am I missing something?
Flags: needinfo?(benjamin)
Flags: needinfo?(aklotz)
The version field is a maximum, that is, we block anything that is less-than or equal to that version. You should have a single line that just blocks 7.6.818.1 and it will take care of older versions as well.
Flags: needinfo?(benjamin)
Flags: needinfo?(aklotz)
Comment hidden (mozreview-request)
(Assignee)

Updated

3 years ago
Depends on: 1294650
(Assignee)

Updated

3 years ago
Attachment #8778181 - Attachment is obsolete: true
(Assignee)

Comment 28

3 years ago
Closing (maybe fixed is not the best) as it has been managed in bug 1294650
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
status-firefox48: affected → fixed
status-firefox49: affected → fixed
status-firefox50: affected → fixed
status-firefox51: affected → fixed
Resolution: --- → FIXED
Added in the 48.0.1 release notes: "Fix a startup crash issue caused by Websense".
relnote-firefox: --- → 48+
I was not able to reproduce this issue on Firefox 47 or on Firefox 48, under Windows 7x86 and under Windows 8x86, using the STR from Comment 0.
Is there a specific platform, build or other STR in order to reproduce this crash?

Note that the issue was tested under Windows 7x86 on two different PC's.
(Assignee)

Updated

3 years ago
Depends on: 1297534
Depends on: 1304434

Comment 31

2 years ago
Hello all, thank you for your efforts to collaborate. I want to provide information for direct contact regarding all product security issues in all Forcepoint products and services:

I am a founding member of the Forcepoint Global Product Security Incident Response Team (PSIRT).

(Forcepoint is the synthesis of Raytheon Cyber Products, Websense, Trusted Computer Solutions, Oakley Networks, Visual Analytics, Stonesoft, Sidewinder, and others)

If you ever need to report a product security vulnerability, contact PSIRT@forcepoint.com. PGP Key: https://www.forcepoint.com/innovation/product-security/product-security-report-issue
You need to log in before you can comment on or make changes to this bug.