1384474 - e10s message pump gets caught in infinite loop causing permanent UI deadlock when packets are captured with Windows Filtering Platform

Status: UNCONFIRMED

(NSS :: Libraries, defect, P3)

Description

[TechnikEmpire]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36

Steps to reproduce:

I ran an internet content filtering software that captures packets via the Windows Filtering Platform, the same way most security software captures packets.


Actual results:

The UI deadlocks because the IPC message pump enters an infinite loop it can never exit from. I had the stack trace in a temp file in Notepad++ and it appears to have been lost, I'll have to attach the debugger again and report back here.


Expected results:

The IPC message pump should be able to handle a potential fault in reading. What appears to happen instead is that it just enters an infinite loop waiting for a message complete signal. There should at the very least be a timeout. Every single other browser I have tested this software with (which is basically every major browser plus some) has no such issue. I am simply analyzing packets like any firewall or IPS/IDS would, I'm using well defined, public operating system API's to do so. Even if my program is causing packets to get lost, FF needs to have the most basic of fault tolerance in the IPC message pump system.

Comment 1

[TechnikEmpire]

Okay so I was off base quite a bit on this one because I got hung up on seeing the callstack looping in the win specific IPC message pump, being stuck forever waiting for the message to complete.

However, the real cause here is actually CA certificates installed with the same subject name . Steps to repro:

1) Enable enterprise roots in your profile.
2) Generate a bunch of CA certs with unrestricted signing constraints set (all signing rights) and with the same subject name and install them in Windows trusted CA store.
3) Run a proxy that uses one of the aforementioned root CA's. Leave all the unused cloned certs installed in OS.
4) Browse for a bit. You'll deadlock and when you try to debug and break all it'll signal an error in NSS with regards to time (can't remember source file name, probably code that checks certificate issue date and validity w/ regards to time constraints but that's pure speculation).

I didn't debug beyond this. I had forgotten to enable a ancient block of code I had written that cleans up old installed root CA's that my transparent filtering proxy generates on startup. When I enabled this code and it removed all orphaned CA's with the same subject name, the issue is resolved. This issue also happened when explicitly disabling e10s, so maybe not related to IPC at all. I'm not sure, I have my head buried deep in my own code. Wish I could offer more help but hopefully what I have offered here will help.