(In reply to Neil Deakin from comment #6)
So, to clarify, with the test you did using WS_EX_TRANSPARENT, the mouse events are fine when the popup is over a different application window?
Over a different application window (or just the desktop window), the mouse transparency takes effect as expected and events don't arrive. Over the parent Firefox window, events appear to get handled as if there were no transparency.
What about when over another Firefox window that the popup doesn't belong to?
I hadn't thought to try that before, but I just did and I don't see any change in the behavior vs. the underlying window being a different application or the desktop; the popup over the other Firefox window is transparent to the mouse.
(In reply to Neil Deakin from comment #7)
Actually, since all we care about is when the click is in the dropshadow area where the popup should close, modifying nsWindow::EventIsInsideWindow to inset the rectangle by the padding size should allow the popup to hide when clicking in the dropshadow area. I don't think that will handle passing the click to whatever is behind the popup though.
It looks like you were right about this; I tried shrinking the window rect in
EventIsInsideWindow and it does get the popup closed, but does not pass the click through to what's under it.