1373639 - Support intercepting WebSocket connections from WebExtensions

Status: RESOLVED FIXED

(WebExtensions :: General, enhancement, P3)

Description

[Dan Callahan [:callahad]]

In discussions following Firefox 54's release, I've seen several requests for websocket inspection / interception. It does not appear that any current APIs allow this.

I'm not sure exactly what shape the API should take, but a design could probably be extracted from existing WebSocket tools like :honza's WebSocket Inspector: https://github.com/firebug/websocket-monitor

Comment 1

[Shane Caraveo (:mixedpuppy)]

Websockets can be inspected/intercepted through WebRequest, though prior to Bug 1367478 you had to watch for https? protocol which is how websocket appears internally.  I am not sure if there something that *cannot* be done via WebRequest, it's possible only some events work with websockets.  Someone would need to investigate further, or provide specific examples (e.g. test cases) of what is needed in WebRequest.

Comment 2

[Dan Callahan [:callahad]]

Oh, hey, that's what I get for just glancing at the docs, seeing "HTTP" and a bunch of related methods, and not actually trying it ;) I *did* search Bugzilla, but Bug 1367478 didn't have "websocket" in the summary, so I missed it.

Comment 3

[Dan Callahan [:callahad]]

Flagging as dev-doc-needed; we should make sure the docs on MDN discuss using webRequest to intercept ws:// and wss://

Comment 4

[Luca Greco [:rpl] [:luca] [:lgreco]]

I agree with Shane that we should check what is possible or not possible in this area with webRequest API,
but I also have the feeling that this could/would have a different role, e.g.:

- it would be an API only available in a devtools contexts (e.g. a devtools panel)
  and it will only target a specific tab (the target of the developer toolbox)
- it would provide an API oriented to listening to the "websocket frames", instead of being "connection" oriented as the webRequest API
- it should provide a way to integrate with the toolbox network panel (e.g. monitor the websocket connection related to a request listed in the network panel)

Another thing that worth a mention is that when :honza has started to work on the websocket-monitor addon linked in comment 0, we have added a new API to make it possible to monitor the websocket connections at that detail level and with reasonable performance (Bug 1203802 - Websocket Frame Listener API for devtool Network Inspector).

Comment 5

[Espen H]

Seeing as the WebSocket Monitor extension is based on the legacy extension API, I decided to look into whether it was possible with today's APIs to rewrite it to the WebExtension API. I found this thread a bit confusing, so I have a question:

Chrome says the following on their documentation for webRequest:

> Starting from Chrome 58, the webRequest API supports intercepting the WebSocket handshake request. Since the handshake is done by means of an HTTP upgrade request, its flow fits into HTTP-oriented webRequest model. Note that the API does not intercept:
> 
>    Individual messages sent over an established WebSocket connection.
>    WebSocket closing connection.
> 
> Redirects are not supported for WebSocket requests. 

I assume this means the same is the case for Firefox. This means that there is currently no way in a WebExtension to read WebSocket messages? If so, that is a blocker for avoiding WebSocket Monitor stops working. 

(In reply to Luca Greco [:rpl] from comment #4)

> Another thing that worth a mention is that when :honza has started to work
> on the websocket-monitor addon linked in comment 0, we have added a new API
> to make it possible to monitor the websocket connections at that detail
> level and with reasonable performance (Bug 1203802 - Websocket Frame
> Listener API for devtool Network Inspector).

This is the API that WebSocket Monitor currently uses, but is not a WebExtension API, and from my understanding will not be allowed from FF57.

Comment 6

[Luca Greco [:rpl] [:luca] [:lgreco]]

(In reply to Espen H from comment #5)
> Chrome says the following on their documentation for webRequest:
> 
> > Starting from Chrome 58, the webRequest API supports intercepting the WebSocket handshake request. Since the handshake is done by means of an HTTP upgrade request, its flow fits into HTTP-oriented webRequest model. Note that the API does not intercept:
> > 
> >    Individual messages sent over an established WebSocket connection.
> >    WebSocket closing connection.
> > 
> > Redirects are not supported for WebSocket requests. 
> 
> I assume this means the same is the case for Firefox. This means that there
> is currently no way in a WebExtension to read WebSocket messages?

Yes, this should also be the case for Firefox, the webRequest API can intercept a WebSocket connection, but cannot intercept the individual messages sent over the established connection.
 
> If so, that is a blocker for avoiding WebSocket Monitor stops working. 
> 
> (In reply to Luca Greco [:rpl] from comment #4)
> 
> > Another thing that worth a mention is that when :honza has started to work
> > on the websocket-monitor addon linked in comment 0, we have added a new API
> > to make it possible to monitor the websocket connections at that detail
> > level and with reasonable performance (Bug 1203802 - Websocket Frame
> > Listener API for devtool Network Inspector).
> 
> This is the API that WebSocket Monitor currently uses, but is not a
> WebExtension API, and from my understanding will not be allowed from FF57.

Jan has recently created an WebExtension Experiment API addon (https://webextensions-experiments.readthedocs.io/en/latest/) and a version of the websocket-monitor addon which uses this experimental API:

- https://github.com/janodvarko/webext-websocket-monitor

The idea is to provide a proper WebExtension API based on the above experiment.

Comment 7

[Firefox Bug Husbandry Bot]

Bulk move of bugs per https://bugzilla.mozilla.org/show_bug.cgi?id=1483958

Comment 8

[Richard Bloor]

(In reply to Dan Callahan [:callahad] from comment #3)

Flagging as dev-doc-needed; we should make sure the docs on MDN discuss
using webRequest to intercept ws:// and wss://

Can I clarify what you anticipate by a "discussion", would the following change to the introductory paragraph be what you had in mind?

"Add event listeners for the various stages of an HTTP request, which includes websocket requests on ws:// and wss://. The event listener receives detailed information about the request and can modify or cancel the request."

or were you envisaging more extensive information, if so could you give me an indication of what that might be? Thank you!

Comment 9

[Dan Callahan [:callahad]]

Sorry for the delay... I'm honestly not sure what would be reasonable here.

The Chrome information quoted above is exactly what I would want to know (e.g., it is possible to intercept initial websocket requests, but with these specific limitations). I'd also expect searching for "websocket" to turn up something on the page.

So if the Chrome information is still valid and equally applicable to our implementation, then I'd just go with that.

Comment 10

[Richard Bloor]

So, any suggestions as to who can answer the question "if the Chrome information is still valid and equally applicable to our implementation"?

Comment 11

[Shane Caraveo (:mixedpuppy)]

Websockets to my knowledge is run through our httpchannel implementation. In that case, anything one could inspect with an http request should be inspectable. There is a basic test here, someone could generate more tests to better check for chrome compatibility if there are known working/non-working listeners.

https://searchfox.org/mozilla-central/source/toolkit/components/extensions/test/xpcshell/test_ext_webRequest_webSocket.js

Comment 12

[Richard Bloor]

Following advice from Chris Mills, updated as per https://bugzilla.mozilla.org/show_bug.cgi?id=1373639#c8