1405706 - (netmonitor-sse-inspector) [meta] Inspect Server-Sent Events/SSE/EventSource Messages

Status: NEW

(DevTools :: Netmonitor, enhancement, P3)

Description

[Jan Honza Odvarko [:Honza] (always need-info? me)]

The net monitor should support so called "Server sent events"

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

"Developing a web application that uses server-sent events is quite easy. You'll need a bit of code on the server to stream the events to the web application, but the web application side of things works nearly identical to handling any other type of event."

Honza

Comment 1

[Dabe Murphy]

FWIW, I came here looking to see if there was an easy way to debug SSE / EventSource connections ("Content-type: text/event-stream") in the Network Monitor.

It would be nice to see[*] those messages as they arrive.



* — and eventually filter/search them, as well, but I don't want to seem greedy! «hehe»

Comment 2

[:Harald Kirschner :digitarald]

Demo page to try it out: https://nostalgic-fireplace.glitch.me/

Comment 3

[:Harald Kirschner :digitarald]

Attachment: Chrome's SSE tab.

Comment 4

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Michal, do you know what API we could utilize on the platform to implement this feature in DevTools?
Honza

Comment 5

[Michal Novotny [:michal]]

I have no idea. The code lives in the DOM tree.

Comment 6

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@hsinyi: Any tips about how we could support "server sent events" inspection in DevTools?
Is this feature/platform API part of DOM push notifications?
Or is there anyone else I could ask?

Thanks!
Honza

Comment 7

[Hsin-Yi Tsai (she/her) [:hsinyi]]

I was thinking about necko folks before reading comment 5. Maybe Olli can shed some light?

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

Jan, you mean EventSource I guess.
And this has nothing to do with DOM Push.

What kind of information would be useful for devtools?
https://html.spec.whatwg.org/#the-eventsource-interface
Getting access to each incoming event? I don't think that is really available currently unless devtools manage to add event listeners to the relevant object.

Comment 9

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Olli Pettay [:smaug] from comment #8)

Jan, you mean EventSource I guess.

Ah, yes, thanks for the pointer.

What kind of information would be useful for devtools?
https://html.spec.whatwg.org/#the-eventsource-interface
Getting access to each incoming event?

Yes

From user perspective:

1. The user should be able to select a URL in the Network panel
2. If there are events sent through the URL there should be a new side panel "Events"
3. The side panel shows every event and its (text) content

From API perspective:

1. The Events side panel should have access to each incoming event (to the MessageEvent object passed to listeners)

2. The Network panel should be able to match an event with specific HTTP channel. So, e.g. having channelId associated
   https://searchfox.org/mozilla-central/rev/d143f8ce30d1bcfee7a1227c27bf876a85f8cede/netwerk/protocol/http/nsIHttpChannel.idl#489

3. The API could look similar to what we have for WebSockets
   https://searchfox.org/mozilla-central/rev/d143f8ce30d1bcfee7a1227c27bf876a85f8cede/netwerk/protocol/websocket/nsIWebSocketEventService.idl#72

• The service allows registering listeners and get notification about every connection created and every frame sent/received.
• Mapping socket connection to HTTP channel is requested in bug 1542170

Olli, does that make sense?

I don't think that is really available currently unless devtools manage to add event listeners to the relevant object.

I am not sure how this would work. We'd at least need a notification about new EventSource object being initialized/destroyed to add a listener to it.

Honza

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

Sorry about delay. It does look reasonable.

So would it be enough to have some notification when EventSource object is created. Then devtools could add event listeners to the object to catch incoming messages.

Comment 11

[James B]

Since Chrome already has this feature, is there a "feature parity" tag that could be added to this issue, that might help visibility?

As it stands, when code uses the EventSource constructor to initiate a connection, an entry is created for the passed URL in the Network tab, but this entry is basically broken. Its status and timing columns in the request list are empty, the Headers, Cookies, and Response tabs in the detail pane all say "No {X} available for this request", and the Timings tab is blank. There is no indication why this is happening, it just looks like the request didn't actually go through.

If implementing the changes needed to fully support live display of incoming events is too much work to implement quickly (and it sounds like it would be), would the team be open to a quick-fix that at least decorates the request as being used for Server-Sent Events (EventSource) and explains that, at this time, no further details are available? It would be helpful to distinguish a successful request that the dev tools don't know how to display, from a request that was e.g. blocked by security policy.

Comment 12

[:Harald Kirschner :digitarald]

This would hook into the existing UI created for WebSocket's, bug 885508 and m1 in bug 1565617. We did create a milestone for this during planning but now need to prioritize against the other work.

Comment 13

[James B]

Harald, in my reading of c9 it sounds like maybe there are missing hook points in the network code, that would need to be available before the existing UI could be wired up. Am I misunderstanding that? If not, would you consider my suggestion from the last paragraph of c11? (Namely, decorating the currently-broken-looking request with a short explanation that yes, this SSE connection is fine, and no, we can't show you any details about it yet.)

Comment 14

[:Harald Kirschner :digitarald]

If not, would you consider my suggestion from the last paragraph of c11? (Namely, decorating the currently-broken-looking request with a short explanation that yes, this SSE connection is fine, and no, we can't show you any details about it yet.)

That seems a short-term fix. On the flipside, there seems nothing to do for a user to make the data appear, so the message might be more like a placeholder for the future feature while not helping. We can continue in bug 1597281.

Comment 15

[:Harald Kirschner :digitarald]

Tie-in with WS Inspector M1 work to re-use components.

Comment 16

[Olli Pettay [:smaug][[e-mail]]]

Jan, looking at comment 9 again, could you still explain the requirements a bit.

First, in which thread should this all work? EventSource does get the data and parses it in worker thread, if one has created EventSource in a worker. nsIChannel wouldn't be available there for JS.
(How does WebSocket stuff work in workers?)

At which point does devtools want to get know about incoming message? When it has been received, or when the relevant event is going to be dispatched to JS?

Does devtools need to be able to stop/break dispatching events to JS?

Comment 17

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Olli Pettay [:smaug] from comment #16)

First, in which thread should this all work? EventSource does get the data and parses it in worker thread, if one has created EventSource in a worker. nsIChannel wouldn't be available there for JS.
(How does WebSocket stuff work in workers?)

Handling WebSocket notifications is done in content process.
https://searchfox.org/mozilla-central/rev/c7b673f443407a359cc0766fb5a4ac323a1d2628/devtools/server/actors/network-monitor/websocket-actor.js#54

We are consequently using DevTools infrastructure to send collected info to the front-end (client) using emit method

E.g. received frames here:
https://searchfox.org/mozilla-central/rev/c7b673f443407a359cc0766fb5a4ac323a1d2628/devtools/server/actors/network-monitor/websocket-actor.js#111

Handling WS events in the parent process would work too (using pretty much the same API).

At which point does devtools want to get know about incoming message? When it has been received, or when the relevant event is going to be dispatched to JS?

I think that DevTools should handle the message before it's going to the dispatched.

Does devtools need to be able to stop/break dispatching events to JS?

@Harald: what do you think, do we need something like that?

Some notes related to the API.

We are using the following API for WebSockets:

• nsIWebSocketEventService
• nsIWebSocketEventListener

I could imagine similar API for EventSource

interface nsIEventSourceService : nsISupports {
  void addListener(aInnerWindowID, aListener);
  void removeListener(aInnerWindowID, aListener);
  bool hasListenerFor(aInnerWindowID);
};

interface nsIWebSocketEventListener : nsISupports {
  void eventSourceConnectionOpened(aConnectionId, aEffectiveURI, aHttpChannelId);
  void eventSourceConnectionClosed(aConnectionId, aReason);
  void eventReceived(aConnectionID, aEvent);
};

Requirements

• DevTools should be able to add necessary listener at the page load begin on the top level document and listen for EvenSource events created anywere on the page (including iframes, just like WebSockets).
• DevTools should know that a connection has been opened.
• DevTools should know what channelId has been used, so we can tie the info to specific request displayed in the Network panel
• DevTools should know that a message has been received so, we can render it to the user.
• DevTools should know that a connection has been closed so, we can indicate that in the UI.

Honza

Comment 18

[Olli Pettay [:smaug][[e-mail]]]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #17)

(In reply to Olli Pettay [:smaug] from comment #16)

First, in which thread should this all work? EventSource does get the data and parses it in worker thread, if one has created EventSource in a worker. nsIChannel wouldn't be available there for JS.
(How does WebSocket stuff work in workers?)

Handling WebSocket notifications is done in content process.
https://searchfox.org/mozilla-central/rev/c7b673f443407a359cc0766fb5a4ac323a1d2628/devtools/server/actors/network-monitor/websocket-actor.js#54

I wasn't asking about process, but thread.

I think that DevTools should handle the message before it's going to the dispatched.
So, right before dispatching, or when the message has been received? Those are a bit different times.

We are using the following API for WebSockets:

• nsIWebSocketEventService
• nsIWebSocketEventListener
  I know, but EventSource is in many way very different.

Does WebSocket devtools API support WebSockets in workers?

I could imagine similar API for EventSource

interface nsIEventSourceService : nsISupports {
  void addListener(aInnerWindowID, aListener);
  void removeListener(aInnerWindowID, aListener);
  bool hasListenerFor(aInnerWindowID);
};

Ok, this smells like main thread only API.

Requirements

• DevTools should be able to add necessary listener at the page load begin on the top level document and listen for EvenSource events created anywere on the page (including iframes, just like WebSockets).
  So getting know when an EventSource object has been created would be enough for this, since after that devtools could just add event listener to the relevant object.

• DevTools should know that a connection has been opened.
  "open" event. Is that not enough?

• DevTools should know what channelId has been used, so we can tie the info to specific request displayed in the Network panel
  Ok, so perhaps EventSource interface could have [ChromeOnly] readonly attribute ChannelIdType channelId; or some such

• DevTools should know that a message has been received so, we can render it to the user.
  event listeners.

• DevTools should know that a connection has been closed so, we can indicate that in the UI.
  this might need something new

Just trying to see how well the existing APIs could work here.
Basically, for main thread case, would it be enough to:
(1) Add a notification when a new EventSource object is created. The EventSource object would be passed as param to the notification callback
(2) Add some [ChromeOnly] attribute to give channelId
(3) Add a way to know when close() has been called.

But again, how workers should be supported affects to this all a lot.

Comment 19

[:Harald Kirschner :digitarald]

@Harald: what do you think, do we need something like that?

We are looking for parity with WS logging, so not atm.

Comment 20

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Olli Pettay [:smaug] from comment #18)

I wasn't asking about process, but thread.

I am unsure about the internals, perhaps :baku knows more?

Does WebSocket devtools API support WebSockets in workers?

Yes

Just trying to see how well the existing APIs could work here.
Basically, for main thread case, would it be enough to:
(1) Add a notification when a new EventSource object is created. The EventSource object would be passed as param to the notification callback
(2) Add some [ChromeOnly] attribute to give channelId
(3) Add a way to know when close() has been called.

Sounds good to me.

But again, how workers should be supported affects to this all a lot.

My perspective is that it should work similarly to WS. The Network panel is properly showing WS frames coming from WS connection running within a worker.

Honza

Comment 21

[Andrea Marchesini [:baku]]

Does WebSocket devtools API support WebSockets in workers?

Yes, nsIWebSocketEventService is a thread-safe service. WebSockets keep a reference to the service and they use it everywhere. Internally, there are runnables dispatched to the main-thread. To avoid extra work when devtools network panel not active, those runnables are not dispatched if there are no WebSocketEventListeners registered.

Comment 22

[Olli Pettay [:smaug][[e-mail]]]

The current plan for this is to have similar-ish API as what we have for WebSockets, but the implementation would be quite different.
EventSource objects would dispatch runnable to the main thread to pass the relevant data to the nsIEventSourceObserverService (or whatever we'll call it). (that all should happen only if the service has any observers)

Comment 23

[Farooq AR]

Hi! I've looked into the platform support for websocket to understand few things. I wasn't able to fully grasp it but I kinda have an idea of the places in dom/base/EventSource.cpp where we'll notify the EventService which in turn will notify the listeners through runnables. I'd definitely love to work on the platform support to learn more about that stuff but I'll need some help regarding this. I gathered some info on this for my own convenience: https://gist.github.com/FarooqAR/7684a0922c49f10b28d6f86372c6f135

Comment 24

[Olli Pettay [:smaug][[e-mail]]]

As I said, the implementation would be quite different to WebSocket service, but API could be similar.
Basically in all the places where EventSource dispatches events to itself one would check if the EventSourceObserverService is active, and if so, create a Runnable to dispatch relevant data to the main thread so that EventSourceObserverService can use it.

Comment 26

[Surya Balakrishnan [:surya_b]]

I would like to be assigned this bug. Can you tell me how I should proceed?

Comment 27

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Surya Balakrishnan from comment #26)

I would like to be assigned this bug. Can you tell me how I should proceed?

Thanks for the interest Surya!

This is a meta bug for all the work and we don't assign metas.

Also, the work will be lead by Farooq who submitted the best proposal for the GSoC project.
We are going to file several dependencies for this and let's see/wait if there is something for you.

Thanks,
Honza