1484743 - [meta] Create IPC between nsHttpChannel and nsHttpTransaction

Status: RESOLVED FIXED

(Core :: Networking: HTTP, enhancement, P3)

Description

[Honza Bambas (:mayhemer)]

Goal: move nsHttpTransaction life and processing to the socket process, leave nsHttpChannel on the parent (main) process.

This is likely only a meta bug, since there is a number of sub-parts we have to deal with (the main thread often accesses the nsHttpTransaction object directly to read and set various properties or even work with connections, each will have to be proxied or redesigned a bit)

Generally, nsHttpTransaction does its job on the socket thread.  nsHttpChannel lives and operates on the main thread.  Hence, we could take this existing cross-thread boundary and turn it to a cross-process boundary - seeing the current parent process socket thread as the new socket process.  Instead of posting thread messages send IPC messages.

How does it work today (single proces):
- when nsHttpChannel decides it needs to talk to the server, it creates nsHttpTransaction object at [1], for subsequent authentication at [2]
- the transaction is then given to the connection manager at [3], this is a dispatch of an event to the socket thread to call nsHttpConnectionMgr::OnMsgNewTransaction with the transaction object, this method tries to dispatch it, or puts it to the queue, the usual stuff
- at [4], during initiation of our nsHttpTransaction object, we get async input stream, which will deliver the response body once the server is sending the response body
- this input stream is async-read using nsIInputStreamPump, created at [5], the pump just simplifies reading the async input stream and delivers to the channel the stream listener notifications: OnStartRequest, OnDataAvailable, OnStopRequest
- under the hood the response body delivery looks like this:
  - the transaction, at the time we initialize it at [4], creates a pipe at [6]
  - the async output stream side of the pipe will be written the data as they come from the socket
  - the async input stream side (where the data written from the socket then appears) is async read by the pump on the main thread
- when we get the OnStartRequest notification in nsHttpChannel, the transaction has already all the response headers available for examination


I can see this bug split to several sub-bugs (to be filed):
- proxy (IPC) some of the connection manager methods working with transactions
- invent an async IPC protocol to mimic the work of the response pipe and the pump reading it
  - keep in mind the need for a flow control for sending the response data (as recently made between the parent process and content processes, maybe move that code?)
  - keep in mind the need to use PHttpBackground (or something else?) to deliver the data to the requesting content process (to bypass the parent process) if not overly complicated for first stages
  - keep in mind we also have to write the data to the cache one day (which will still live on the parent process)
- serialize response headers, ProxyConnectFailed flag, security info, local+peer address (and probably more) to the parent process (via the new "OnStartRequest" or other IPC messages)
- deal with security callbacks [18], although this may be part of the PSM IPCing work
- make sure file/data upload (POST) still works and we don't copy the file content unnecessarily
- serialize transaction timing to the parent process
- make TransactionObserver::Complete work again [7]
- allow setting an Http2PushedStream on a transaction [8]
- propagate NS_HTTP_STICKY_CONNECTION flag changes from the socket process to the parent process [9][10]
- allow telling the transaction's connection to not-be-reused [11], can be async, note that connection objects will no longer exist on the parent process
- allow transfer of a connection between authentication loops [12][13]
- allow transfer of a connection (or of its socket) to update handlers (websockets, webrtc proxy connect) [14][15]
- serialize response trailer in onstoprequest [16]
- drop of caps flags, like [17]

[1] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#1278
[2] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#8231
[3] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#882
[4] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#1318
[5] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#1334
[6] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpTransaction.cpp#426
[7] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/AlternateServices.cpp#723
[8] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/Http2Push.cpp#44
[9] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpTransaction.cpp#2038
[10] https://searchfox.org/mozilla-central/search?q=mTransaction-%3ECaps()%20%26%20NS_HTTP_STICKY_CONNECTION&case=false&regexp=false&path=
[11] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#5948-5956
[12] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#7306
[13] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#8235-8236
[14] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#7332
[15] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#7423
[16] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#7351
[17] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#8504
[18] https://searchfox.org/mozilla-central/rev/847b64cc28b74b44c379f9bff4f415b97da1c6d7/netwerk/protocol/http/nsHttpChannel.cpp#8633

Comment 1

[Junior [inactive]]

- support request context
https://searchfox.org/mozilla-central/rev/2fe43133dbc774dda668d4597174d73e3969181a/netwerk/protocol/http/nsHttpChannel.cpp#1324

- NotifyActiveTabLoadOptimization
https://searchfox.org/mozilla-central/rev/55da592d85c2baf8d8818010c41d9738c97013d2/netwerk/protocol/http/nsHttpTransaction.cpp#939