1689149 - Move from ReadStream/StreamList/StreamControl/CacheStreamControl to RemoteLazyInputStream

Status: NEW

(Core :: Storage: Cache API, enhancement, P3)

Description

[Andrew Sutherland [:asuth] (he/him)]

The Cache API has a mechanism for ensuring that Response objects provided by match()/matchAll():

• Reference count the Response records on disk so Cache.delete does not impact previously issues Response objects.
• Are lazily opened so that matchAll() will not open all the FDs at once (bug 1397128)
• cloning the Responses maintains the above invariants and avoids bloating process memory by creating redundant pipes that could instead be backed by FDs.

Note: Response objects currently are not eligible for structured serialization, but there is a possibility for that happening in the future.

This mechanism pre-dates RemoteLazyInputStream (formerly IPCBlobInputStream until renamed in bug 1648141), but likely can be massively simplified by converting to RemoteLazyInputStream. The primary concern is ensuring that the previously issued Response objects are still viable after Cache.delete.

This came up in a discussion with Nika in the Workers & Storage channel at https://matrix.to/#/!uiRoQJJYaVaHnqwrBF:mozilla.org/$jb63Qx-fsh9Uz-NzISs8BD-tPNIzWDvqj5x8ik8ZbME?via=mozilla.org&via=t2bot.io&via=matrix.org in response to the enhancement bug 1681529 that makes it so that nsIInputStream is serialized as aDelayedStart by default. This has also been discussed in passing with Ben Kelly.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

One additional complicating factor I omitted above but is important is that the Cache streams are currently compressed using Snappy and need to be wrapped with a SnappyUncompressInputStream which currently:

• Only wraps blocking nsIInputStream instances (assuming a file stream) and only exposing nsIInputStream itself.
• Can't be serialized.