1545423 - WebExtension API for the amount of data sent/received

Status: RESOLVED FIXED

(Core :: Networking: HTTP, task, P1)

Comment 1

[Nhi Nguyen (:nhi)]

we're not doing this at this time

Comment 2

[Andrea Marchesini [:baku]]

I think this is important to have, in order to know how much data we send through the proxy.
Dragana, Wennie and I had a meeting last week to discuss the importance of having such information.
I'm reopening this bug.

Comment 3

[Junior [inactive]]

1. nsHttpTransaction::mContentLength accumulates bytes of received headers and content. (Fix: content only)
   However, nsHttpTransaction::mRequestSize is bytes sent headers and upload before header compression

Looks like nsHttpTransaction::OnTransportStatus form nsSocketTransport2/Http2Steam is more reliable for the underlying bytes sent/received.
https://searchfox.org/mozilla-central/rev/696cf3b52a9aa04aa5013008a8b448904a298356/netwerk/protocol/http/nsHttpTransaction.cpp#573

2. localhost might be excluded in extension, since we're going to exposed (privileged only?) in details.byteWritten/byteRead of browser.webRequest.onCompleted.

Comment 4

[Junior [inactive]]

(In reply to Junior [:junior] from comment #3)

1. nsHttpTransaction::mContentLength accumulates bytes of received headers and content. (Fix: content only)
   However, nsHttpTransaction::mRequestSize is bytes sent headers and upload before header compression

Looks like nsHttpTransaction::OnTransportStatus form nsSocketTransport2/Http2Steam is more reliable for the underlying bytes sent/received.
https://searchfox.org/mozilla-central/rev/696cf3b52a9aa04aa5013008a8b448904a298356/netwerk/protocol/http/nsHttpTransaction.cpp#573

2. localhost might be excluded in extension, since we're going to exposed (privileged only?) in details.byteWritten/byteRead of browser.webRequest.onCompleted.

We have many similar attributes:
nsIHttpChannel::transferSize: size consumed by the response header fields and the response payload body
which is from nsHttpTransaction::mTransferSize.
The name is transfer but it's about the received response. (In other words, the size which resource transfers to the client)
Note that the header is after decompressed in H2, so the number will be larger than the real bytes through the wire.

transferSize is used for ResourceTiming. (also some rcwn telemetry)
https://www.w3.org/TR/resource-timing-2/#dom-performanceresourcetiming-transfersize

It doesn't explicitly say it should or should not take HPACK into account.
It does mention the H2 overhead, but header compression isn't overhead IMO.
So I'm not sure if I should overwrite mTransferSize by use the size of compressed header.

We also have nsIHttpChannel::decodedBodySize which is about the content compression. Out of topic.
nsHttpTransaction::mContentLength is about the response body only, indicated by response header.

To conclude, instead of bike-shedding too much at the moment, I'd like to use nsHttpTransaction::mTransferSize and nsHttpTransaction::mRequestSize for the first version.

Comment 5

[Junior [inactive]]

Attachment: Bug 1545423 - P1 Expose requestSize to nsIHttpChannel

Comment 6

[Junior [inactive]]

Attachment: Bug 1545423 - P2 web request data API implementation

Depends on D50404

Comment 7

[Junior [inactive]]

Attachment: Bug 1545423 - P3 test for new attributes in browser.webRequest.onCompleted

Depends on D50405

Comment 8

[Junior [inactive]]

We might need to disable android since extension process. Let's see.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed0208c01a8270cd4cec0bcd7ecfa2a44f977049

Comment 9

[Shane Caraveo (:mixedpuppy)]

Philip, seems like this api addition is reasonable to make public. See D50405.

Comment 10

[Junior [inactive]]

Attachment: Bug 1545423 - P4 remove privileged restriction

Comment 11

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

This sounds like a good addition, let's add it to the public API.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f7f0e4c65da
P1 Expose requestSize to nsIHttpChannel r=dragana
https://hg.mozilla.org/integration/autoland/rev/3ae45f3be2e1
P2 web request data API implementation r=mixedpuppy
https://hg.mozilla.org/integration/autoland/rev/35758baf44e3
P3 test for new attributes in browser.webRequest.onCompleted r=baku,mixedpuppy
https://hg.mozilla.org/integration/autoland/rev/0abbf10b68b0
P4 remove privileged restriction r=mixedpuppy

Comment 13

[Andreea Pavel [:apavel]]

Backed out for mochitest failures at

Push with failures https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&revision=0abbf10b68b04c3922c86a92a2b4cb2e7d72c4fa&selectedJob=273804400

Failure log https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273804400&repo=autoland&lineNumber=7506

Backout: https://hg.mozilla.org/integration/autoland/rev/88e4bf1dea72f8e3cf094f2fd3a86832b67909b9

Comment 14

[Junior [inactive]]

Looks like conflict with test just landed today :(

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74229ac0f43d
P1 Expose requestSize to nsIHttpChannel r=dragana
https://hg.mozilla.org/integration/autoland/rev/1f11b971420d
P2 web request data API implementation r=mixedpuppy
https://hg.mozilla.org/integration/autoland/rev/f6b678baa610
P3 test for new attributes in browser.webRequest.onCompleted r=baku,mixedpuppy
https://hg.mozilla.org/integration/autoland/rev/a0c27d107be7
P4 remove privileged restriction r=mixedpuppy

Comment 16

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/74229ac0f43d
https://hg.mozilla.org/mozilla-central/rev/1f11b971420d
https://hg.mozilla.org/mozilla-central/rev/f6b678baa610
https://hg.mozilla.org/mozilla-central/rev/a0c27d107be7

Comment 17

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/74229ac0f43d
https://hg.mozilla.org/integration/mozilla-inbound/rev/1f11b971420d
https://hg.mozilla.org/integration/mozilla-inbound/rev/f6b678baa610
https://hg.mozilla.org/integration/mozilla-inbound/rev/a0c27d107be7