Response payload is base64 encoded
Categories
(DevTools :: Netmonitor, defect, P3)
Tracking
(Not tracked)
People
(Reporter: pkukielka, Unassigned)
References
(Blocks 1 open bug)
Details
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0
Steps to reproduce:
request:
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0
Accept: application/json, text/plain, /, application/msgpack
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded, application/msgpack
X-Requested-With: XMLHttpRequest
Content-Length: 203
Origin: https://example.com:8080
Connection: keep-alive
Referer: https://example.com:8080/api
Cookie:
Pragma: no-cache
Cache-Control: no-cache
TE: Trailers
response:
HTTP/2 200 OK
server: nginx
date: Wed, 18 Mar 2020 12:14:45 GMT
content-type: application/msgpack
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
access-control-allow-credentials: true
access-control-allow-methods: post, get
access-control-allow-origin: https://example.com:8080
access-control-allow-headers: content-type, x-requested-with
strict-transport-security: max-age=15768000
X-Firefox-Spdy: h2
im seding this response body (its php msgpack_pack(['data' => ['me' => null]]);):
¤data¢meÀ
Actual results:
response payload is base64 encoded in dev tools ie:
gaRkYXRhgaJtZcA=
note:
fetch with 'response.arraybuffer()' gives correct data
Expected results:
one of:
- explain behavior at https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor/request_details
- add notice to dev tools that what i see is not what server send
- dont encode but it might be some security setting
additional note:
doesnt happen in chrome
|
||
Comment 1•4 years ago
|
||
Moving to the right component so the development team can further analyze this.
|
||
Comment 2•4 years ago
|
||
Thanks for the report!
Can you please provide a simple page (a test case) we could use to reproduce the issue on our machines?
That would already be a great help!
Honza
link:
https://mozbug1623503.000webhostapp.com/index.php
src:
<?php
$headers = getallheaders();
if (isset($headers['accept']) && $headers['accept'] === 'application/msgpack') {
header('content-type: application/msgpack');
echo 'meh';
die;
?>
<?php } else { ?>
<html>
<body>
<button id="bug" onclick="send()">Click me</button>
<script>
function send() {
async function get() {
const response = await fetch('https://mozbug1623503.000webhostapp.com/index.php', {
method: 'POST',
mode: 'cors',
cache: 'no-cache',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/msgpack',
'accept': 'application/msgpack',
},
redirect: 'manual',
referrerPolicy: 'no-referrer',
body: 'test'
});
return await response.arrayBuffer();
};
get().then((res) => {
const arr = new Uint8Array(res)
console.log(arr);
const str = arr.reduce((acc, item) => acc += String.fromCharCode(item), '' )
console.log(str);
});
}
</script>
</body>
</html>
<?php } ?>
simplified to just echo 'meh' gives
{"Response Payload":{"EDITOR_CONFIG":{"text":"bWVo","mode":"application/msgpack"}}}
also not sure if its clear but using "Developer Edition" of FF
|
|
||
Comment 4•4 years ago
|
||
Thanks for the test case!
I uploaded that online here:
http://www.janodvarko.cz/tests/bugzilla/1623503/index.php
But I am getting TypeError: NetworkError when attempting to fetch resource. when clicking the button.
What's wrong there?
Honza
hello,
not sure it kind of works for me (i get normal render as with GET but its probably your server config [probably nothing set in getallheaders function]), thats why to be sure ive uploaded it to some free hosting with link
ive chaned php version to match yours (php 5.6) and updated code to not use getallheaders):
<?php
if (!empty(file_get_contents('php://input'))) {
header('content-type: application/msgpack');
echo 'meh';
die;
?>
<?php } else { ?>
<html>
<body>
<button id="bug" onclick="send()">Click me</button>
<script>
function send() {
async function get() {
const response = await fetch('./index.php', {
method: 'POST',
mode: 'cors',
cache: 'no-cache',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/msgpack',
'accept': 'application/msgpack',
},
redirect: 'manual',
referrerPolicy: 'no-referrer',
body: 'test'
});
return await response.arrayBuffer();
};
get().then((res) => {
const arr = new Uint8Array(res)
console.log(arr);
const str = arr.reduce((acc, item) => acc += String.fromCharCode(item), '' )
console.log(str);
});
}
</script>
</body>
</html>
<?php } ?>
link is still the same
https://mozbug1623503.000webhostapp.com/index.php
sorry for ads on it :/
|
||
Comment 6•4 years ago
|
||
The priority flag is not set for this bug.
:Honza, could you have a look please?
For more information, please visit auto_nag documentation.
|
|
||
Comment 7•4 years ago
|
||
Thanks, I can reproduce the problem now!
STRs:
- Open DevTools and select the Network panel
- Load http://www.janodvarko.cz/tests/bugzilla/1623503/index.php
- Click the button on the page
- Select the request and check out the Response panel
- The response is
bWVo(based64 formeh)
decodeURIComponent(atob("bWVo")) => meh
Comments:
-
The Network panel backend is endoing all non-text responses in base-64 using
btoa
https://searchfox.org/mozilla-central/rev/b7f3977978922d44c7d92ae01c0d4cc2baca7bc2/devtools/server/actors/network-monitor/network-response-listener.js#490 -
The list of text mime-types is here
https://searchfox.org/mozilla-central/rev/b7f3977978922d44c7d92ae01c0d4cc2baca7bc2/devtools/shared/webconsole/network-helper.js#492
application/msgpack is not in the list.
- The Backend is setting a flag when encoding the response
response.encoding = "base64";but, it isn't checked out on the front end when rendering the response text in the Response panel.
- We already have a parser for
application/msgpackso, we could use it in the Response panel too.
https://searchfox.org/mozilla-central/rev/b7f3977978922d44c7d92ae01c0d4cc2baca7bc2/devtools/client/netmonitor/src/components/websockets/msgpack.js#20
Honza
|
|
||
Updated•4 years ago
|
|
||
Comment 8•4 years ago
|
||
Honza, should we pivot this bug into supporting msgpack formatting for responses; or is this affected other mime-types as well?
|
|
||
Comment 9•4 years ago
|
||
So, I am seeing several issues here:
-
The way we are detecting textual response is weak - we are using fixed list of known textual mime types. It would be safer to check whether the response itself is binary or text. See bug 752288 (detect binary responses)
-
This bug isn't only about
application/msgpackit's about any binary response that is base64 encoded (to be sent to the client over RDP) but, not decoded when displayed to the client. The client is not checking thebase64field (set on the backend when encoding happens). Properly show binary response to the client is blocked by bug 1555641 (binary viewer) -
It would be great to parse
application/msgpackresponses and show formatted view (JSON tree) to the user. Just like we do for WS frames. I am unsure how we want to show binary fields within theapplication/msgpackJSON structure (there might be some, right?). Perhaps keep them base64 decoded?
So, yes we can use this bug to properly parse application/msgpack if it helps.
Honza
|
|
||
Comment 10•4 years ago
|
||
Related, I saw in user feedback a case where utf8 encoded responses got base64-encoded – presumably also because of missing mime types. So solving this for beyond msgpack would be good to keep in mind.
|
||
Comment 11•4 years ago
|
||
Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is P3 (Backlog,) indicating it has been triaged, the bug's Severity is being updated to S3 (normal.)
Description
•