1778462 - When a response contains base64 and Content-Transfer-Encoding header is set to base64 the response tab under Network section of dev tool will r-eencode the response.

Status: NEW

(DevTools :: Netmonitor, defect, P3)

Description

[Daniel Fisher]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0

Steps to reproduce:

Prerequisites:
Have a web service that responds to a GET request with the response header Content-Transfer-Encoding: base64. For instance: https://testrfc7030.com/.well-known/est/csrattrs

• Hit F12 to open the DEV-Tools
• Entered the URL of the web service into the address bar and hit [Enter] to make a GET request (For the given sample URL the certificate is not trusted and the issue shall be ignored)
• Look at the response tab of the recorded request in the network section.

Actual results:

The value shown in the response tab of the network section of the DEV Tools is TURFR0J5c0dBUUVCQVJZR0NTcUdTSWIzRFFFSkFRWUZLNEVFQUNJR0NXQ0dTQUZsQXdRQ0FnWUpLb1pJaHZjTkFRa0g

Expected results:

As the downloaded file, the value shown in the response tab of the network section of the DEV Tools should have been: MDEGBysGAQEBARYGCSqGSIb3DQEJAQYFK4EEACIGCWCGSAFlAwQCAgYJKoZIhvcNAQkH

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'DevTools::Netmonitor' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Julian Descottes [:jdescottes]]

We currently show the base64 encoded payload. It could be nice to surface this information, and maybe offer a way to encode/decode on the fly. But the current approach doesn't seem completely wrong.

Comment 3

[Daniel Fisher]

Nope. It's completely wrong.

The Response contains Base64 data, that you take the bytes of and encode it with base64 AGAIN.

Here is some pseudocode:

let responseBody = response.Body;
if(response.Headers["Content-Transfer-Encoding"] == 'base64'){
  responseBody  = btoa(responseBody );
}
showInUI(responseBody )

let responseBody = response.Body;
showInUI(responseBody )

There is an unnecessary base64 encoding on the data when shown in the UI. The representation shown is not what is on the wire.

What I could imagine is that the original intend was to decode the base64 data to make it readable - what would make sense as log as it is not binary...

That we would have:

let responseBody = response.Body;
if(response.Headers["Content-Transfer-Encoding"] == 'base64'){
  responseBody  = btoa(responseBody );
}
showInUI(responseBody )

let responseBody = response.Body;
if(response.Headers["Content-Transfer-Encoding"] == 'base64'){
  responseBody  = atob(responseBody );
}
showInUI(responseBody )

Comment 4

[Julian Descottes [:jdescottes]]

Indeed, my bad. That seems related to the following code https://searchfox.org/mozilla-central/rev/284187d0a7130d21042beeff0af0627c8e68cacc/devtools/server/actors/network-monitor/network-response-listener.js#504-520

    try {
      response.mimeType = this.request.contentType;
    } catch (ex) {
      // Ignore.
    }

    if (
      !response.mimeType ||
      !NetworkHelper.isTextMimeType(response.mimeType)
    ) {
      response.encoding = "base64";
      try {
        response.text = btoa(response.text);
      } catch (err) {
        // Ignore.
      }
    }

If I understand correctly, when the content is not text mime type, we encode it to base64 before sending it to the client. When this was originally introduced in Bug 787981, it seems that the response content was not displayed when we couldn't get the mimeType. But nowadays we try to display it. For other supported mime types (such as images) we use the base64 encoded string in data:urls for our previews, so this is fine. But here we basically have text which got base64 encoded because we couldn't guess the mimetype, but which we still preview as text.

Honza, what do you suggest to do for such responses? At least decode the base64 encoded string?

Comment 5

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Yes, if Content-Transfer-Encoding: base64 header is detected we should try to decode the base64 encoded string and show it in the UI (instead of encoding it again).

Comment 6

[Daniel Fisher]

I would decode only if there is the indication that the content is not a binary format using a isTextMimeType call:

    let isTextMimeType = NetworkHelper.isTextMimeType(response.mimeType)
    if (
      !response.mimeType ||
      !isTextMimeType
    ) {
      response.encoding = "base64";
      try {
        if(response.Headers["Content-Transfer-Encoding"] == 'base64'){
          if(isTextMimeType){
              responseBody  = atob(responseBody );
          }
        } else {
          response.text = btoa(response.text);
        }
      } catch (err) {
        // Ignore.
      }
    }

Comment 7

[Jan Honza Odvarko [:Honza] (always need-info? me)]

I would decode only if there is the indication that the content is not a binary format using a isTextMimeType call:

Yes, good call.