978144 - Remote debugging protocol: multipart/form-data request: headers are not exposed in the request header array

Status: RESOLVED INACTIVE

(DevTools :: Netmonitor, defect)

Description

[Thomas Andersen]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20140212131424

Steps to reproduce:

For http multipart/form-data requests the Content-Type and Content-Length headers are only to be found in the payload text header field for the request.

According to Mihai these headers should be found in the requestHeaders.headers array too. See the discussion here: https://bugzilla.mozilla.org/show_bug.cgi?id=859059#c66

Comment 1

[Thomas Andersen]

Hi again

Just noted that the ContentType request header exists in the headers.requestHeaders array when the form is created with pure JS / FormData and XHR

http://mr-andersen.no/curl/

See Multipart Form Test and Multipart with FormData and XHR





978144

Comment 2

[Thomas Andersen]

Hi again Mihai

Please take a look at this:

http://mr-andersen.no/curl/

1. Try the old-school multipart request (Mulitpart Form Test)
2. Then try Upload image (Multipart with FormData and XHR2)

compare the results using the Network monitor

#1 Content-Type is only to be found in the request payload.
#2 Content-Type is only to be found in the request headers array.

Question: When this bug is fixed, does that mean that Content-Type always will exist in the requestHeaders array?

Comment 3

[Thomas Andersen]

continuation of comment #2:

in other words when a XHR request is sent with a serialized string

eg.: xhr.send("param1=value1&param2=value2&param3=value3");

The devtools view is different from when the xhr.send() method gets a FormData instance.

Comment 4

[Thomas Andersen]

Just so I get this right.
The request post data/request payload should never have a first part with the request's request headers. 
Today the Content-XXX headers for the request is to be found in the first part of the request payload (see example below).
The request headers should be found in headers.requestHeaders

This should apply to whether the request is performed with XHR or not.

Example from Chrome devtools (old-school multipart request):

------WebKitFormBoundarywZyWdLvxzBGPDBH8
Content-Disposition: form-data; name="file"; filename="Screen Shot 2014-03-07 at 12.00.48.png"
Content-Type: image/png


------WebKitFormBoundarywZyWdLvxzBGPDBH8
Content-Disposition: form-data; name="html"

Text echoed back to request
------WebKitFormBoundarywZyWdLvxzBGPDBH8--

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Example from Firefox (old-school multipart request)
Note the first part is containing the Content-XXX headers for the request


Content-Type: multipart/form-data; boundary=---------------------------1744130710782707572029599513
Content-Length: 196615

-----------------------------1744130710782707572029599513
Content-Disposition: form-data; name="file"; filename="Screen Shot 2014-03-07 at 12.00.48.png"
Content-Type: image/png

PNG

Comment 5

[Thomas Andersen]

And Chrome devtools removes the binary from file uploads. Why not do the same in Firefox devtools?

Comment 6

[Thomas Andersen]

Sorry for comment flooding this issue. This is the correct Firefox example (ref comment #4):

Content-Type: multipart/form-data; boundary=---------------------------2462418983865919171312316844
Content-Length: 196615

-----------------------------2462418983865919171312316844
Content-Disposition: form-data; name="file"; filename="Screen Shot 2014-03-07 at 01.00.48.png"
Content-Type: image/png

PNG
... binary data
-----------------------------2462418983865919171312316844
Content-Disposition: form-data; name="html"

Text echoed back to request
-----------------------------2462418983865919171312316844--

Comment 7

[Mihai Sucan [:msucan]]

Thomas, I should clarify what I meant in bug 859059.

(In reply to Thomas Andersen from comment #0)
> User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0)
> Gecko/20100101 Firefox/27.0 (Beta/Release)
> Build ID: 20140212131424
> 
> Steps to reproduce:
> 
> For http multipart/form-data requests the Content-Type and Content-Length
> headers are only to be found in the payload text header field for the
> request.
> 
> According to Mihai these headers should be found in the
> requestHeaders.headers array too. See the discussion here:
> https://bugzilla.mozilla.org/show_bug.cgi?id=859059#c66

This is not entirely accurate. For multipart/form-data requests the browser sends the request headers which include Content-Type and a ;boundary attribute that tells the boundary identifier used for separating payloads. Each payload has its own headers, including, but not limited to, Content-Type and Content-Length. As such, these should not be confused with the multipart/form-data request headers. In comment #0 you seem to suggest that I said that the Content-Type header of a payload should be part of the server reply packet for getRequestHeaders() [1]. That's not exactly what we want.

In bug 859059 I asked if the *main* multipart/form-data includes the ;boundary attribute, and you were right to point out it's not included in the Content-Type header value, because we get the parsed value from Necko. In one of the comments I replied we need to extend our network logging to include the raw request headers - see bug 859133. These would still only be the *main* multipart/form-data request/response headers. And they might be part of a separate packet - maybe not in the same getRequestHeaders() reply.

This bug should be about using the raw Content-Type header that will be available when bug 859133 is fixed, for removing the binary data in Curl.jsm. Currently the script manually tries to detect the payload boundary, instead of reading it from the Content-Type header.

Separately, it would be nice if the network monitor actors would expose payloads, individually, for each request, so we could ask from the client for payloads, their headers, and raw data. The summary of this bug seems to suggest these kind of capabilities.


Thanks for your investigation, and apologies for any confusion.

[1] https://developer.mozilla.org/en-US/docs/Tools/Web_Console/remoting#The_getRequestHeaders_and_other_packets

Comment 8

[Thomas Andersen]

Mihai, I am sorry for that I misunderstood what you meant and misquoted you in bug 859059. 
I guess I was a bit confused for a while. Thanks for clearing up things in comment #7.

Anyway, I am now pretty sure the binary removal in bug 859059/copy-as-curl is buggy now :) and will submit a new patch for this.

Comment 9

[Jan Honza Odvarko [:Honza] (always need-info? me)]

No activity for 7 years and there is a good chance that his has been fixed, closing.

Please feel free to report a new bug if you can still see the issue.

Honza