Closed
Bug 635309
Opened 15 years ago
Closed 5 months ago
Give httpd.js the ability to return multipart range responses
Categories
(Core :: Networking: HTTP, defect, P5)
Tracking
()
RESOLVED
WONTFIX
People
(Reporter: benjamin, Unassigned)
References
Details
(Whiteboard: [necko-would-take])
Attachments
(1 file)
|
18.06 KB,
patch
|
Waldo
:
feedback-
|
Details | Diff | Splinter Review |
In order to test multipart range requests that plugins make (and which have really sucky code), I need httpd.js to be able to actually handle multiple range requests and return the data reasonably. Anyone on the cc list want to volunteer or know of somebody who would be willing?
Comment 1•15 years ago
|
||
ccing some neckoish folks.
So, here's a WIP patch to add this functionality. I'm no HTTP expert, so someone needs to check if the actual protocol stuff is handled properly. The other sticky bit is whether what I'm doing here (writing a temporary file to construct the multipart response body) is the right thing to do.
Big changes:
* tried to reuse a bunch of the existing range request code, that's going to allow multipart range requests like this: 10-15,17- and I don't know if that is per spec or not.
* I'm detecting whether I have a multipart request or not based on whether the range part of the request contains commas. Is that a valid way to do this?
* Factored out the actual response writing out of writeFileResponse so that it could be used by the multipart writer as well (I needed a callback to clean things up).
It's not complete, but would this work for what you need Benjamin? Waldo, is this a good approach (with the temp file and all that), or should I have done this a different way? This is my first serious foray into httpd.js, so all feedback is welcome.
Attachment #516010 -
Flags: feedback?
Comment 3•15 years ago
|
||
Comment on attachment 516010 [details] [diff] [review]
WIP patch, take 1
Writing to a temporary file won't be acceptable, no.
>diff --git a/netwerk/test/httpserver/httpd.js b/netwerk/test/httpserver/httpd.js
> if (metadata._httpVersion.atLeast(nsHttpVersion.HTTP_1_1) &&
> metadata.hasHeader("Range") &&
> this._getTypeFromFile(file) !== SJS_TYPE)
> {
>- var rangeMatch = metadata.getHeader("Range").match(/^bytes=(\d+)?-(\d+)?$/);
>+ // Not sure how to do this with one regex, so ensure we have bytes= as we expect
>+ if (!metadata.getHeader("Range").match(/^bytes=/))
>+ throw HTTP_400;
If you're just testing, there's a test() (or search()?) method on regular expressions for that purpose.
A first check here, then chopping off bytes=, splitting on ',', and pattern-checking each individual component seems like the right way to do this.
>+ // Figure out if we have a multipart range or a single range request
>+ // TODO: Better way to do this? I'm looking for commas in the request
>+ var isMultipart = false;
>+ var multipartBuffer = "";
>+ var boundary = "***\n***\n";
>+ if (metadata.getHeader("Range").split(",").length > 1)
Get the header once, then act on that single value -- no reason to be looking it up multiple times.
I have not fully studied multipart format, but my recollection is that you need a boundary string that doesn't exist in any individual part of the response. I would guess servers just generate some random characters and use that and "hope" there's no collision, but I don't actually know. Anyone else know? Hard-coding a boundary string is definitely not right.
>+ {
>+ isMultipart = true;
>+
>+ // Go ahead and set the outer headers
>+ response.setStatusLine(metadata.httpVersion, 206, "Partial Content");
>+ response.setHeader("Content-Type", "multipart/byteranges", false);
>+ multipartBuffer += "boundary=" + boundary;
>+ }
>+
>+ rangeregex = /(\d+)?-(\d+)?/g;
>+ rangeMatch = rangeregex.exec(metadata.getHeader("Range"));
Again getting the header! :-)
This confuses me. Shouldn't you be doing this for every individual range in the header? So shouldn't this be in a loop over a split? Without investigating harder, I'm somewhat doubtful this really works.
Regarding the questions:
* tried to reuse a bunch of the existing range request code, that's going to
allow multipart range requests like this: 10-15,17- and I don't know if that is
per spec or not.
I suspect it is, but I don't know offhand.
* I'm detecting whether I have a multipart request or not based on whether the
range part of the request contains commas. Is that a valid way to do this?
I'd have to check spec whether bytes=1-3,4-6 is supposed to be a multipart or not, but that seems like the big case. And I don't remember if ranges are allowed to overlap, and whether they should be combined if so, or whatever, another thing to check.
* Factored out the actual response writing out of writeFileResponse so that it
could be used by the multipart writer as well (I needed a callback to clean
things up).
I didn't look at this, so I don't have anything to say yet about it.
Hopefully that's a reasonable start on feedback, keep asking questions if you have them.
Attachment #516010 -
Flags: feedback? → feedback-
| Reporter | ||
Comment 4•15 years ago
|
||
When I generate multipart/form-data requests, I generate a random boundary string but then search the data to see if it is present. If there's a conflict, I just regenerate a new random string.
(In reply to comment #4)
> When I generate multipart/form-data requests, I generate a random boundary
> string but then search the data to see if it is present. If there's a conflict,
> I just regenerate a new random string.
this makes some good sense. I was worried about this, but the spec is pretty light on the details. I'll look into that.
thanks for the feedback guys.
(In reply to comment #3)
> Comment on attachment 516010 [details] [diff] [review]
> WIP patch, take 1
>
> Writing to a temporary file won't be acceptable, no.
Do you have other ideas on how you'd like to see it implemented? Is there a good cache I can use to construct the response in memory?
>
> >diff --git a/netwerk/test/httpserver/httpd.js b/netwerk/test/httpserver/httpd.js
> > if (metadata._httpVersion.atLeast(nsHttpVersion.HTTP_1_1) &&
> > metadata.hasHeader("Range") &&
> > this._getTypeFromFile(file) !== SJS_TYPE)
> > {
> >- var rangeMatch = metadata.getHeader("Range").match(/^bytes=(\d+)?-(\d+)?$/);
> >+ // Not sure how to do this with one regex, so ensure we have bytes= as we expect
> >+ if (!metadata.getHeader("Range").match(/^bytes=/))
> >+ throw HTTP_400;
>
> If you're just testing, there's a test() (or search()?) method on regular
> expressions for that purpose.
true, thx.
>
> A first check here, then chopping off bytes=, splitting on ',', and
> pattern-checking each individual component seems like the right way to do this.
>
cool. I can modify it to do that.
>
> >+ // Figure out if we have a multipart range or a single range request
> >+ // TODO: Better way to do this? I'm looking for commas in the request
> >+ var isMultipart = false;
> >+ var multipartBuffer = "";
> >+ var boundary = "***\n***\n";
> >+ if (metadata.getHeader("Range").split(",").length > 1)
>
> Get the header once, then act on that single value -- no reason to be looking
> it up multiple times.
Gotcha.
>
> I have not fully studied multipart format, but my recollection is that you need
> a boundary string that doesn't exist in any individual part of the response. I
> would guess servers just generate some random characters and use that and
> "hope" there's no collision, but I don't actually know. Anyone else know?
> Hard-coding a boundary string is definitely not right.
>
>
> >+ {
> >+ isMultipart = true;
> >+
> >+ // Go ahead and set the outer headers
> >+ response.setStatusLine(metadata.httpVersion, 206, "Partial Content");
> >+ response.setHeader("Content-Type", "multipart/byteranges", false);
> >+ multipartBuffer += "boundary=" + boundary;
> >+ }
> >+
> >+ rangeregex = /(\d+)?-(\d+)?/g;
> >+ rangeMatch = rangeregex.exec(metadata.getHeader("Range"));
>
> Again getting the header! :-)
>
> This confuses me. Shouldn't you be doing this for every individual range in
> the header? So shouldn't this be in a loop over a split? Without
> investigating harder, I'm somewhat doubtful this really works.
>
Sorta. The header for the entire response needs to have certain parameters set. It's a whole lot less clear to me what needs to be set in the headers on the individual ranges. Also, I think calling this code multiple times will only reset the top header. I did it this way, because in the IDL it doesn't appear that the response object can differentiate between the header for the entire response and the individual range headers. Since both headers are using many of the same headers with different values, I was trying to work around that.
Maybe I'm attacking this at the wrong level, and the right fix is to make the response object aware of ranges, and let the response object use it's code for setting the proper headers.
What do you think?
Updated•10 years ago
|
Whiteboard: [necko-would-take]
Comment 7•9 years ago
|
||
Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258
Priority: -- → P5
Updated•4 years ago
|
Severity: normal → S3
Comment 8•5 months ago
|
||
I think we can use NodeServer for features not present in httpd.sys.mjs
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•