843508 - Remove support for multipart XHR responses

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Henri Sivonen (:hsivonen)]

Rewriting multipart XHR responses to work with off-the-main-thread XML parsing would be an annoyance, other browser don't support the feature and telemetry introduced in bug 701361 shows that only 0.000184% of XHR responses use multipart. (Considering the total number of telemetry submissions for this metric, this means about 90 recorded multipart responses out of 49 million.)

Since the feature is used extremely little, is not supported by others and would cause extra work when rewriting XML parsing, I think we should remove the feature.

Comment 1

[Henri Sivonen (:hsivonen)]

Also worth noting that chunked XHR responses and Web Socket now exist.

Comment 2

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Sounds good. Though I think we need to warn about this for at least a couple of releases before we drop it. I believe it's been used in a few places as a way to do "comet streaming" or some such, so I think we'll want to give websites a chance to upgrade.

If we land a warning now we can probably uplift it to aurora.

Comment 3

[Henri Sivonen (:hsivonen)]

(In reply to Jonas Sicking (:sicking) from comment #2)
>  I believe it's been used in a few places as a
> way to do "comet streaming" or some such,

It's been used *somewhere*, because the telemetry didn't report an exact zero.

> so I think we'll want to give websites a chance to upgrade.

Do we have any indication whether previous such messages have had an effect?

Comment 4

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I'm pretty sure we've heard back from authors when we've added warnings, though I don't recall which feature(s) off the top of my head. And we've likewise heard complaints from authors when we drop features without having a warning.

I don't know that we've ever added a warning and then watched its effects on telemetry. It's hard to know how much such effects would vary from feature to feature though. I'd imagine there are lots of things that affect how effective such a warning would be (age of feature, how easy it is to migrate off of, how long the warning is in effect, etc).

That said, the cost of adding a warning is very low, and putting it in does IMO make us a better web citizen. I wouldn't r+ a patch to remove this feature without having had a warning first, but I'm not the DOM module owner.

Comment 5

[A. Przygienda]

well, I for my part ship a commercial backend product that will rely on it largely & that is NOT good news.

Comment 6

[Henri Sivonen (:hsivonen)]

(In reply to A. Przygienda from comment #5)
> well, I for my part ship a commercial backend product that will rely on it
> largely & that is NOT good news.

What do you do in non-Gecko browsers?

Comment 7

[A. Przygienda]

non gecko is hidden frames which is a hilarious pain to debug & keep up IMHO. Pure hack if there ever was one.

More info on the multipart-replace bug here:

Spend 2 days & 1 nite over it ;-{

and had to implement CORS in my server (don't ask, must XSite XHR on the product) and now stuff works like a charm after a pre-flight! I was going before through Apache proxy and that gives a completely unpredicatble behavior for SOME connections, if you just open it and push some small status into it (even bigger), it never delivers the first message. I played with proxy buffers, verified that proxy is not corrupting, forced chunked, switched gzip off, still completely random behavior.  Something in the proxy is making it choke & also introduces timing issues like I described before (i.e. loosing messages in the XML stream by complaining about invalid XML).

Comment 8

[Henri Sivonen (:hsivonen)]

Attachment: Remove support for multipart XHR

The warning landed as bug 844792.

Comment 9

[Henri Sivonen (:hsivonen)]

(In reply to A. Przygienda from comment #7)
> non gecko is hidden frames which is a hilarious pain to debug & keep up
> IMHO. Pure hack if there ever was one.

OK, no you have a code path that works without multipart XHR.

Do you have a reason not to use https://developer.mozilla.org/en-US/docs/Server-sent_events (cross-browser), Web Sockets (cross-browser) or chunked responses (Gecko-only). They are all less weird than multipart.

> More info on the multipart-replace bug here:

Are you saying multipart XHR isn't working for you currently?

Comment 10

[Henri Sivonen (:hsivonen)]

s/OK, no/OK, so/

Comment 11

[A. Przygienda]

web-socket is still wet behind the ears & I didn't look @ chunked responses. I have to automate things like airports & I can't rely on stuff that is 1 year into the spec  & beta (for my standards, no matter what people THINK it is). 

seems Firefox wasn't able to fix multipart for 7 years (and I developed for 4+ years towards this technology primarily), so what makes me believe you'll get any 'new sliced bread' correct ? 

And nope, mixed-replace is not working really 100% for me, I don't be able to proxy the  stuff to Firefox as described & it seems to loose data with multiple connections @ high speed, see bug https://bugzilla.mozilla.org/show_bug.cgi?id=292622. I look @ pushing 1000's of datapoints (better 10E4) into the browser and it does not seem to do that (at least not on windows @ 19.0, LInux @ 12.0 seems to work better). 

and hidden frames is an abomination to work with & ask customers to program to it, that's why I would prefer largely the multipart XHR.

Well, my best path will be probably go & help debug the WebSocket stuff on Chrome

Comment 12

[Henri Sivonen (:hsivonen)]

(In reply to A. Przygienda from comment #11)
> web-socket is still wet behind the ears & I didn't look @ chunked responses.
> I have to automate things like airports & I can't rely on stuff that is 1
> year into the spec  & beta (for my standards, no matter what people THINK it
> is). 

Why do you consider a non-standard single-browser feature (multipart XHR) something you could rely on but don't consider a standard multi-browser feature (Web Socket) as something you could rely on?

Also, https://xkcd.com/1172/

Comment 13

[A. Przygienda]

yes, thanks for the ironic link, really helps this defective software here shipping features that never get debugged properly & then get removed again. Won't waste probably more time posting into your bugzilla. I have to ship embedded products I can charge money for & my customers rely on and this kind of logic is not really helping me with that.

Comment 14

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I think your best bets for replacements here are (in order)

* XHR + .responseText. You wouldn't need to change your sever code *at all*.
  Works in all browsers that support XHR. In most modern browsers you can use the
  "progress" event. In older browsers you'd have to use setTimeout and poll.
* Server sent events has been around for a while and is quite stable. Uses same
  protocols as XHR, i.e. just a standard HTTP server. Doesn't work in IE.
  http://dev.w3.org/html5/eventsource/
  http://caniuse.com/#feat=eventsource
* WebSocket. Would require bigger changes of your server side code since websocket
  doesn't use HTTP. Works in latest version of all major browsers.
  http://caniuse.com/#feat=websockets
  http://dev.w3.org/html5/websockets/
* XHR + chunked responses. Similar to XHR + .responseText but uses less resources.
  only works in Gecko.
  https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest


All of these except the last one works in more browsers than multipart XHR. The first one would require just a relatively small amount of additional javascript and requires no other changes of client-side or server-side code.

Comment 15

[A. Przygienda]

Jonas, thanks, good overview. Problem is on the client side, on the server side I can do whatever I want but I have to teach my customers how to program against my feeds. So far multipart was the simplest to show & teach, just do multipart & get XML parsed pronto into your lap. all the other stuff is either BETA (sockets) or has same problems as multipart (not supported on all servers). I'll probably limp along on hidden frames & dying multipart support here & wait for the websocket to stabilize which is the right technology for the job (full bidirectional socket like in Flash which was actually where I started & that's why XML [old flash could only do \0 terminated XML messages])

Comment 16

[Henri Sivonen (:hsivonen)]

Web Socket is not in beta. All major browsers implement the standard version of the protocol:
https://developer.mozilla.org/en-US/docs/WebSockets

Web Socket is in all ways less experimental and more widely supported than multipart.

Comment 17

[Henri Sivonen (:hsivonen)]

Thanks landed.
https://hg.mozilla.org/integration/mozilla-inbound/rev/50d1db66fb1e

Comment 18

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Wait, this isn't giving developers the couple of releases of warnings requested in comment 2, does it?

Comment 19

[Henri Sivonen (:hsivonen)]

(In reply to Jonas Sicking (:sicking) from comment #18)
> Wait, this isn't giving developers the couple of releases of warnings
> requested in comment 2, does it?

It's giving a warning for one release. Should I back out and land after 6 weeks to give two releases of warning, request warning uplift all the way to beta or leave as-is?

Comment 20

[Henri Sivonen (:hsivonen)]

Requested warning uplift to beta in bug 844792 comment 16.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/50d1db66fb1e

Comment 22

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

What would be delayed if this couldn't land until after the next beach point.

Comment 23

[Henri Sivonen (:hsivonen)]

(In reply to Jonas Sicking (:sicking) from comment #22)
> What would be delayed if this couldn't land until after the next beach point.

The patch would likely rot. The warning was uplifted all the way to beta. There's now a couple of releases of warning. Let's leave this in.

Comment 24

[Henri Sivonen (:hsivonen)]

Documented:
https://developer.mozilla.org/en-US/docs/Site_Compatibility_for_Firefox_22
https://developer.mozilla.org/en-US/docs/Firefox_22_for_developers
https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest