File upload blocks the following XHR requests in LAN

RESOLVED FIXED in Firefox 53

Status

()

Core
Networking: HTTP
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: johnhu, Assigned: schien)

Tracking

50 Branch
mozilla53
Points:
---

Firefox Tracking Flags

(firefox-esr45 ?, firefox50 wontfix, firefox51 wontfix, firefox52 wontfix, firefox53 fixed)

Details

(Whiteboard: [necko-active])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(3 attachments)

Created attachment 8824332 [details]
log.txt-main.5594.zip

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161208153507

Steps to reproduce:

0. prepare a webserver for uploading and responsing status at "LAN"
1. use XHR post to upload a big file (2GB+)
2. use XHR to get the status of uploading from server periodically  <== NG, no response until file had been uploaded

Please find the about:networking log file at attachment.




Actual results:

The second, third, forth, ... requests are blocked until file had been uploaded.


Expected results:

The second, third, forth, ... requests should be sent to server asap....
We found this issue at a testing LAN environment with server hardware. Those two devices are connected with gigabits network. The reproduce ratio is 100% in this network.

If we move the same settings to slower network environment, the reproduce ratio is pretty low. But, it is still reproducible in a slower LAN, like 100MB, but not 100%.
Created attachment 8824335 [details]
screen shot of web console when the issue happened

This attachment is the screen shot of web console when the issue happened.

The "stream" request is the file uploading XHR. Once this one had issued, the other requests, like "status", "history", "session" are pended until 26 seconds later.
BTW,

We can reproduce this issue at Firefox 50.1.0 and Nightly..
From the nspr log in attachment, STS thread is trapped in busy loop writing HTTP request to socket output stream.
https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsHttpConnection.cpp#1692

The first nsSocketOutputStream::AsyncWait is hit after uploading 500MB of the file.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Component: Untriaged → Networking: HTTP
Product: Firefox → Core
We can try breaking this loop by limiting the total bytes we can write in one OnSocketWritable call. In this case STS thread will be able to process more transactions in parallel.
Comment hidden (mozreview-request)
Comment on attachment 8824362 [details]
Bug 1329089 - yield to other transactions while reaching the max write attempts.

limit the maximum loop count per OnSocketWritable, so that STS thread will not be blocked by one large HTTP request.
Attachment #8824362 - Flags: feedback?(honzab.moz)
Comment on attachment 8824362 [details]
Bug 1329089 - yield to other transactions while reaching the max write attempts.

thanks. I actually think simpler is better here given the corner case; drop the pref and just hardcode a number. It will be much less code. Even as low as 128 might be worth trying.
Attachment #8824362 - Flags: feedback?(honzab.moz) → feedback+
SC,

I had downloaded the test version from your server and do the 10+ tests. This issue disappeared in this test build.

Please try to push to Firefox beta if possible.

Thank you very much.....
Comment hidden (mozreview-request)
status-firefox50: --- → affected
status-firefox51: --- → affected
status-firefox52: --- → affected
status-firefox53: --- → affected
status-firefox-esr45: --- → ?

Comment 12

a year ago
mozreview-review
Comment on attachment 8824362 [details]
Bug 1329089 - yield to other transactions while reaching the max write attempts.

https://reviewboard.mozilla.org/r/102884/#review103958

lgtm thanks.. I don't think this meets uplift criteria fwiw
Attachment #8824362 - Flags: review?(mcmanus) → review+
(In reply to Patrick McManus [:mcmanus] from comment #12)
> Comment on attachment 8824362 [details]
> Bug 1329089 - yield to other transactions while reaching the max write
> attempts.
> 
> https://reviewboard.mozilla.org/r/102884/#review103958
> 
> lgtm thanks.. I don't think this meets uplift criteria fwiw

uplift to beta may be too exaggerated, how about aurora?
Flags: needinfo?(mcmanus)

Comment 14

a year ago
Pushed by schien@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/49414a3861de
yield to other transactions while reaching the max write attempts. r=mcmanus

Updated

a year ago
Assignee: nobody → schien
Whiteboard: [necko-active]

Comment 15

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/49414a3861de
Status: NEW → RESOLVED
Last Resolved: a year ago
status-firefox53: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
this is a tail use case - it can ride the trains and doesn't meet the uplift criteria imo
Flags: needinfo?(mcmanus)

Updated

a year ago
See Also: → bug 690633
cf comment #16
status-firefox50: affected → wontfix
status-firefox51: affected → wontfix
status-firefox52: affected → wontfix
You need to log in before you can comment on or make changes to this bug.