408979 - Downloads are canceled on Exit when using a proxy server (HTTP/1.0)

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3pre) Gecko/2007121805 Minefield/3.0b3pre

Today I was wondering why a running or paused download gets canceled on exit of Firefox. At home the resuming is working fine on OS X and also WinXP. So something is going wrong at the company I work for. After thinking about I realized that it could be happen due to our proxy server. So I switched over to our external wlan with direct access. Restarting Firefox and repeating the steps shows me that it is working now. Downloads don't get canceled when exiting Firefox.

We are using several proxies due to load-balancing. All of them have squid/2.5.STABLEx installed.

Reproduce:
1. Entering proxy information within network configuration
2. Running a download from the above URL
3. Exiting Firefox

Now you get a warning that running downloads will be canceled. Instead of canceling the user should be able to resume the download later.

I'm only able to test it under XP but this bug should also happen under different OS.

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

[2:51am] whimboo: if i don't have a proxy and restart firefox the already downloaded size is shown. let's say 134MB of 650MB
[02:51am] whimboo: And i can resume the download
[02:52am] whimboo: but with a proxy it's always at zero size after a restart
[02:52am] stephend: yeah, I think I know where this is going
[02:52am] stephend: ok
[02:52am] stephend: gotcha now, thanks
[02:52am] stephend: it's the difference between "resume upon restart" and "save as 0 bytes and retry"

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I have to correct: it's already set to zero when you select Cancel inside the warning dialog and Firefox is closed.

Comment 3

[Shawn Wilsher :sdwilsh]

If your proxy server doesn't support download resume, I can't see it working (I'm not 100% sure how proxy servers work though...)

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The proxy supports resume due to flashget is running fine and can resume paused downloads after restart.

As I can see within moz_downloads we don't get an EntityID for downloads initiated from behind a proxy.

Comment 5

[Shawn Wilsher :sdwilsh]

If we don't get an entity id, we can't resume...

Comment 6

[Christian :Biesinger (don't email me, ping me on IRC)]

Is that an HTTP/1.0 server? What's network.http.proxy.version set to?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #6)
> Is that an HTTP/1.0 server? What's network.http.proxy.version set to?

No, the version is set to 1.1. As Christian pointed out here is the code where the entity id is generated:

http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/http/src/nsHttpChannel.cpp#4696

I'll try to get a log from Live HTTP headers later.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Log with Live HTTP headers

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: HTTP log

The proxy is really using HTTP/1.0. After talking with biesi on IRC I'll check tomorrow why it shouldn't be possible with HTTP/1.0.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

After having a detail view on the log and several documents I also believe it cannot be done with the current implementation. Due to we are using ETag which is only send by HTTP/1.1 servers we cannot build our EntityID.

As long there is no intention to implement an alternative way this should be WONTFIX?

Comment 11

[Shawn Wilsher :sdwilsh]

I think that WONTFIX might be the right thing here.

Comment 12

[Christian :Biesinger (don't email me, ping me on IRC)]

We don't require ETags. We use them when we have them, but we support resuming even when we don't.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: wget log (with response heads)

It's really interesting. I made a test with wget similar to the one on bug 401846. Running with the parameter -S shows the response heads of the HTTP requests. The final request still contains an ETag, Last-Modified and Content-Range header. So we still fail due to the HTTP/1.1 limitation.

I'll remove the check for >=HTTP/1.1 for my debug build and will have a look, what happens. For now it looks like we shouldn't resolve this bug as WONTFIX.

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Christian, after removal of the HTTP/1.1 check the resuming works fine. It's a good example why we shouldn't exclude all HTTP/1.0 servers. The question is what headers do we really need to have a resumable download?

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

And further do we have to differentiate between HTTP/1.0 proxies and servers?

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As I have seen proxy servers forward each header. So even if a HTTP/1.0 proxy is in usage it forwards ETag and Last-Modified headers from a HTTP/1.1 server. Against strictly blocking HTTP/1.0 servers we should check if such needed headers exist? Which headers do we really need?

Comment 17

[Christian :Biesinger (don't email me, ping me on IRC)]

we don't need any header. but having last-modified and content-length would be good.

Comment 18

[Christian :Biesinger (don't email me, ping me on IRC)]

maybe it's ok to just remove the 1.1 check. I'd r+ a patch to do that. the only bad thing that can happen, I think, is that we only detect later that resuming is not possible.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Remove HTTP/1.0 check

I'll start Tryserver builds later so we could see what happens.

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Tryserver builds are available at:
https://build.mozilla.org/tryserver-builds/2008-01-28_00:[e-mail]-bug408979/

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Finding a real HTTP/1.0 server seems to be not possible. But running the downloads over a HTTP/1.0 proxy works perfectly with my tryserver build.

Mardak or Sdwilsh, when I close Firefox I get a cancel dialog. Is this expected? We have a resumable download here so why that dialog is shown?

If anyone could tell me a HTTP/1.0 server I would be happy. Otherwise as biesi already told we should risk this way.

Comment 22

[Ed Lee :Mardak]

(In reply to comment #21)
> Mardak or Sdwilsh, when I close Firefox I get a cancel dialog. Is this
> expected? We have a resumable download here so why that dialog is shown?
See bug 401172 and bug 414161.

Comment 23

[Damon Sicore (:damons)]

Comment on attachment 299695 [details] [diff] [review]
Remove HTTP/1.0 check

a1.9+=damons

Comment 24

[Reed Loden [:reed]]

Checking in netwerk/protocol/http/src/nsHttpChannel.cpp;
/cvsroot/mozilla/netwerk/protocol/http/src/nsHttpChannel.cpp,v  <--  nsHttpChannel.cpp
new revision: 1.326; previous revision: 1.325
done

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Verified with Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3pre) Gecko/2008013004 Minefield/3.0b3pre ID:2008013004