448652 - Very Long URL seems to be blocked by "Security Zone Policy"

Status: RESOLVED FIXED

(Toolkit :: Downloads API, defect)

Description

[vwm]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1

The Download of resources with "very long URLs" (i.e. 1683 Characters and more) seems to be blocked by Security Zone Policy. Slightly shorter URLs (i.e. 1674) from the same domain do not have that problem.

The Example-URL will only work if you are logged in at zyb.com (registration is free) - I am sorry about that, but I am not aware of other public servers that generate urls long enough.

Reproducible: Always

Steps to Reproduce:
1. Log-in at zyb.com (any login should do)
2. Enter / select the URL mentioned above
3. When ask what to do, select "Save File"
4. Select Path and Filename
Actual Results:  
The file is generated with file size 0 and without content.

In the "Downloads" List, the File is listed along with the error Message
"This download has been blocked by your Security Zone Policy" (in my German Version "Dieser Download wurde durch Ihre Sicherheitszonen-Regel blockiert")

Expected Results:  
The file should be properly saved instead. When I use a shorter link (Example below), it is.

Additionally, I would expect some kind of notification if a download fails. With the zero-size file and no visible message one might wrongly assume that the download was successful.

I am using a German version of Firefox, so sorry in case I mistranslated some the messages that Firefox generates!

The Examples have been anonymized -- therefore a "successful" download will also generate a zero-size file, but no error message in the downloads. If you substitute the "12345678"'s by the number of an actual ZYB-Contact, the successful download will provide a file with data, the non-successfull will still provide an empty file.

Some URL that will *not* generate an Error:
http://zyb.com/core/contacts/export/?ids=12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,
12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678


Some URL that will generate the Error:
http://zyb.com/core/contacts/export/?ids=12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,
12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678,12345678

Comment 1

[Martijn Wargers (dead)]

Confirmed, using current trunk build on windows.

Comment 2

[Samuel Sidler (old account; do not CC)]

Let's look at this for 1.9.0.3.

Comment 3

[Samuel Sidler (old account; do not CC)]

Shawn, can you look at this bug?

Comment 4

[Shawn Wilsher :sdwilsh]

(In reply to comment #3)
> Shawn, can you look at this bug?
jimm has done all the work with this code in the past.  I don't really know windows APIs, which is what this is.

Comment 5

[Samuel Sidler (old account; do not CC)]

Jim, do you have time to look? :)

Comment 6

[Jim Mathies [:jimm]]

sure. we'll have a patch that shunts off the security zone stuff, so there will be a work around. but maybe we can do something specific too for this.

Comment 7

[Jim Mathies [:jimm]]

Note we now have two prefs that allow users to disable security features that may enforce url length restrictions. The first is - 

browser.download.manager.skipWinSecurityPolicyChecks

Which disables security policy calls on windows. The second is

browser.download.manager.scanWhenDone

which disables all virus scanning. 

Both of the apis used here and the services they call into potentially place restrictions on url length.  I haven't had a chance to test this yet but my guess is skipWinSecurityPolicyChecks will probably solve the problem. I'm not a fan of filtering corner case long urls out before making these calls, so If this is the problem I'd prefer we rely on disabling the service so users know full well what security restrictions are in effect. I'll do some testing to confirm this.

Comment 8

[vwm]

The setting of browser.download.manager.scanWhenDone does not affect that error in the current Nightly Build (3.1b2pre.en-US) and the current release (Firefox/3.0.3), so I am going to test browser.download.manager.skipWinSecurityPolicyChecks
 as soon as I get a build that offers this setting.

Comment 9

[Jim Mathies [:jimm]]

Did a little testing, this is definitely caused by a url length restriction in the IAttachementExecute interface. Since it's an extreme case, I leaning toward leaving the code as is unless we get more complaints from additional sites. The functionality can be disabled through the skipWinSecurityPolicyChecks currently available on trunk.

Comment 10

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.1

This patch smooths things out a bit by detecting invalid argument errors in the virus interface calls. I've also added a length check on the source that chops off uri path and cgi data before making the final Save() call. The specs don't require the full uri, so I think this is safe to do.

Comment 11

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.1

Slight cleanup on the length check.

Comment 12

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.1

One more time, with white space removed.

Comment 13

[Daniel Veditz [:dveditz]]

Too late to land in 1.9.0.4, request branch approval when it's been verified on mozilla-central.

Comment 14

[Shawn Wilsher :sdwilsh]

Comment on attachment 344297 [details] [diff] [review]
IAE invalid arg patch v.1 

>diff --git a/toolkit/components/downloads/src/nsDownloadScanner.cpp b/toolkit/components/downloads/src/nsDownloadScanner.cpp
>   nsCAutoString origin;
>-  rv = uri->GetSpec(origin);
>+  // Certain virus interfaces do not like extremely long uris.
>+  // Chop off the path and cgi data and just pass the base domain. 
>+  if (origin.Length() < 1683)
>+    rv = uri->GetSpec(origin);
>+  else
>+    rv = uri->GetPrePath(origin);
two things - origin.Length() is going to always be zero when you check it with the current code, and please pull out that number into a constant, with a comment explaining what it is.

Comment 15

[Jim Mathies [:jimm]]

(In reply to comment #14)
> (From update of attachment 344297 [details] [diff] [review])
> >diff --git a/toolkit/components/downloads/src/nsDownloadScanner.cpp b/toolkit/components/downloads/src/nsDownloadScanner.cpp
> >   nsCAutoString origin;
> >-  rv = uri->GetSpec(origin);
> >+  // Certain virus interfaces do not like extremely long uris.
> >+  // Chop off the path and cgi data and just pass the base domain. 
> >+  if (origin.Length() < 1683)
> >+    rv = uri->GetSpec(origin);
> >+  else
> >+    rv = uri->GetPrePath(origin);
> two things - origin.Length() is going to always be zero when you check it with
> the current code, and please pull out that number into a constant, with a
> comment explaining what it is.

Doh! I'll go back to the first patch I submitted, that one was correct. I'll split the constant out and resubmit.

Comment 16

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.2

updated.

Comment 17

[Shawn Wilsher :sdwilsh]

Comment on attachment 344675 [details] [diff] [review]
IAE invalid arg patch v.2

>+// Maximum length for URI's passed inot IAE
>+#define MAX_IAEURILENGTH 1683
I think you want "into"

r=sdwilsh

Comment 18

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.2

text touched up.

Comment 19

[Serge Gautherie (:sgautherie)]

Comment on attachment 344931 [details] [diff] [review]
IAE invalid arg patch v.2

abort: bad hunk #3 @@ -485,32 +491,40 @@ nsDownloadScanner::Scan::Start()
 (33 32 40 40)

Comment 20

[Jim Mathies [:jimm]]

Attachment: IAE invalid arg patch v.2 resubmit [Checkin: Comment 21]

lets try again...

Comment 21

[Serge Gautherie (:sgautherie)]

Comment on attachment 345737 [details] [diff] [review]
IAE invalid arg patch v.2 resubmit
[Checkin: Comment 21]

http://hg.mozilla.org/mozilla-central/rev/cdb84fbfb5bb

Comment 22

[Andrew B. Berhow]

The URLs listed are actually 2083 and 2074 characters long, not 1683 and 1684 (count them). 2083 characters is the Maximum URL length in Internet Explorer. This appears to be an Internet Explorer issue which is why it's coming back with the Security Zone Policy error.

http://support.microsoft.com/kb/208427

Comment 23

[Andrew B. Berhow]

FYI, I have the same issue on an internal site which uses a GET to post data to a PERL script. Smaller data posts go through (<2083), larger ones produce the same Security Zone Policy error (>=2083).