ftp via proxy broken in newcache

VERIFIED FIXED in mozilla0.9



18 years ago
18 years ago


(Reporter: bbaetz, Assigned: darin.moz)



Firefox Tracking Flags

(Not tracked)


(Whiteboard: [cache], URL)


(2 attachments)



18 years ago
To reproduce:
1. Set up an ftp proxy (I'm using squid)
2. Go to ftp://ftp.mcgill.ca
3. click on "pub"
4. click on "README"
5. Click back

Expected results:
ftp://ftp.mcgill.ca/pub is shown

Actual results:
The readme file is still shown, even though the urlbar is updated. It is
impossible to visit any other proxied ftp site without deleting the NewCache

It doesn't matter which directory/file is chosen - once you stop going forward,
it doesn't work any more. Shift-reload doesn't help either, and neither does
typing an ftp url in the url bar.

I first noticed this with gopher - I suspect that the dummy URI in
nsFtpProtocolHandler::NewChannel is confusing something.

Comment 1

18 years ago
not sure if there is a duplicate bug report opened elsewhere, but this is
definitely a known problem related to the VALIDATE_ONCE_PER_SESSION load
Keywords: nsbeta1
Whiteboard: [cache]
Target Milestone: --- → mozilla0.9

Comment 2

18 years ago
Hmmm, I see a slightly different behavior (Linux build 2001032521). I can
navigate through ftp directories, but when I click back in the mozilla there is
loaded some completely messed up document which contains parts of other
previously loaded documents.

I use squid proxy too.

Comment 3

18 years ago
Created attachment 28780 [details]
Screenshot of messed up display after clicking back button

Comment 4

18 years ago
Assignee: neeti → gordon

Comment 5

18 years ago
t8m@centriz.cz: Thats another bug, which is apparently known, although I
couldn't find a bug on it.

OK, despite what I said to darin last night on IRC, this is not a
revalidate_once bug - I can get this behaviour when set to revalidate_always,
its just a tiny bit harder to trigger:

Clear your mozilla cache (by removing the NewCache dir)
Using ns4, load ftp://ftp.mozilla.org/ and ftp://ftp.mozilla.org/bin/ (so that
the proxy has it stored)
Start mozilla
Go to ftp://ftp.mozilla.org/
Click on bin/
Observe that the pages are the same.

Note that about:cache has no ftp entry, but http://test.com does. I'll attach an
nspr.log file for http and ftp - all the cache entries refer to
http://test.com/. This log file shows mozilla sending an If-Modified-Since
header for ftp://ftp.mozilla.org/bin, even though it has never been fetched
before. The proxy still has this url cached, so it sends back a 304 response,
and so mozilla shows the old page.

The htp://test.com thing is bug 58872. After doing this, visiting
http://test.com/ gives you the ftp dir listing, unless you shift-reload.

The cache lookup should probably be keying off the overrideRequestSpec instead.

The attached log files are formed by doing ./mozilla ftp://ftp.mozilla.org/ then
clicking on the bin link. (Clicking on the bin link again tries to load
ftp://ftp.mozilla.org/bin/bin which fails)

Comment 6

18 years ago
Created attachment 28795 [details]

Comment 7

18 years ago
I'm a bit confused since ftp isn't using the new cache yet.  Is the proxy using 

QA Contact: gordon → tever
ftp proxies simply forward to http.  Note, that there are a bunch of ftp proxy 
bug scheduled for 0.9.1.

Comment 9

18 years ago
> Is the proxy using HTTP?

Yep - its probably an HTTP bug, using the wrong uri. This should be dealt with
before 0.9.1 though - it makes using ftp through a proxy almost impossible
(unless you shift-reload each page, and you need the patch for bug 73447 for
that to work)

Comment 10

18 years ago
I suspect that this is a dupe of two problems:

1) FTP does not transfer its load attributes to the HTTP channel (bug 73447)

2) HTTP does not handle VALIDATE_ONCE_PER_SESSION properly (bug 73099)
Component: Networking: Cache → Networking: HTTP


18 years ago
Depends on: 73099, 73447


18 years ago
No longer depends on: 73447

Comment 11

18 years ago
darin: No - I retested both with VALIDATE_ALWAYS (or whatever its called), and
with my patch for 73447.

Comment 12

18 years ago
ok... if using validate always didn't help, then this is not a dupe of 73099.
Assignee: gordon → darin
No longer depends on: 73099

Comment 13

18 years ago
Please, please fix this bug in Mozilla 0.9. It really makes the ftp through
proxy completely unusable.


18 years ago
Depends on: 75679

Comment 14

18 years ago
it sounds to me as though the HTTP proxy channel is not seeing the same load
flags as the FTP channel.  i believe this is because the load group's default
load flags are never anything other than LOAD_NORMAL.

Comment 15

18 years ago
Darin: when clicking back/forwards, the load attributes should be load_normal,
shouldn't they? The problem (from the logs) is that the "uri" http://test.com is
always the same, and so the cache thinks that it already has the result for that
uri. The ftp channel could force revalidate_always when talking to the proxy,
but thats a workarround, not a solution.

See bug 58872.

Comment 16

18 years ago
bbaetz: when clicking back/forward the load flags are to never validate, which
makes sense since it is just browsing through history.  but, i see your point
that this is likely due to HTTP caching everything to the same URL.  why should
HTTP be caching at all?  this seems wrong.  i'm working on a patch which will
add an INHIBIT_CACHING bit to the load flags... FTP (and other protocols that
proxy via HTTP) should really set this bit IMO.

Comment 17

18 years ago
darin: Why shouldn't http be caching the response? It is connecting to an HTTP
proxy which is sending back a cachable response, and sends a 304 response (with
an Age header) back to an If-Modified-Since request. Theres very little point in
an organisation setting up a caching proxy if mozilla is going to force it to
always revalidate.

The proxy server can have huristics set up for how long to cache for - squid
uses a minimum 24 hours for both ftp and gopher, by default. (roughly)

My uni's servers (where I get my mail) are all down for upgrades until this
evening - find me on IRC if you want to discuss this :)

Comment 18

18 years ago
My point was not that the proxy server would not do caching, but rather that we
would not need to do caching on behalf of FTP or other protocols b/c the proxy
would most likely be doing it.

At any rate, I landed changes on Friday along with my patch for bug 75679 that
makes HTTP use the ProxyRequestURI for the cache key instead of the regular URI.
This may very well solve this problem, though I've not tested it. 

Comment 19

18 years ago
I think I misunderstood what you were saying earlier.

I presume you meant:

"... but rather that we would not need to do caching on behalf of FTP or other
protocols b/c the proxy *channel* would most likely be doing it."

Anyway, I tested, and its fixed now, but I'll leave it open in case you want to
make those changes.

Comment 20

18 years ago
bbaetz: i really meant that the proxy server would be caching on behalf of
FTP and other protocols.  my initial feelings on the matter were that FTP
would take care of its own caching.  but, with the simple change of making
HTTP use the OverrideRequestURI for the cache key, we can have HTTP cache
the FTP requests.  i'm okay with this... this should probably be the default
behavior.  though, when dougt starts adding caching of FTP directory lists,
he may want to step back and consider whether or not that's necessary in the
proxy case... it may still be.

marking FIXED per bbaetz comments that it's working.
Last Resolved: 18 years ago
Resolution: --- → FIXED

Comment 21

18 years ago
Mozilla 0.9 all plats.
You need to log in before you can comment on or make changes to this bug.