594902 - httpd.js should support If-Modified-Since

Status: NEW

(Testing :: General, defect)

Description

[Bobby Holley (:bholley)]

(Followup bug for my tests over in bug 592641)

If I load the image from that test from a web server, hitting the reload button loads it from the cache. However, with the mochitest server we do a full reload.

I ran it through a debugger and it looks like the crucial difference is here:

http://hg.mozilla.org/mozilla-central/file/23b8229b4cef/modules/libpr0n/src/imgLoader.cpp#l1773

The caching channel is loading from the cache in the webserver case, but is not in the mochitest case. I've tried tweaking the headers to match to no avail.

I don't have time right now to go delving into necko and httpd.js on account of a test, so I'm leaving this as a followup bug. If jduell has any insights though, that could be helpful.

Comment 1

[Jason Duell]

> The caching channel is loading from the cache in the webserver case, but is not
in the mochitest case. I've tried tweaking the headers to match to no avail.

"To no avail" meaning you've actually got the headers to match?   Have you tried using wireshark to capture the network traffic so you can verify that you've got the same headers being sent by the server?

Cc-ing usual suspects for our cache code.

Comment 2

[Bobby Holley (:bholley)]

(In reply to comment #1)
> > The caching channel is loading from the cache in the webserver case, but is not
> in the mochitest case. I've tried tweaking the headers to match to no avail.
> 
> "To no avail" meaning you've actually got the headers to match?   Have you
> tried using wireshark to capture the network traffic so you can verify that
> you've got the same headers being sent by the server?

Not very rigorously. I wiresharked them and they looked the same for all intensive purposes, modulo an ETag. So I added that ETag to ^headers^ file, and verified it in Wireshark. Aside from that though, I didn't do anything to make the headers identical.

In general, I haven't investigated this very thoroughly. I just got to the point where I decided it's not worth spending time on right now (given that it's just for a mochitest). So if there are any off-hand ideas I'd like to hear them, but otherwise I'd be inclined to worry about this later.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Can you attach an HTTP log showing us loading just that test and then reloading it?  That'd at least be a starting point for looking for off-hand ideas; right now I literally have nothing to go on.

Comment 4

[Bobby Holley (:bholley)]

Attachment: Full Log

Comment 5

[Bobby Holley (:bholley)]

Attachment: First Load snippet

Comment 6

[Bobby Holley (:bholley)]

Attachment: Reload snippet

Comment 7

[Bobby Holley (:bholley)]

I made a plain mochitest-browser-chrome test with the following contents:

function test() {
  waitForExplicitFinish();
}

I then fired it up with NSPR_LOG_MODULES=nsHttp:5 and NSPR_LOG_FILE=~/log.txt. I loaded the image, waited a moment, and then hit the reload button. I then closed the browser.

I've attached the full log. I also attached what should be the relevant parts for the first and second loads (though I'm not sure I got all the relevant lines of setup and teardown).

Comment 8

[Boris Zbarsky [:bzbarsky]]

Thanks for those logs!  So here's the relevant part of the reload:

  1893604384[100a136e0]: Validating based on VALIDATE_ALWAYS load flag

(that flags is set because it's a reload via the reload button).  And then we do an HTTP GET with an If-Modified-Since, etc.  Note that at this point we have a cached image; that's why we have something to revalidate at all.

Now a real web server (well, a decent one, at least) will respond with an HTTP 304 response if the resource has not been modified, and then we'll read it from the cache.  If the image has been modified, the server will respond with an HTTP 200 and the new data.

The test server responds with an HTTP 200 resend of the data.  So we have to throw away our cached copy and read the "new" data.

I see no mention of If-Modified-Since or 304 responses in httpd.js, offhand.  So it sounds like it just doesn't support that stuff.  So any actual reload (using the reload button, not to be confused with just doing a normal load of a URI that's been loaded before) will be an effective cache miss when running against the test server.

Comment 9

[Boris Zbarsky [:bzbarsky]]

Ah, I suppose you could make the right thing happen by hand in your test using some sjs love or whatever it is that lets you just custom-write your responses in JS, right?

Comment 10

[Bobby Holley (:bholley)]

(In reply to comment #9)
> Ah, I suppose you could make the right thing happen by hand in your test using
> some sjs love or whatever it is that lets you just custom-write your responses
> in JS, right?

Indeed. I ended up using a different trick that Neil suggested (just leaving the first tab and opening up a second one), which did the trick for my test.

Thanks a ton bz for looking into that. I'm relabeling this as a Mochikit bug. I think it'd be pretty useful functionality to have, and the fact that we don't have it doesn't speak well of our cache test coverage...

Comment 11

[Bobby Holley (:bholley)]

> I'm relabeling this as a Mochikit bug.

And by that I mean httpd.js. I didn't know we had a component for it.

Comment 12

[Jeff Walden [:Waldo]]

Oh, *this* is what you were trying to say to me earlier today.  (Long day, brain frazzled, less so right now but still some work to do yet.)  Yes, httpd.js's don't-do-extra-work support is limited to Last-Modified right now, with nothing more fancy.

This could be a fun diversion once the JS/ES5 pace has slowed a little bit further -- maybe I can get to it in a few weeks' time.