280255 - www.websina.com/bugzero/jsp/home.jsp doesn't work well when cache option is set to "Every time I view the page".

Status: RESOLVED INCOMPLETE

(Core :: Networking, defect)

Description

[Daniel Brahneborg]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041217
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041217

Javascript is used to reload a page with slightly different parameters, but
Mozilla uses the old data from the cache instead.

It works fine in Firefox 1.0, and if an explicit "Clear Cache" is done just
before the page refresh.

Reproducible: Always

Steps to Reproduce:
1.Login as guest/guest.
2.Click on the headers, which should sort the list according to the specified field.
3. Alternately, select a fields from the "Sort Order" drop down.
4. The page reloads slightly, but the main contents is taken from the cache.
Actual Results:  
The new content is taken from the cache, making it appear as if the sorting
doesn't work.

Expected Results:  
The new content should be loaded from the server.

The cache is set to use the new page "Every time I view the page".

I set the Severity to Major, since Mozilla shows incorrect data, making some web
applications useless.

Comment 1

[WADA:World Anti-bad-Duping Agency]

WORKSFORME with trunk-2005012005(Win-2K).
Time stamp data (text data in HTML written by server) following "My Favorites"
was updated every time I clicked header for sort, and sorting was done.

Do you use proxy server?

By the way, JavaScript has no relation, since the headers for sort is only a link.
> <a href="/bugzero/servlet/sorting?parent=home.jsp&sort_column=entryId"
>    class="button">ID#</a>
> <a href="/bugzero/servlet/sorting?parent=home.jsp&sort_column=synopsis"
>    class="button">Synopsis</a>
Change summary to appropriate one.

Comment 2

[Daniel Brahneborg]

I tried both with and without Privoxy as a proxy server, but it didn't make any
difference.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #2)
> I tried both with and without Privoxy as a proxy server, but it didn't make any
> difference.
Install Live HTTP Headers Extention and see HTTP Header log.
 http://livehttpheaders.mozdev.org/index.html
What header data is sent/received when the sort field is clicked?

Comment 4

[Daniel Brahneborg]

Attachment: HTTP headers on first request after "clear cache"

These are the headers that are sent on the first request after clearing the
cache. The new data is used correctly.

Comment 5

[Daniel Brahneborg]

Attachment: HTTP headers on second request

These are the headers that are sent on the second request. It looks as if the
"max-age=0" is ignored for some reason.

Comment 6

[WADA:World Anti-bad-Duping Agency]

Correction of my "WORKSFORME" in comment #1.
Sorting of search result(table rows) was not achieved when header for sort is
clicked.
(Time stamp was updated always.)

Following is overview of HTTP headers.

[1.Mozilla->Server]
http://lina.infoflex.se/bugzero/servlet/sorting?parent=home.jsp&sort_column=priority
GET /bugzero/servlet/sorting?parent=home.jsp&sort_column=priority HTTP/1.1
Host: lina.infoflex.se

[2.Mozilla<-Server]
HTTP/1.x 302 Moved Temporarily
Location: http://lina.infoflex.se/bugzero/jsp/home.jsp?startIndex=0

[3.Mozilla->Server]
http://lina.infoflex.se/bugzero/jsp/home.jsp?startIndex=0
GET /bugzero/jsp/home.jsp?startIndex=0 HTTP/1.1
Host: lina.infoflex.se

[4.Mozilla<-Server]
HTTP/1.x 200 OK
Content-Type: text/html;charset=ISO-8859-1
Transfer-Encoding: chu

[5.Mozilla->Server]
GET /bugzero/jsp/script/utils.js HTTP/1.1
and 
GET /bugzero/jsp/script/query.js HTTP/1.1

(When second request case. nexts are not sent when after "clear cache")
If-Modified-Since: Sat, 02 Oct 2004 02:41:18 GMT
If-None-Match: W/"977-1096684878182"
Cache-Control: max-age=0

[6.Mozilla<-Server]
(when after "clear cache" case)
HTTP/1.x 200 OK
(When second request case)
HTTP/1.x 304 Not Modified

"Timestapm field is already update" means that no cached data for HTML text is used.
Used cached data on second case is utils.js & query.js.

Daniel Brahneborg, how "bugzero/jsp/home.jsp?startIndex=0" know original sorting
information of "&sort_column=priority" after redirect?
Save "&sort_column=priority" in disk file before return 302 and redirect?
(If this is done by changing utils.js and query.js content, this bug's
phenomenon can be understood, although it doesn't look so.)
If sort key is required by home.jsp, I think server(jsp code) should return
something like "bugzero/jsp/home.jsp?startIndex=0&sort_column=priority" when
redirect.

"Every time I view the page" seems to be not applied on <script src> in the HTML.
I don't know whether it is spec or bug, violation of standard or not. 

Comment 7

[bill_hen]

Hi, this is Sam, I am an developer of the Bugzero application.

When do sorting, the sort_column is sent to the server
as a parameter in URL (a GET request to a servlet).
The server sends back the data, but the client URL (jsp)
stays the same, the sort_column is not appended on the URL.

The strange thing is, this problem has never been reported.
And we did tested on Mozilla previous versions extensively
before.

Here is our demo site:
http://www.websina.com/bugzero/jsp/login.jsp
and below is the related apache access log:
(The client is Firefox 1.0 on XP,
Apache sits before Tomcat)

192.168.16.25 - - [31/Jan/2005:22:58:07 -0800] "GET /bugzero/jsp/home.jsp
HTTP/1.1" 200 7993 "http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0"
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5)
Gecko/20041107 Firefox/1.0"    <-- click on HOME

192.168.16.25 - - [31/Jan/2005:22:58:19 -0800] "GET
/bugzero/servlet/sorting?parent=home.jsp&sort_column=synopsis HTTP/1.1" 302 0
"http://www.websina.com/bugzero/jsp/home.jsp" "Mozilla/5.0 (Windows; U; Windows
NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0"  <-- sort on synopsis

192.168.16.25 - - [31/Jan/2005:22:58:19 -0800] "GET
/bugzero/jsp/home.jsp?startIndex=0 HTTP/1.1" 200 8016
"http://www.websina.com/bugzerocn/jsp/home.jsp" "Mozilla/5.0 (Windows; U;
Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0"
--> display the result after sort on synopsis

192.168.16.25 - - [31/Jan/2005:22:58:44 -0800] "GET
/bugzero/servlet/sorting?parent=home.jsp&sort_column=assignable HTTP/1.1" 302 0
"http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0" "Mozilla/5.0
(Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0"
--> click on sort assignable

192.168.16.25 - - [31/Jan/2005:22:58:44 -0800] "GET
/bugzero/jsp/home.jsp?startIndex=0 HTTP/1.1" 200 8016
"http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0" "Mozilla/5.0
(Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0"
--> display on assignable

The size (8016) of the page stayed the same because I have only one page
and the sorting won't change the total bytes. But the page did get changed.
If I have multiple pages, sorting does change the page size.

Comment 8

[bill_hen]

Just want to make things a little bit more clear:

When a sort is clicked, the sort_column is sent
to a servlet, which does the sorting and put the
sorted list into an http session object.
It then re-direct to the original jsp page.
This jsp page simply loads up the list
from the session and displays it.
The sort_column is obtained from the list object,
but it is used only for the order image (up/down),
not for sorting, which is already done.

-Sam.

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> When a sort is clicked, the sort_column is sent
> to a servlet, which does the sorting and put the
> sorted list into an http session object.

What is the session object? (Sorry but I only know on Apache/PHP case.)  
Where and how is it saved?
How will it be passed to next process(=redirected URI)? 

ie.
What is your expectation on how "which session object is to be used" is passed
to next(redirected) "/bugzero/jsp/home.jsp?startIndex=0"?
If IP address/Port number base, it'll fail nealy always when NAT is used.
(NAT is usually active when router type ADSL/VDSL modem is used, and I use
router type VDSL modem.)

Comment 10

[Daniel Brahneborg]

Maybe interesting note: Even though the dropdown box and clicking on the headers
work fine, pressing Ctrl-R to refresh the pages uses the wrong sort order again,
even though the preselected value in the dropdown is correct.

Comment 11

[Boris Zbarsky [:bzbarsky]]

So could someone give a clear description of the problem here?  The steps in
comment 0 work for me (I see the table get sorted and all) in a current trunk
build...

Comment 12

[Daniel Brahneborg]

Boris: The data get sorted any number of times, not just the first one? Can you
reproduce the bug with version 1.7.5? If so, something has obviously changed
since then.

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
Boris, if the "HTTP session object" of JSP is asumed to be identified by
IP-address/Port-number pair, problem can be explaied as follows.
 (1) Mozilla(IP#1/Port#1) -> Server
     GET /bugzero/servlet/sorting?parent=home.jsp&sort_column=priority
 (2) Application(/bugzero/servlet/sorting) saves sort result
     with ID = IP#1/Port#1 ("HTTP session object")
 (3) Mozilla <- Server
     302 Moved Temporarily
     Location: /bugzero/jsp/home.jsp?startIndex=0
 (4) Mozilla(IP#2/Port#2) -> Server
     GET /bugzero/jsp/home.jsp?startIndex=0
 (5) Application(/bugzero/jsp/home.jsp) search sort result
     by ID = IP#2/Port#2
     (5-A) If NAT is not used,
           IP#1 == IP#2 usualy, and Port#1 == Port#2 in many cases.
           Then sort result("HTTP session object") is found.
           -> sort is done
     (5-B) Even If NAT is not used,
           IP#1 == IP#2 usualy, but Port#1 != Port#2 may occur.
           Then sort result("HTTP session object") is not found.
           -> sort is not done
     (5-C) If NAT is used,
           IP#1 == IP#2 usualy, but Port#1 != Port#2 usually.
           Then sort result("HTTP session object") is not found
           -> sort is not done
My case is probably (5-C) (I use VDSL modem and NAT is used)
Reporter's case in comment #0 sounds (5-C).
bill_hen(application dveloper)'s case in the past sounds (5-A).
Your case sounds (5-A).

I don't believe above is exact scenario, and I don't know on "HTTP session
object" of JSP, but I think similar scenario is applicable to this bug's
problem, because no parameter in URL/no cookie for identification of "HTTP
session object" of JSP.
See [Reference-1] for PHP case.

[Reference-1] PHP's "session-id" and "session variables".
              ( http://www.php.net/manual/en/ref.session.php )
> CVIII. Session Handling Functions
> Introduction
> 
> Session support in PHP consists of a way to preserve certain data across
> subsequent accesses. This enables you to build more customized applications
> and increase the appeal of your web site.
> 
> A visitor accessing your web site is assigned an unique id, the so-called
> session id. This is either stored in a cookie on the user side or is
> propagated in the URL. 
> 
> The session support allows you to register arbitrary numbers of variables
> to be preserved across requests. When a visitor accesses your site, PHP will
> check automatically (if session.auto_start is set to 1) or on your request
> (explicitly through session_start() or implicitly through session_register())
> whether a specific session id has been sent with the request.
> If this is the case, the prior saved environment is recreated. 

Comment 14

[WADA:World Anti-bad-Duping Agency]

(Correction of my comment #9)
Cookie for JSP's "HTTP session object" was sent/received successufuly.
(I removed too many data from HTTP header log when I checked log data before.)
Sorry for spam and mis-description.

Following is HTTP summary of header log
 (1) logoff -> Cookie is changed
 (2) login by guest/gues at login.jsp
     (Cookie is changed correctly)
 (3) Result is returned
 (4) Click link of sort key(Detail colomn)
 (5) Result is returned (But not sorted)
Since cookie(JSESSIONID) is correctly sent/received, passing JSESSIONID to JSP
is done successufully.

What is inhibiting expected result by application in my environment?
Format of Cookie: header data sent by Mozilla?

> ----------------------------------------------------------
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b)
>             Gecko/20050203
> ----------------------------------------------------------
>(1) logoff
> GET /bugzero/servlet/login?action=logoff HTTP/1.1
> Host: www.websina.com
> Referer: http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0
> Cookie: JSESSIONID=F9A59CA5259B33CF15E79E2174300F12
>
> HTTP/1.x 302 Moved Temporarily
> Location: http://www.websina.com/bugzero/jsp/login.jsp
> ----------------------------------------------------------
>(1-A) login screen is retunred
> GET /bugzero/jsp/login.jsp HTTP/1.1
> Host: www.websina.com
> Referer: http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0
> Cookie: JSESSIONID=F9A59CA5259B33CF15E79E2174300F12
>
> HTTP/1.x 200 OK
> Set-Cookie: JSESSIONID=2C346A0260DF0DB678D2F3C1C45B5A3B; Path=/bugzero
> ----------------------------------------------------------
>(2) login by guest/gues at login.jsp
> POST /bugzero/servlet/login HTTP/1.1
> Host: www.websina.com
> Referer: http://www.websina.com/bugzero/jsp/login.jsp
> Cookie: JSESSIONID=2C346A0260DF0DB678D2F3C1C45B5A3B
>
> HTTP/1.x 302 Moved Temporarily
> Location: http://www.websina.com/bugzero/jsp/home.jsp
> ----------------------------------------------------------
>(3) Result is returned
> GET /bugzero/jsp/home.jsp HTTP/1.1
> Host: www.websina.com
>
> Referer: http://www.websina.com/bugzero/jsp/login.jsp
> Cookie: JSESSIONID=2C346A0260DF0DB678D2F3C1C45B5A3B
>
> HTTP/1.x 200 OK
> ----------------------------------------------------------
>(4) Click link of sort key(Detail colomn)
> GET /bugzero/servlet/sorting?parent=home.jsp&sort_column=detail HTTP/1.1
> Host: www.websina.com
> Referer: http://www.websina.com/bugzero/jsp/home.jsp
> Cookie: JSESSIONID=2C346A0260DF0DB678D2F3C1C45B5A3B
>
> HTTP/1.x 302 Moved Temporarily
> Location: http://www.websina.com/bugzero/jsp/home.jsp?startIndex=0
> ----------------------------------------------------------
>(5) Result is returned (But not sorted)
> GET /bugzero/jsp/home.jsp?startIndex=0 HTTP/1.1
> Host: www.websina.com
> Referer: http://www.websina.com/bugzero/jsp/home.jsp
> Cookie: JSESSIONID=2C346A0260DF0DB678D2F3C1C45B5A3B
>
> HTTP/1.x 200 OK
> ----------------------------------------------------------

Comment 15

[Boris Zbarsky [:bzbarsky]]

WADA, this is not a NAT issue (I don't use NAT, and I just managed to reproduce
the problem).  Also, longish HTTP logs would do better as attachments, not as
comments.  This bug is rapidly approaching the "it's impossible to find useful
information in it" point, as things stand.

The problem is present only if cache is set to refetch "every time I view the
page".  If I set it to the default value of "when the page is out of date",
things work fine.

Over to networking, but I'm suspecting that when we do an actual refetch of one
of the pages involved instead of just getting it from cache the server gets very
confused...  It _is_ sending us data (since the timestamp changes), but it's
clearly sending the wrong data.

Comment 16

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #15)
> this is not a NAT issue
> The problem is present only if cache is set to refetch "every time I view
> the page".  If I set it to the default value of "when the page is out of
> date", things work fine.
Comfirmed.
Thanks for clarifying.

From client side view, no cached data is involved in this case, as I say from first.
 (a) Sending data
     - Mozilla merely issues HTTP GET to static <LINK> URL.
 (b) Receiving data
     - Mozilla merely displays returned HTML,
       and timestamp data text in the HTML is updated every time.
     - "304 Not Modified" is returned for data outside of the HTML
       such as <script>, <img> when requested with "If-Modified-Since".

However, from server side view, cached data may affect JSP application
execution, since Mozilla's request on cache depends on cache option.
 (a) "when the page is out of date"    
     No "Cache-Control:" header is sent.
 (b) "every time I view the page" 
     "Cache-Control: max-age=0" is sent on every LINK click
If "Cache-Control: max-age=0" is sent by client, Apache clears cache data for
the client.
This may affect on JSP's session control, JSP's application control, or JSP
application execution.

I think there is nothing Mozilla can do for this bug's problem.
Boris, what do you think?

Comment 17

[Boris Zbarsky [:bzbarsky]]

I don't really know enough about the HTTP specification to say anything useful.
 Hopefully darin will look and comment...

Comment 18

[WADA:World Anti-bad-Duping Agency]

Sounds violation of next rule on HttpSession object, or abuse of JSP objects
other than HttpSession object, which depend on cached data.
http://www-306.ibm.com/software/webservers/appserv/doc/v35/ae/infocenter/was/0404010108.html
> Do not try to save and reuse the HttpSession object outside of each servlet or
> JSP. The HttpSession object is a function of the HttpRequest (you can get it
> only through req.getSession() ), and a copy of it is valid only for the life of
> the service() method of the servlet or JSP. You cannot cache the HttpSession
> object and refer to it outside the scope of a servlet or JSP.

See next site for description on (HTTP) Session Management (WebSphere).
http://publib.boulder.ibm.com/infocenter/ws60help/index.jsp?topic=/com.ibm.websphere.nd.doc/info/ae/ae/tprs_sesi.html

Comment 19

[WADA:World Anti-bad-Duping Agency]

Attachment: HTTP Header [Case-1] "When the page is out of date"

[Case-1] "When the page is out of date"

(1) Change cache option/Clear cache
    - Compare the page in the cache to the page on network
      "When the page is out of date"
    - Clear chace
(2) Remove cookie of www.websina.com JSESSIONID
(3) Go http://www.websina.com/bugzero/jsp/home.jsp
    -> blank login page
(4) login guest/guest
    -> Result returned (Feb 8, 2005 3:37:23 PM)
(5) Click ID#
    -> Result returned (Feb 8, 2005 3:38:39 PM), Sorted
(6) Click Description
    -> Result returned (Feb 8, 2005 3:39:29 PM), Sorted

Comment 20

[WADA:World Anti-bad-Duping Agency]

Attachment: HTTP Header [Case-2] "Every time I view the page"

[Case-2] "Every time I view the page"

(1) Change cache option/Clear cache
    - Compare the page in the cache to the page on network
      "Every time I view the page"
    - Clear chace
(2) Remove cookie of www.websina.com JSESSIONID
(3) Go http://www.websina.com/bugzero/jsp/home.jsp
    -> blank login page
(4) login guest/guest
    -> Result returned (Feb 8, 2005 3:49:48 PM)
(5) Click ID#
    -> Result returned (Feb 8, 2005 3:50:52 PM), Sorted
(6) Click Description
    -> Result returned (Feb 8, 2005 3:51:34 PM), Not Sorted
(7) Click ID# again
    -> Result returned (Feb 8, 2005 3:52:01 PM), Not Sorted
(8) Click Description again
    -> Result returned (Feb 8, 2005 3:53:13 PM), Not Sorted

Comment 21

[WADA:World Anti-bad-Duping Agency]

When [Case-2] Step (5), "Cache-Control: max-age=0" is NOT sent, because first
access to the LINK target(/bugzero/servlet/sorting).
This is the next by reporter ( comment #12 )
> Boris: The data get sorted any number of times, not just the first one?

Comment 22

[kimdvm]

When I load new pages for the website I administer and hit reload in Firefox
(Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107
Firefox/1.0) it continues to use the cached page. Even after exiting the browser
completely and restarting. Only manually clearing the cache allowed the new page
to load. I was able to reproduce this in Netscape 7.1 until I changed the
setting to "Every time I view the page"

Comment 23

[Matthias Versen [:Matti]]

for comment#22:
Realod will reload from the cache if the server reports that the image is
unchanged. You can use "shift+reload" to force a reloading over the network.

Comment 24

[Tiago Freire]

Currently using Firefox 1.5 on Linux.
I am programming an application in php and I have set the following headers on Apache:

  Header append Cache-Control: "max-age=1, no-cache, no-store, must-revalidate"

Nevertheless, if I use a simple DOM javascript to insert a SCRIPT element on the HEADER of the page, it will always return the same file (cache). I can edit teh file and reload the page, but the old script will execute. I checked this behavior by adding and commenting a simple alert('fubar'); line on the script. Other js files existing on the html file are reloaded normally.
If I type the  url of the javascript file, the updated file will be displayed normally, but trying to use it will still return the old, cached file, until the browser is closed and opened again.

To make matters worse, it only happened while loading throught the web server (https), but when I opened the files locally, i could not reproduce this problem.

ok.js:
------------
//Changes in this file are seen normally
//In other words, this file is not unduly cached.
//It reloads every time.
var e = document.createElement('script');
e.setAttribute('type','text/javascript');
e.setAttribute('src','bugger.js');
document.getElementsByTagName('head')[0].appendChild(e);
------------

bugger.js:
------------
//you may comment and uncomment the alert, or change the message,
//but the browser will load the file only once.
//You will have to close and open the browser to see the differences.
alert('fubar');
------------

html:
------------
<html>
<head>
<script type="text/javascript" src="ok.js">
<head>
<body>
<p>Hello, bug!</p>
</body>
</html>
------------

Comment 25

[Tiago Freire]

Just to complete: It happens on http access as well as https, but not with file access. 

Comment 26

[Christian :Biesinger (don't email me, ping me on IRC)]

(In reply to comment #24)
>   Header append Cache-Control: "max-age=1, no-cache, no-store, must-revalidate"

you need to set that header on the file that you don't want cached (that's bugger.js in your example, I think)

Comment 27

[WADA:World Anti-bad-Duping Agency]

Changing summary, because no client side cached data is involved in this bug's issue, and in order to avoid confusion like comment #22 or comment #24.

I think this bug's Product is better to be changed to Tech Evang.

Comment 28

[Matthias Versen [:Matti]]

Is this still an issue ?