Closed Bug 21313 Opened 25 years ago Closed 24 years ago

Need to cache document charset

Categories

(Core :: Networking: Cache, defect, P1)

Product:
Core
Component:
Networking: Cache
Type:
defect

Tracking

()

Status:
VERIFIED FIXED

People

(Reporter: bobj, Assigned: ftang)

References

Details

(Keywords: perf, Whiteboard: [nsbeta2-][nsbeta3+], fix in hand. need review.) 

Bob Jung wrote:

  Do you have a spec or some sorta doc on your cache work?
  For I18N, we would like to cache the charset last used to render a page.
  We talked about it, but never did this in previous (4.x and before) browsers.

There is a very detailed description of the cache API, in the form of the
commented IDL files (attached).  However, the information that you're
talking about (charset) is at the protocol-level or above, which is at a
higher level than the the cache manager deals with.  It is possible to
store opaque information in each cache entry to be associated with a URL
using the setProtocolPrivate() API.  It sounds like this is what you
would want to do.

-Scott

See:
http://lxr.mozilla.org/seamonkey/source/netwerk/cache/public/nsINetDataCacheMana
ger.idl
http://lxr.mozilla.org/seamonkey/source/netwerk/cache/public/nsICachedNetData.id
l
Status: NEW → ASSIGNED
See bug 20939, which also addresses the issue of storing URL metadata in the
cache.
Note that the cache method name was changed to SetAnnotation() from
SetProtocolPrivate().
Bulk move of all Cache (to be deleted component) bugs to new Networking: Cache
component.
Target Milestone: M14
See bug 20939 for an attachment which contains
    "code to store image dimensions in cache (untested)"

We should be able to study that for the implementing charset caching.
Added beta1 in keywords.
Keywords: beta1
Keywords: beta1
Reassigned to jbetak for M15.
Assignee: bobj → jbetak
Status: ASSIGNED → NEW
Target Milestone: M14 → M15
Status: NEW → ASSIGNED
Keywords: beta2
Target Milestone: M15 → M16
Keywords: nsbeta2
Putting on [nsbeta2+][5/16] radar.  This is a feature MUST complete work by 
05/16 or we may pull this feature for PR2.
Whiteboard: [nsbeta2+][5/16]
Added perf keyword
Keywords: perf
Putting on [nsbeta2-] radar. Missed the Netscape 6 feature train.  Please set to 
MFuture.
Whiteboard: [nsbeta2+][5/16] → [nsbeta2-]
Target Milestone: M16 → Future
This is really easy to do - it pretty much maps whatever we did for the 
bookmarks. The only difference is that we do not need to change the cache code 
to achive document charset persistency in the cache. The cache service provides 
a neat interface (getAnnotation, setAnnotation) for opaque attributes 
associated with a URL.

http://lxr.mozilla.org/seamonkey/source/netwerk/cache/public/nsICachedNetData.id
l#162

All we need to do is to get the cache service in nsHTMLDocument.cpp (look how 
we are getting the bookmarks service), use the document URL as key to access 
the stored doument charset. Once a new document charset has been determined in 
nsHTMLDocument.cpp we need to store it using the URL as key. That's pretty much 
it. I already included a new member in the enum struct in nsIParser.h to 
reflect the neccessary priority for the charset determination algorithm in 
nsHTMLDocument.cpp.
Assignee: jbetak → ftang
Status: ASSIGNED → NEW
Blocks: 41047
performance issue. mark nsbeta3
Status: NEW → ASSIGNED
Keywords: nsbeta3
set it to P1 M18
Priority: P3 → P1
Target Milestone: Future → M18
nsbeta3+ per i18n bug meeting. this may also help 45408
Whiteboard: [nsbeta2-] → [nsbeta2-][nsbeta3+]
Whiteboard: [nsbeta2-][nsbeta3+] → [nsbeta2-][nsbeta3+]ETA:8/11
take out ETA.
I probably can now work on this since other important problem are all waiting 
for some other stuff.
Whiteboard: [nsbeta2-][nsbeta3+]ETA:8/11 → [nsbeta2-][nsbeta3+]
Here are some example how to get the nsINetDataCacheManager and nsICachedNetData

from 
http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/jar/src/nsJARChannel.cp
p#383
381     nsCOMPtr<nsINetDataCacheManager> cacheMgr;
382     nsXPIDLCString jarBaseSpec;
383     nsCOMPtr<nsICachedNetData> cachedData;

423     cacheMgr = do_GetService(NS_NETWORK_CACHE_MANAGER_PROGID, &rv);
424     if (NS_FAILED(rv)) goto error;
425 
426     rv = mJARBaseURI->GetSpec(getter_Copies(jarBaseSpec));
427     if (NS_FAILED(rv)) goto error;
428 
429     rv = cacheMgr->GetCachedNetData(jarBaseSpec, nsnull, 0, 
nsINetDataCacheManager::CACHE_AS_FILE,
430                                     getter_AddRefs(cachedData));
Here is the patch http://warp/u/ftang/tmp/fix21313.txt

Index: src/nsHTMLDocument.cpp
===================================================================
RCS file: /m/pub/mozilla/layout/html/document/src/nsHTMLDocument.cpp,v
retrieving revision 3.263
diff -u -r3.263 nsHTMLDocument.cpp
--- nsHTMLDocument.cpp  2000/08/18 00:47:00     3.263
+++ nsHTMLDocument.cpp  2000/08/18 08:09:05
@@ -103,6 +103,8 @@
 #include "nsIDocumentCharsetInfo.h"
 #include "nsIDocumentEncoder.h" //for outputting selection
 #include "nsIBookmarksService.h"
+#include "nsINetDataCacheManager.h"
+#include "nsICachedNetData.h"
 #include "nsIXMLContent.h" //for createelementNS
 #include "nsHTMLParts.h" //for createelementNS
 
@@ -486,6 +488,7 @@
     }
   }
 
+  nsCOMPtr<nsICachedNetData> cachedData;
   nsresult rv = nsDocument::StartDocumentLoad(aCommand,
                                               aChannel, aLoadGroup,
                                               aContainer,
@@ -501,6 +504,10 @@
   
   nsAutoString lastModified;
   nsCOMPtr<nsIHTTPChannel> httpChannel = do_QueryInterface(aChannel);
+
+  PRBool bTryCache = PR_FALSE;
+  PRUint32 cacheFlags = 0;
+
   if (httpChannel) 
   {
     nsXPIDLCString lastModHeader;
@@ -572,6 +579,22 @@
       }
     }
 
+    if(kCharsetFromCache > charsetSource )
+    {
+      PRUint32 loadAttr = 0;
+      rv = httpChannel->GetLoadAttributes(&loadAttr);
+      NS_ASSERTION(NS_SUCCEEDED(rv),"cannot get load attribute");
+      if(NS_SUCCEEDED(rv) )
+      {
+         // copy from nsHTTPChannel.cpp
+         if(loadAttr & nsIChannel::CACHE_AS_FILE)
+             cacheFlags = nsINetDataCacheManager::CACHE_AS_FILE;
+         else if(loadAttr & nsIChannel::INHIBIT_PERSISTENT_CACHING)
+             cacheFlags = nsINetDataCacheManager::BYPASS_PERSISTENT_CACHE;
+         bTryCache = PR_TRUE;
+      }
+    }
+
     // Don't propogate the result code beyond here, since it
     // could just be that the response header wasn't found.
     rv = NS_OK;
@@ -778,6 +801,9 @@
     nsXPIDLCString scheme;
     aURL->GetScheme(getter_Copies(scheme));
 
+    nsXPIDLCString urlSpec;
+    aURL->GetSpec(getter_Copies(urlSpec));
+
     if (scheme && nsCRT::strcasecmp("about", scheme) && (kCharsetFromBookmarks 
> charsetSource))
     {
       nsCOMPtr<nsIRDFDataSource>  datasource;
@@ -786,8 +812,6 @@
            nsCOMPtr<nsIBookmarksService>   bookmarks = 
do_QueryInterface(datasource);
            if (bookmarks)
            {
-                 nsXPIDLCString urlSpec;
-                 aURL->GetSpec(getter_Copies(urlSpec));
 
                  if (urlSpec)
                  {
@@ -805,6 +829,34 @@
        }
     } 
 
+    if(bTryCache)
+    {
+       nsCOMPtr<nsINetDataCacheManager> cacheMgr;
+       cacheMgr = do_GetService(NS_NETWORK_CACHE_MANAGER_PROGID, &rv);       
+       NS_ASSERTION(NS_SUCCEEDED(rv),"Cannot get cache mgr");
+       if(NS_SUCCEEDED(rv) && urlSpec)
+       {
+          rv = cacheMgr->GetCachedNetData(urlSpec, nsnull, 0, cacheFlags, 
getter_AddRefs(cachedData));
+          NS_ASSERTION(NS_SUCCEEDED(rv),"Cannot get cached net data");
+          if(NS_SUCCEEDED(rv)) {
+              bTryCache = PR_TRUE; // since we can get cachedData;
+              if(kCharsetFromCache > charsetSource) 
+              {
+                nsXPIDLCString cachedCharset;
+                PRUint32 cachedCharsetLen = 0;
+                rv = cachedData->GetAnnotation( "charset", &cachedCharsetLen, 
+                      getter_Copies(cachedCharset));
+                if(NS_SUCCEEDED(rv) && (cachedCharsetLen > 0)) 
+                {
+                   charset.AssignWithConversion(cachedCharset);
+                   charsetSource = kCharsetFromCache;
+                }
+              }    
+          }
+       }
+       rv=NS_OK;
+    }
+
     if (kCharsetFromParentFrame > charsetSource) {
       if (dcInfo) {
         nsCOMPtr<nsIAtom> csAtom;
@@ -856,6 +908,13 @@
 
   rv = this->SetDocumentCharacterSet(charset);
   if (NS_FAILED(rv)) { return rv; }
+
+  if(bTryCache)
+  {
+       rv=cachedData->SetAnnotation("charset",charset.Length()+1,
+               NS_ConvertUCS2toUTF8(charset.GetUnicode()));
+       NS_ASSERTION(NS_SUCCEEDED(rv),"cannot SetAnnotation");
+  }
 
   // Set the parser as the stream listener for the document loader...
   if (mParser)
Whiteboard: [nsbeta2-][nsbeta3+] → [nsbeta2-][nsbeta3+], fix in hand. need review.
fix and check in.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago
Resolution: --- → FIXED
verif.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.