I've got a mail folder called "Local Folders/Old Mail/ÖVP" which seems to be the source of a problem I ran into with localstore.rdf - it grew up to over 16 MB with multiple calls of mailnews! The following are lines I extracted from a 2 MB version of localstore.rdf: <RDF:Description about="chrome://messenger/content/messenger.xul"> <NC:persist resource="mailbox://nobody@Local%20Folders"/> <NC:persist resource="mailbox://nobody@Local%20Folders/Old Mail"/> <NC:persist resource="mailbox://nobody@Local%20Folders/Old Mail/Ã?Â?Ã?Â[...]?Â? Ã?Â?VP"/> where [...] is to be replaced by about 250K of that "Ã?Â?Ã?Â?Â?Ã?Â" crap. I'll attach a 31K zip of that 2M version of localstore.rdf Seen with multiple nightly M16 builds, last of them was 2000052615 (win32). Note that Mozilla is able to start up and work properly with that localstore.rdf files, it only takes ages to launch.
reassign to putterman I haven't seen this problem, working fine before 2000052615?
Assignee: nhotta → putterman
That did happen with a few builds for me after M15... could also be that it's generated if I want to use the right-click popup menuitem "Move To" to move a message from Inbox to that folder, or if I try to do the same via d&d. That's the only actions I've done to that folder (or its subfolders) afaik.
Can you try starting with a new profile and seeing if this happens on a current build? Things that make a folder appear in localstore.rdf include changing to threaded mode, changing the sort, or opening the folder if it has subfolders.
putterman, yes, it's still in recent builds. Verified again with 2000-05-29-08. I now realized that's it's not going from almost zero to 16M in one step... If it gests into localstore.rdf once (I made a "ÖVP" folder with two subfolders and viewed them), the crappy code doubles (or in some other form multiplies) with every call of Mozilla, even browser! So once it's in, it grows and grows almost "forever"...
I created the same folder that you listed below and I can't seem to reproduce this. cc'ing waterson in case he has any ideas. cc'ing momoi to see if he can reproduce this. If this is easily reproduced on I18N machines then this should probably be an nsbeta2 bug. But I'd like to find that out before doing that.
OK, tried again in 2000060308-win32. Exact steps to reproduce: 1) start Mozilla -ProfileManager 2) Select "Create Profile" and create a nerw Profile (used the default settings) 3) Launch Mozilla 4) Open up Mozilla Mail 5) create an existing Mail account (used "foo" settings) 6) click the twisty to open up "Local Mail" folders in sidebar 7) right-click on "Local Mail", selcet "New Folder" 8) Enter "ÖVP" as folder name (be sure you enter "Ö" <iso-8559-1: #214>) 9) Click on the new folder in sidebar 10) Quit Mozilla 11) View localstore.rdf and see "<NC:persist resource="mailbox://nobody@Local%20Folders/Ã–VP"/>" somewhere in the file 12) Launch Mozilla again with that Profile (it's enough to launch Browser) 13) Quit Mozilla again 14) View localstore.rdf again and see the line now telling "<NC:persist resource="mailbox://nobody@Local%20Folders/ÃƒÂƒÃ‚Â–VP"/>" it's growing everytime you launch Mozilla again.
this is a problem with nsLocalStore or the RDF/XML serializer. stealing from putterman.
Assignee: putterman → waterson
Product: MailNews → Browser
Status: NEW → ASSIGNED
Component: Internationalization → RDF
Target Milestone: --- → M18
This issue ocuurs "file://" of IE Import Favorties other than mailbox, too. Chris, is charset of localstore.rdf UTF-8??
Hmm. I don't know if there is *any* encoding set for this file. See http://lxr.mozilla.org/mozilla/source/rdf/datasource/src/nsLocalStore.cpp#355 which is the code that first creates the file. That said, shouldn't the file be read and written using some sane encoding? I'm suspicious that there may be other code that's not I18n-friendly along the chain that writes out the file...
This problem is the following line. http://lxr.mozilla.org/seamonkey/source/rdf/base/src/nsRDFXMLDataSource.cpp#1303 http://lxr.mozilla.org/seamonkey/source/rdf/base/src/nsRDFXMLDataSource.cpp#1394 But I don't know good fix code... And uri of mailbox will change by bug 39757.
makato: what do you think the problem is?
Hi, can we do something about this bug soon? It's preventing my work. In the last 2-3 days, I had to trash 2 profiles for Local Folder testing because the memomry consumption gets to be 100% of my 256MB RAM! I see that my localstroe.rdf file is now 36 MB. I'm designatingt his as a dogfood bug because it greatly hamper my local folder testing work.
Putting on [dogfood+] radar.
I am going to nominate this for nsbeta2 because it is happening to me all too often when I am doing international test involving Japanese, Latin 1 or UTF-8 messages. I didn't realize until the last several days that what was causing interminable slowdown such as waiting for a window to open for nearly 5 minutes was due to this bug. According to Scott P., growth of localstore.rdf could be due to changing to threaded mode, changing sort order, and opening sub-folders, etc. These are pretty common activities. I would like to know what other activities could be contributing to this growth because I am not necessarily doing these things but yet managed to grow localstore.rdf to 15MB or more in less than 2 hours of normal use. This slows down all other applications on my 256MB RAM Windows to a point I cannot do anything but terminate Mozilla before doing anything else. I don't think average users will ever figure out what is wrong and probably stop using this product immediately. I don't think we can ship PR2 with this bug.
RDF (like XML) by default is encoded in UTF-8 (unless there is a Unicode Byte Order Mark at the begining of the file then it is UTF-16), so the localestore.rdf should be encoded in UTF-8. By convention, all the internal Seamonkey XML, XUL, and RDF should be in UTF-8. The original string, Ã–VP, looks like correct UTF-8 for "ÖVP", if rendered as Latin1. The expansion looks suspiciously like the UTF-8 string was mistakenly run through a Latin1-to-UTF8 converter. So I'd suggest looking at how we are converting the strings.
Putting on [dogfood+][nsbeta2+] radar.
Whiteboard: [dogfood+] → [dogfood+][nsbeta2+]
Created attachment 11623 [details] [diff] [review] use appropriate string conversion routines on outbound text
cc'ing rjc for some r= lovin'. rjc: see the attached patch. I've changed all the stuff that assumed that URLs were encoded in ASCII to assume that URLs are encoded in UTF-8. The problem was that we would take a UTF-8 encoded value, widen the bytes "as is" to UCS-2 (or UTF-16, or whatever) without decoding it, and then write it out. Doing this over and over would basically double the number of characters in a URL for each write. And that gets to be big pretty quick!
Whiteboard: [dogfood+][nsbeta2+] → [dogfood+][nsbeta2+] FIX IN HAND
Looks good. r=rjc
fix checked in
Status: ASSIGNED → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → FIXED
OK. I have been using the same environment which caused a huge bloat in the past doing the same sorts of things which led to the slowdwon problem. I have not run into this problem after 10 days or so. So as imprecise as this would seem, in the absence of precise steps, I think this is good enough. Marking it verified as fixed.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.