Open Bug 1305624 Opened 8 years ago Updated 3 years ago

Data manager stops responding upon being opened

Categories

(SeaMonkey :: General, defect)

Product:
SeaMonkey
Component:
General
Version:
SeaMonkey 2.46 Branch
Platform:
Unspecified
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

People

(Reporter: rexyrexy2, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46
Build ID: 20160923004119

Steps to reproduce:

Opened Data manager, either windowed or tabbed.


Actual results:

SeaMonkey stops responding, "chrome://communicator/content/dataman/dataman.js:805"


Expected results:

It should not have stop responding.
NOT reproducible with unofficial (akalla) English SeaMonkey 2.46  (Windows NT 6.1; WOW64; rv:49.0)  Gecko/20100101 Firefox/49.0 Build 20160923004119  (Default Classic Theme)  on German WIN7 64bit:

0. ˋdownload Installer from <https://l10n.mozilla-community.org/~akalla/unofficial/seamonkey/nightly/latest-comm-release-windows32/> → Unzipˊ
1. Launch SeaMonkey 2.46 via Profile Manager with a newly created User Profile
   (or create new Profile via 'Tools → Switch Profile' after normal launch)
   » Default SeaMonkey project page appears in Browser
2. Menu  ˋTools → Data Managerˊ
   » Data Manager appears
     no crash / hang / any other problem

@reporter:
a) your result with newly created profile?
b) your result in safe mode without active add-ons?
Flags: needinfo?(rexyrexy2)
Please add the bool variable data_manager.debug and set it to true. Open the web or error console and check if you can spot any unusual debug message when opening Data Manager.
>> this.displayedDomains.push(this.domainObjects[domain]);

It just goes over the domains in a loop. Could you check your permissions.sqlite and see how many rows are in it. Not accurate because storage and passwords are not in this db but an indication.
I got this on browser console:

TypeError: chromeWindow.gBrowser is undefined[Learn More]devtools-browser.js:489:7



Error: Script terminated by timeout at:
domain_sort@chrome://communicator/content/dataman/dataman.js:779:5
domain_search@chrome://communicator/content/dataman/dataman.js:807:5
domain_addDomainOrFlag@chrome://communicator/content/dataman/dataman.js:606:9
loader@chrome://communicator/content/dataman/dataman.js:355:9
nextStep@chrome://communicator/content/dataman/dataman.js:277:7


and:

16:52:38.843 IndexedDB Unable to open IndexedDB database, schema is too high!: ActorsParent.cpp:44991(unknown)

16:52:38.851 UnknownError1indexed-db.js:59:9

16:52:38.852 undefined1Promise-backend.js:940




(in safe mode)



After stopping it, on ERROR console, I get this after I hit the debug button:

Timestamp: 10/2/2016 4:48:11 PM
Error: TypeError: chromeWindow.gBrowser is undefined
Source File: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/client/framework/devtools-browser.js
Line: 489


And always this:

Timestamp: 10/2/2016 4:50:38 PM
Warning: Error: Script terminated by timeout at:
storage_loadList@chrome://communicator/content/dataman/dataman.js:2678:25
loader@chrome://communicator/content/dataman/dataman.js:353:7
nextStep@chrome://communicator/content/dataman/dataman.js:277:7

Source File: chrome://communicator/content/dataman/dataman.js
Line: 2678





Also, in the error and browser console it keeps saying Loaded, cached, added domain, found pref, etc, forever if I keep letting it run.
>>
16:52:38.843 IndexedDB Unable to open IndexedDB database, schema is too high!: ActorsParent.cpp:44991(unknown)
<<

This is likely the problem. You have a corrupt profile or rather a profile which was upgraded by a more recent SeaMonkey. Did you downlevel the version eg from a1 to a2 or to beta lately?

Delete the storage directory in your profile and see if the error goes away. Usually harmless but better make a backup before.

FRG
With my existing work profile the data manager was very slow under Windows. After deleting

storage/*.*, 
content-prefs.sqlite, 
cookies.sqlite, 
formhistory.sqlite, 
permissions.sqlite, 
places.sqlite, 
storage.sqlite and 
webappsstore.sqlite 

Seamonkey was working good again. Under Linux no problems even in used profiles.
This has been reported now a few times already. It seems that the fixed calls to the storage manager impact performance and mostly cause this. In this cases just renaming or deleting webappsstore.sqlite fixes it.

This is directly linked to cookie settings. If you only accept session cookies you won't see the problem because the storage is deleted between sessions.

Problem is higher to occur if you profile is very very old. The accumulated cruft takes its toll here. Depending on size permissions.sqlite and storage.sqlite might also cause it.

To fully fix it Data Manager would probably need a complete rewrite to do async operations.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(rexyrexy2)
Summary: Data manger stops responding upon being opened, dataman.js line 805 → Data manger stops responding upon being opened
>> 16:52:38.843 IndexedDB Unable to open IndexedDB database, schema is too high!: 
>> ActorsParent.cpp:44991(unknown)

Just to be clear. If you see these errors in the log as originally reported your profile is corrupted or not compatible with the used SeaMonkey version. In these cases either restore a backup or at least delete the storage directory, storage.sqlite and webappsstore.sqlite from your profile.
(In reply to Frank-Rainer Grahl from comment #7)
> This has been reported now a few times already. It seems that the fixed
> calls to the storage manager impact performance and mostly cause this. In
> this cases just renaming or deleting webappsstore.sqlite fixes it.

I am also experiencing this problem, using Mozilla/5.0 (X11; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46.  Deleting webappsstore.sqlite worked around the issue for me.
> Deleting webappsstore.sqlite worked around the issue for me.

Thanks for confirming. I looked at the code and didn't find anything wrong with it. So far I was unable to reproduce the problem. Can you tell me how bug your webappsstore.sqlite was?
(In reply to Frank-Rainer Grahl from comment #10)
> Thanks for confirming. I looked at the code and didn't find anything wrong
> with it. So far I was unable to reproduce the problem. Can you tell me how
> bug your webappsstore.sqlite was?

Before I deleted it (or rather, moved it out of the way), it was 112 MB in size.  Now after using my browser for a few hours, it's 32K.
I had the same problem on the unofficial akalla build of SeaMonkey 2.47 on Windows 7 (Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0 SeaMonkey/2.47).  Renaming webappsstore.sqlite to webappsstore.bak fixed the problem.  Previously, it was 94MB.
hello, i have the same bug on win7 x64 with 
User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46
Build identifier: 20161213183751


i have a very old profil since more than 15years i think, so i guess there is something to update.
it wasn't append in the previous version. i guess my old profile must contain a lot of information, fields to update and check...
Summary: Data manger stops responding upon being opened → Data manager stops responding upon being opened
OS: Due to comments 0 and 9
OS: Unspecified → All
someone said "Summary: Data manger stops responding upon being opened → Data manager stops responding upon being opened"

it is wrong, in fact it stops and produces this error "Script: chrome://communicator/content/dataman/dataman.js:779"  after you entered your password in the dialog box.

if you close the dialog box requesting your password , then the datamanager runs fine (but you can"t see the private infos off course).

the bug appears once you validated you password from the dialog box.
> // Do the actual sorting of the array.
> this.displayedDomains.sort(compfunc);

Line 779 is a sort call which should not cause problems. Its likely that the script just runs too long. 

> if you close the dialog box requesting your password

What password for what? Master password for password manager?
(In reply to Frank-Rainer Grahl from comment #16)
> > // Do the actual sorting of the array.
> > this.displayedDomains.sort(compfunc);
> 
> Line 779 is a sort call which should not cause problems. Its likely that the
> script just runs too long. 
> 
> > if you close the dialog box requesting your password
> 
> What password for what? Master password for password manager?

yes, the master password .
> yes, the master password .

I am unable to reproduce the problem with just setting master password. The first time it is needed a dialog box will appear. Data Manager will just continue to show the content afterwards. This is with 2.50a2. Need to check with a lower version because password permissions are stored differently in 2.48 and up but I suspect you have a lot of junk in your permissions db or storage db. 

I would try the above workaround first:

> Renaming webappsstore.sqlite to webappsstore.bak fixed the problem.
Hello, everyone

I can confirm the problem and add a few detail: upon opening the data manager stops responding rather quickly. After a while there appears a warning about an unresponsive script (Script: chrome://communicator/content/dataman/dataman.js:779)
If I press "Continue" it apperars again and again and after three times the data manager finally starts and does its job.
I tried the suggested workaround and renamed webappsstore.sqlite to webappsstore.sqlite.bak which fixed the problem.

I'm using:
User agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46
Build identifier: 20161213183751

In fact the problem appeared earlier, after updating to Seamonkey 2.40 or 2.41 (can't remember exact number). And yes, my profile is pretty old (about 5-10 years or even older)
and yes, old webappsstore.sqlite is about 43 MB
the bug is still here, there i mean.
my profil is very old too, between 15 and 20 years i think.
if you have the way to provide this info, i'll be interested in ! it might be written somewhere in the data i guess, isn't it ?
i just checked, my webappsstore.sqlite is 230MB
webappsstore.sqlite of 33MB caused password manager to populate very slowly on open; removing the file fixed it.
(In reply to Steve Wendt from comment #23)
> webappsstore.sqlite of 33MB caused password manager to populate very slowly
> on open; removing the file fixed it.

hey, deleting this fiel to solve a bug IS NOT a proper way to manage it !

by deleting this file, i will lost some data according to the faq :
http://kb.mozillazine.org/Webappsstore.sqlite

[quote]Deleting
Deleting webappsstore.sqlite will delete any data web sites have stored there. A new file will be created when it is needed. 
[/quote]

i have to say my computer hardware is high ( 6-cores at 3.33GHz and 12MB of memory RAM), and seamonkey starts to bug only in 4-5 seconds ! so this is not a ressources issues due to overload, but a real bug like a overflow or something like that!

now, many months passed still i need this bug be solved because i can't access this part of seamonkey.
my webappsstore.sqlite is 230MB which is correct because my profil age is more than 10years i think (maybe since Y2005)

at least, shall you advice me a free too to edit this database and clean all of the old data only ?

there is a feature i wish in seamonkey is to clear data from domain i didnot visited since more than 5 year.

or a feature to order/sort the domain by capacity data stored to know which domain use so much data on my computer.
maybe it is a very old one.

there is a history feature in seamonkey which is not able to delete the old cookies...but only the history infos.

i dream of a tool to order/sort (and clean) my stored data in this webappsstore.sqlite !
The bug isn't fixed or it would be closed. It can't be fixed because I can't reproduce it and I tried with very old versions.

>> by deleting this file, i will lost some data according to the faq :

Usually you will loose nothing because the data will be recreated when you visit the web site. In 99% of all cases its just junk to track you anyway. Not sure if Flash stores some data and keys here so just back it up and try. I think its corrupt already or the Data Manager would open just fine.
Clear private data can clean all cookies but you can clean all your old cookies with the old integrated cookie viewer too:

chrome://communicator/content/permissions/cookieViewer.xul

Use it all the time and fixed it so that it still works. Complements the Data Managerr very well.
(In reply to Frank-Rainer Grahl from comment #27)
> Clear private data can clean all cookies but you can clean all your old
> cookies with the old integrated cookie viewer too:
> 
> chrome://communicator/content/permissions/cookieViewer.xul
> 
> Use it all the time and fixed it so that it still works. Complements the
> Data Managerr very well.

i tried it, it works but it is useless for me because it won't order the cookie by create date, only by end date.

is it normal to have search year (expires : dimanche 5 août 16210 13:05:24) in cookie ? it is like unlimited date cookies.

the year  16210  !!


[quote]Usually you will loose nothing because the data will be recreated[/quote]

i just want to keep my password and login. The other things, i don't care. so i can't delete this sqlite file. i would lost some precious information.
Apparently the problem afflicts Firefox as well (although the symptoms obviously are different, due to data manager not being present there):
https://www.reddit.com/r/firefox/comments/40gbos/psa_large_webappsstoresqlite_file_in_your_profile/

(In reply to Michael REMY from comment #28)
> > Clear private data can clean all cookies

cookies.sqlite contains those - different file.

> i just want to keep my password and login.

Those are in logins.json (previously signons.sqlite) - another different file.

> so i can't delete this sqlite file. i would lost some precious information.

I believe webappsstore.sqlite is just HTML5 offline storage.  I am assuming it caches texture data for WebGL games and such; I doubt it has any "precious information."  But rename it / move it somewhere else, instead of deleting it, so you can get it back if you feel the need.  There are a variety of SQLite editors around, if you want to start poking at the actual contents.
Read these old bugs and weep:
https://bugzilla.mozilla.org/show_bug.cgi?id=857888
https://bugzilla.mozilla.org/show_bug.cgi?id=1238771
https://bugzilla.mozilla.org/show_bug.cgi?id=742822
https://bugzilla.mozilla.org/show_bug.cgi?id=1286798

TL;DR - this problem has been around in Firefox for 5 years, and is most likely afflicting SeaMonkey recently due to having to use more of the Core / Toolkit code (whatever they call it these days).
well, it's been one week since i did what you adviced me : backup the monstruous 250MB webappsstore.sqlite file and delete it.

after one week, i lost nothing i cannot detect (no password, no session, no form suggest data content) a loss at all.
The data manager is back alive.
the actual size is now only 5MB.
I think the the problem is not the size, but the amount of entries. I inspected the sqlite-files with a SQLite Browser like http://sqlitebrowser.org/

- permissions.sqlite (4 MB, 60k entries)
Deleting the file did not help, it was recreated with the same amount of entries. So I deleted the entries via the following commands:
DELETE FROM moz_perms;
VACUUM;

- webappsstore.sqlite (100 MB, 23k entries)
Find biggest entries: SELECT length(value) AS size,scope,key FROM webappsstore2 ORDER BY size DESC LIMIT 100
Find biggest keys: SELECT SUBSTR(key,1,15) AS shortkey,SUM(LENGTH(value)) AS size FROM webappsstore2 GROUP BY shortkey ORDER BY size DESC
Find frequent keys: SELECT SUBSTR(key,1,15) AS shortkey,COUNT(*) AS amount FROM webappsstore2 GROUP BY shortkey ORDER BY amount DESC
Find keys from frequent domains: SELECT originKey,COUNT(*) AS amount FROM webappsstore2 GROUP BY originKey ORDER BY amount DESC
My biggest keys: SELECT SUM(LENGTH(value)) AS key FROM webappsstore2 WHERE key REGEXP "^(CLOUDFLARE|Hyphenator|MediaWikiModuleStore|font_css_|google_|_kxadaudience|kxadaudience|optimizely|at-rand|_chartbeat|axd)";
Delete biggest keys and compress:
DELETE FROM webappsstore2 WHERE key REGEXP "^(CLOUDFLARE|Hyphenator|MediaWikiModuleStore|font_css_|google_|_kxadaudience|kxadaudience|optimizely|at-rand|_chartbeat|axd)";
VACUUM;

it is happening again..
so i'm deleting the webappsstore.sqlite which is only 45MB this time.

then after re-deteling the webappsstore.sqlite , it works fine.

why does it happen ?

You need to log in before you can comment on or make changes to this bug.