231941 - Optimize panacea.dat file on MailNews shutdown

Status: RESOLVED DUPLICATE of bug 267345

(MailNews Core :: Database, defect)

Description

[:aceman]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031216 Firebird/0.7+
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031216 Firebird/0.7+

The panacea.dat file can contain references to mail folders and other data
deleted long ago, which no longer exist. The file and program work correctly in
spite of this. Anyway, cleaning the file would be useful. It would make it
smaller, thus faster. And it would show relevant data to those who like to look
into it.

Reproducible: Always

Steps to Reproduce:

Comment 1

[David :Bienvenu]

yeah, we should be doing a compress commit if possible.

Comment 2

[Ross Kendall]

>>The file and program work correctly in spite of this

Actually, I would be bold enough to suggest that the file and program do not
work properly, and that this should be a bug not an enhancement request.

I seem to have noticed on occasion that the references to old folders does cause
problems. (try importing messages from M$ Outlook on windows, restarting, and
then trying to do it again).

Could these be related?
<a href="http://bugzilla.mozilla.org/show_bug.cgi?id=46365">bug 46365</a>
<a href="http://bugzilla.mozilla.org/show_bug.cgi?id=65303#c12">bug 65303</a>

Anyway.

Comment 3

[David :Bienvenu]

It turns out we are doing a compress commit of the db. So the entries in
panacea.dat exist in the internal folder hierarchy datastructures at shutdown
time...So the problem is not with panacea.dat but with the datastructures in memory.

Comment 4

[:aceman]

David, yes, because they are always loaded from the file again. The problem then
is, the datastructures should have been optimized at the moment the folders were
deleted. That bug 65303 (comment 12) may really have something related. Or the
same cause.

Comment 5

[:aceman]

And to make it more interesting, the same problem applies to other files using
the mork db. I can see email addresses and names in the abook.mab file which I
have deleted year ago and there are no visible cards which would contain them.

Comment 6

[Ross Kendall]

Would it be of any use for me to do some testing around the problem?  I am happy
to try various scenarios which may dig up some useful information.  (can try on
windows, linux, and mac)

Comment 7

[:aceman]

I tried to delete all the cards in the addressbook. After closing Mozilla, all
the data was still in the file (abook.mab). Of course, the cards were no longer
visible. I tries to create cards with the same names, but no problem was found.

My testing may be skewed by the fact that this profile was created with 1.3 and
migrated through all versions to 1.7. Maybe 1.3 wrote something bad in the file
and newer versions don't do it now.

If you have some time, you can try the scenarios mentioned here. Try it with a
clean profile in 1.7.2. Make some cards in address book, close Moz, start,
delete them, close Moz. Are the people names still visible in abook.mab?
The same can be done with mail - create a new account in this profile, make some
folders, restart, delete them. Are they still mentioned in panacea.dat?

Comment 8

[WADA:World Anti-bad-Duping Agency]

David already implemented "rebuild panacea.dat on shutdown, if a folder has been
deleted" in patch for Bug 267345.
See Bug 267345 Comment #14.
The patch was checked-in on 2004-11-09.
aceman and Ross Kendall, does poblem still remain after the patch?    

(In reply to comment #7)
aceman, do you mean "deleted but uncleared data in abook.mab" was saved in
panacea.dat in your environment?
If not, that is completely different issue from this bug.
(Something like "Deleted mail is not cleared until compact folder")
Open separate bug for it if "deleted but uncleared data in abook.mab" caused
problems on your environment.

Comment 9

[David :Bienvenu]

yes, thanks m-wada, this is a dup, and aceman's issue is different.

*** This bug has been marked as a duplicate of 267345 ***

Comment 10

[:aceman]

>*** This bug has been marked as a duplicate of 267345 ***

It isn't nice to dupe a bug against a higher numbered one:) But if it helps...

(In reply to comment #8)
> David already implemented "rebuild panacea.dat on shutdown, if a folder has been
> deleted" in patch for Bug 267345.
> See Bug 267345 Comment #14.
> The patch was checked-in on 2004-11-09.

Yes, it seems that that implementation may accidentally fix the problem reported
here. But as we commented on it further, it may be that the problem isn't only
with panacea.dat, see below.

> aceman and Ross Kendall, does poblem still remain after the patch?

Will check in 1.8 final...

> (In reply to comment #7)
> aceman, do you mean "deleted but uncleared data in abook.mab" was saved in
> panacea.dat in your environment?
> If not, that is completely different issue from this bug.
> (Something like "Deleted mail is not cleared until compact folder")

No data from abook.mab in panacea.dat, that is a misunderstanding.
I wanted to say that:
deleted data is still preserved in abook.mab
IN THE SAME WAY AS
deleted data is still preserved in panacea.dat
IN THE SAME WAY AS
deleted data is still preserved in mbox files
BUT
there is a 'compaction' operation for mbox files (folders)
BUT there is no (visible) one for abook.mab and panacea.dat.

So, either there is some hidden one, or it kicks in automatically (on the first
quarter of the moon), or there is a bug and the data stays there forewer.

> Open separate bug for it if "deleted but uncleared data in abook.mab" caused
> problems on your environment.

It caused no problems so far, but it is a privacy issue, atleast.