144992 - [FIX] BookmarksUtils.cloneFolder does not clone subfolders (hordes of separators created in bookmarks)

Status: RESOLVED FIXED

(SeaMonkey :: Bookmarks & History, defect)

Description

[oliv]

This bug is related to bug 142986.
It was present in older versions of Mozilla, dissappeared, then reappeared when
bug 142986 was solved. That is it exists with RC2, 2002051008 but not older
2002050706 build.

If a folder of bookmarks is copied and pasted somewhere else, in all the
arborescence of the folder (copy and/or original arborescence) bunches of
separators will be added at the end of the subfolders that originally contained
separators.

I created an attachement in bug 142986 that I'll repost here later. I also gave
there an extreme example: After a few hours of use of Mozilla, a folder that
originally had 3 subfolders, 11 bookmarks and 2 separators had 510 supplementary
separators at the end.

Only folders that had separators before the copy paste action will present this
issue. This leads to the quick and dirty workaround : delete all you separators
and this bug won't hit you.

Please note that this bug is strongly related to bug 142986 (copy cut and paste
of bookmarks FOLDERS). Everytime one has been solved, the other has reappeared,
so please test both for eventual effect of a patch.

Comment 1

[oliv]

Attachment: image showing a bunch of separators

Comment 2

[oliv]

Attachment: example of bookmark tree

From a recent test, I can add that this seems to appear only for folders that
have bookmarks. That is if you only have subfolders and separators in the
folder, you will not have separators explosion.

I created yesterday a arborescence that you can see in the image attached. A
folder called "A" with subfolders, bookmarks and separators was in a folder
called "C" in a folder called "temp" (not shown in the screenshot)
I copied this folder in the "B" folder and in my "Linux" folder of my
"Computer" Folder, thus there was three copies: temp->C->A (original),
temp->B->A and computers->linux->A

The problem appeared in Aaa folder only. I had 242 new separators (was it 248,
I don't remember exactly) after a day of work (that is maybe 10-20 times
opening, closing Mozilla) and suddenly Mozilla crashed (Real Player plugin
seems to make it crash from time to time on some web sites). I restarted
Mozilla, and I had 652 separators at the end of the Aaa folder, after the
cnn.com bookmark.

I count these bookmarks by cutting them and pasting them in a text editor,
where they are displayed as "null" strings, thus can be easily counted.

When I have some time, I'll test it at home on linux, but that's maybe only
this weekend (If i have time...)

Comment 3

[mental]

This resembles bug 116362.

Comment 4

[Pierre Chanial]

Confirming, I have seen additional separators appended to the first (in tree
hierarchy) pasted folder.
I could trigger the growth, actually. I deleted the extra <HR> in the bookmark
file but got two of them instead. Do you have more info?

Comment 5

[oliv]

Attachment: result of the previous arborescence after a few days

so here is what I have in both B and C folders after a few days (I did not
access my computer in the weekend and yesterday, so, this is only in a few
days, not a week)

I think the screenshot is nice enough to demonstrate the problem. Do note the
arrows saying that there are (many) more separators that cannot be displayed on
screen (actually, there are 242 superfluous separators). Please remember that
only the bookmark at the top, before cnn.com was added by me, all the others
are generated by the bug.

I have to say I'm not sure for the keyword "dataloss" here. What happens is a
pile of **** is added to the bookmarks, but nothing is lost. You just need a
lot of time to get rid of that ****.

Comment 6

[oliv]

Here are more details:

The separators are created every time Mozilla is started.
Let's take the test case I presented before (the folder Aaa with a separator
followed by a cnn.com link).

After deleting all of the added separators to start with a clean folder, I have
used mozilla during sevral hours without closing it. No separators were added.
So I decided to close it, re-open it and count the number of separators in this
folder.
Original situation: 1 separator
First restart: 2 separators had been added -> 3 separators
Second restart: 6 more -> 9 separators
Third restart: 18 more -> 27 separators
Fourth restart: 54 more-> 81 separators
I did not go further, but the next number would be 243, that is 242 excess
separators, that is the number I mentionned in a previous post

I think you get the picture. Every time I start Mozilla, It multiplies the
number of separators by the number of original separators +2.
This leads to an nearly exponential increase of the number of separators in the
folder! This IS serious!

Note: the first example I mentionned was a folder with 2 original separators and
510 excess separators: It would be explained by the algorithm like this
1)starts with 2 thus 2+2=4 is the multiplication factor
2) 2*4=8
3) 8*4=32
4) 32*4=128
5) 128*4=512 -> 510 excess separators + the 2 original

Comment 7

[Pierre Chanial]

Thank you very much for the info!
I never hit this bug before, but whaoo, it's really insane.
I think I have identified the cause of the problem:
the folder that goes crazy shares the same randomly generated identifier as the
the source folder. you can check it in your bookmark.html file.
It means that subfolders are not properly cloned during copy or more likely that
the random identifier is not regenerated each time one pastes.
Does the problem happens if you only paste once?

Comment 8

[oliv]

Attachment: start of the test case

Okay, here are more details. I'm sorry to attach mainly images, but most are
not big.

Let's start with an arborescence (Aa in A1) (it seems at least 2 levels of
depth are needed to trigger the bug, but this need to be confirmed)
In one folder, we have one or more bookmarks, and N(0) separators (here N(0)=3)


Let's copy this arborescence in order to have Nc equivalent copies (the
original being counted as a copy, here Nc=3)

Then after one restart of Mozilla, the number of separators in the Aa directory
is Nc*N(0) (here 9)
After two restarts, Nc*(Nc*N(0)), after m restarts (Nc^m) * N(0)

I have tested with many many different combinations to have this formula.
Moreover, it can be formulated in an iterative way N(m+1)=N(m)*Nc which is
clearer.
I'll attach step1 and step2 images of the testcase

Comment 9

[oliv]

Attachment: after one restart. separators on the top are original, others are duplicates

Comment 10

[oliv]

Attachment: after 2 restarts. N(0)*Nc*Nc=27 (step 3 has 81 separators, image is big)

Comment 11

[Pierre Chanial]

*** Bug 148288 has been marked as a duplicate of this bug. ***

Comment 12

[Pierre Chanial]

Attachment: Patch v1.0

This patch recursively gives a unique ID to the folder to be cloned and to all
its subfolders.
It fixes the issue reported in this bug but may also fix other weirdness.

Comment 13

[Pierre Chanial]

taking, asking r= and sr=

Comment 14

[Chris Waterson]

The changes to RDF container are fine with me. You can count me as r= on that.

Comment 15

[Blake Ross]

Comment on attachment 85863 [details] [diff] [review]
Patch v1.0

r=blake

Comment 16

[Pierre Chanial]

nominating:
- see attachment 84476 [details] for a easily reproduceable ugliness that is fixed by the
patch.
- the bookmark.html file gets corrupted (two folders with identical id) and is
never cleaned on exit/startup
- may also fix other bookmark strange behavior/crash
- the patch is there and has been tested.

Comment 17

[jag (Peter Annema)]

Comment on attachment 85863 [details] [diff] [review]
Patch v1.0

sr=jag

Comment 18

[Pierre Chanial]

checked in by timeless.
Olivier, could you verify it with a fresh profile and a tomorrow's build?

Comment 19

[oliv]

Attachment: a new bug has appeared?

Build 2002071008 build, but old profile (sorry, but a profile is just too many
hours of work to create a new one)
The separator issue is gone, but it seems a new bug has appeared, as
illustrated of the attached image :)
The A1 folder is the original. I copied its Aa arborescence in A2 and A3. Why
has 'CNN' bookmark moved from subfolder Aa to Aaa?

Okay, I know, this is different, I must open a new bug for it :)
Oh, by the way, I can't export bookmarks :) but its another story (bug
151837)... I hate Mozilla bookmarks :)

Well, thanks for the fix.

Comment 20

[WD]

I'm not certain if this bug is the cause, but bug 156719 seems to have appeared
around the time this bug was fixed.   (both are bookmark-related)

Can somebody check this out?