1245349 - display and update bookmarks tree very long

Status: RESOLVED DUPLICATE of bug 1405242

(Firefox :: Bookmarks & History, defect)

Description

[TREMBLAY,Bernard]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20160123151951

Steps to reproduce:

Manipulate the bookmark tree in lateral panel or in manager :
- add folder
- add element (url)
- move folder
etc.
This on a large bookmarks structure (near 30,000 url and 12,000 folders)


Actual results:

The update take a very long time and uses a lot of CPU :
- till 30s using 2 cores on a core I7 2630qm with 16Go Ram

This is not a classical bug, but an unknown limitation of the tool which become redhibitory to use Firefox for now.

This is not new with 44.


Expected results:

Quite instantaneous update.

In my experience :

There is quite no reason to have longer span of time than with windows explorer.

So I am searching where are the sources elements to look at -(I got a GIT workcopy)

In my opinion the data structure and algorithms are probably not optimized (seems that in json data an element points only to next), then the update need often to walk all along the structure to change all "next" or "subfolder".

I had to do it long years ago. The lonely solution which lead to a quite completely independent  of the size of the tree, is to have a minimum of four pointers for each element :
- next brother
- previous brother
- parent
- first child
- optionally : 
    - next brother of parent (earns one step for each last element into a level)
    - last child : quick add of new element at the end of children
- then any new element is appended as new, never change of internal index of object array
- then any change have just to modify from 2 to 4 max records which index are known. And after walk for the display of the visible elements.
- navigation into the tree is very easy (gotonextbrother, gotopreviousbrother, gotoparent, gotonextinparentlevel)
A short lib can hold the array.
It is only when there are to export the whole tree that it can take a more long time.

At rare time to optimize the use of memory it is useful to sort the array and reindex (too for searching elements).
Some parts of the tree (from a main folder) can get a "archived" attribute then the json can be split and the archived sub-tree quite never loaded. 
This is my experience

If this is not the cause, what's ?

Note about the history of the bookmarks base : 
I got a so large tree because of failed import (timeout for script twenty times...) so the import result have been added each time.
The tree had been moved firstly to google-chrome, later after the failure of chrome at update version ? (04/2015) made lost of a great part of the url (remaining folder tree) some not well held by XMarks (upload failure and crash because of corrupted data by chrome auto-upgrade) bookmarks upgrades (saved as html ) had to be re-imported to Firefox... from html backups.

With such long time to make changes (reduce and suppress duplicates), needs, estimated, four years of job have to be spent to clean up... (to get no more than 20,000 elements needed). 

The other solution is to create a special soft to treat the html exported or better directly the json.

The better is an optimized Firefox library... may be

Comment 1

[Marco Bonardo [:mak]]

We actively work on improving the performance of batch operations, there isn't a silver bullet that will solve all the issues here.