1562490 - Deleting large amount of bookmarks and folders hangs/locks/freezes Mozilla Firefox completely for some time

Status: NEW

(Firefox :: Bookmarks & History, defect, P5)

Description

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

STR:

1. Delete 99% of bookmarks and folders (~24MB, ~85.000 bookmarks) to create test bookmarks backup for bug #1562489
   and enjoy nice amount of hang for some time, Firefox windows and Library window goes black in this time

Comment 1

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

Attachment: hang of Firefox.png

Comment 2

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

Attachment: hang of Library.png

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

:Virtual, if you think that's a regression, then could you try to find a regression range in using for example mozregression?

Comment 4

[Mark Banner (:standard8)]

As we've said before, anything greater than ~2000 bookmarks is really an edge case. On beta less than 1% of users have more than 2k bookmarks.

Some of the notifications changes may help with this, and the recent annotation removals may also help.

Comment 5

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

On Chromium browsers I don't have this problem. Everything is nearly instant, even with that much amount of bookmarks that I have now.

I was thinking about making this task in other process, like Library one, so it won't hang Firefox completely and Firefox could still be operative as browser for browsing website pages under this task. Hanging main Firefox process, especially when we have already e10s, fission in mind, and that much other processes like GPU, Extensions, Plugins, Privileged Abouts, Network etc., is kinda unacceptable. Any plans for this? What do you think about this idea? As I mainly annoyed by complete hang of Firefox, not with amount of processing time which I can live due to massive amount of bookmarks, but good that you still thinking about making it faster.

Comment 6

[Mark Banner (:standard8)]

(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #5)

On Chromium browsers I don't have this problem. Everything is nearly instant, even with that much amount of bookmarks that I have now.

Chrome has a different, simpler bookmarks model. I believe they also keep everything in-process.

I was thinking about making this task in other process, like Library one, so it won't hang Firefox completely and Firefox could still be operative as browser for browsing website pages under this task. Hanging main Firefox process, especially when we have already e10s, fission in mind, and that much other processes like GPU, Extensions, Plugins, Privileged Abouts, Network etc., is kinda unacceptable. Any plans for this? What do you think about this idea? As I mainly annoyed by complete hang of Firefox, not with amount of processing time which I can live due to massive amount of bookmarks, but good that you still thinking about making it faster.

From past experience, the main thing we need to do is to improve our notifications and also how our views work. Our views are currently generated via synchronous queries, and there's also expensive updates for the current selection when the views change. There's a patch in progress by a contributor for removing bookmarks that may help work towards this some of this.

Unfortunately, there's a lot of non-trivial work there to fix all of that which we do want to do at some stage. In this case, helping less than 1% of users isn't going to get it prioritised.

That said, we are trying to get some work underway to generally improve the library, however there's nothing confirmed on that yet.

Comment 7

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

Out of curiosity, did you try removing 85k bookmarks at once in Chrome, and it was instantly updating the ui?

Comment 8

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

(In reply to Mark Banner (:standard8) from comment #6)

Chrome has a different, simpler bookmarks model. I believe they also keep
everything in-process.
[...]
Unfortunately, there's a lot of non-trivial work there to fix all of that
which we do want to do at some stage. In this case, helping less than 1% of
users isn't going to get it prioritised.

I know, but my bookmarks won't be getting smaller only larger and using it with Mozilla Firefox is nightmare compared to Chromium usage, but how about "copying" Chromium model database of bookmarks? Even if Mozilla Firefox loose some features with less than 1% of users are using to gain massive amount of performance and speed, seems to me like something to reconsider or maybe write it completely from scratch like new Firefox Mobile? Providing extension by Mozilla to recover these features will be also nice for these users.

(In reply to Marco Bonardo [::mak] from comment #7)

Out of curiosity, did you try removing 85k bookmarks at once in Chrome, and
it was instantly updating the ui?

Yes and yes, with small <1s hang when I moved from Bookmarks tab to website page tab. Also I was able to browse website pages under this very short process there as it took less than 5 seconds there.

I will do detailed test and post it in bug #1320534, like I done this before in past.

There are also test bookmarks (only 1 folder of spam news) under bug #1320534 comment #11 provided by me, if you want to test by yourself.

Comment 9

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

I want to add that same thing happens when I'm deleting whole history (>100.000 items [if I remember correctly ~130.000-140.000, so probably maximum limit, if there is one of course]). Probably same thing will happen when Downloads will be full of items.

STR:

1. Press "Alt" keyboard button to see "Menu Bar"
2. Select "History" item in Menu Bar
3. Select "Clear Recent History..." item in History menu to open "Clear All History"
4. In "Time range to clear:" select "Everything"
5. Next select "Browsing & Download History"

Workarounds:

• browse less website pages ;)
• remove history each Firefox shutdown or after some specific time like week or month etc.

Comment 10

[Mark Banner (:standard8)]

History is already covered by bug 734643. There's different algorithms in play there, so we should keep these separate. Downloads has a completely different system even though it is in the same window, so that might not be affected in the same way.

Comment 11

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #8)

I know, but my bookmarks won't be getting smaller only larger and using it with Mozilla Firefox is nightmare compared to Chromium usage, but how about "copying" Chromium model database of bookmarks?

There isn't a database model advantage to copy, Unless things changed recently, Chrome just loads all the bookmarks in memory (bookmarks are stored in a simple JSON, not even compressed!), it manipulates this gigantic object in memory and dumps it to disk at regular intervals.
That system has compromises, like memory usage and dataloss (indeed if you do a few large operations and it crashes before dumping to disk...). We have users with really low memory availability to support, we ship in more than 70 locales. We have undo feature, we have tags, we have transactions protecting every operation.

With this I'm not saying we can't improve, there's plenty of things we can do, I'm just saying that systems with different initial requirements will obviously diverge in functionality, resources and performance. The work we are doing with bookmarks feature simplification is clearly aimed at reaching a point where it will be easier to make those improvements.

Comment 13

[Gingerbread Man]

(In reply to Mark Banner (:standard8) (afk 9 - 18 Aug) from comment #10)

History is already covered by bug 734643. There's different algorithms in play there, so we should keep these separate.

I don't understand why bug 1574144 was re-duped to this given the above comment.

Comment 14

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

Because this bug is not about Firefox performance of deleting history and how long time it takes for Firefox to deletes history completely filled up.
This bug is about complete hang, lock and freeze of whole browser GUI with unavailability to use it because of this, when deleting task is processed.
It won't be fixed by bug #734643, only hang time will decreases.

Comment 21

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

(In reply to Marco Bonardo [::mak] (Away 7-25 Aug) from comment #11)

(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #8)

I know, but my bookmarks won't be getting smaller only larger and using it with Mozilla Firefox is nightmare compared to Chromium usage, but how about "copying" Chromium model database of bookmarks?

There isn't a database model advantage to copy, Unless things changed recently, Chrome just loads all the bookmarks in memory (bookmarks are stored in a simple JSON, not even compressed!), it manipulates this gigantic object in memory and dumps it to disk at regular intervals.
That system has compromises, like memory usage and dataloss (indeed if you do a few large operations and it crashes before dumping to disk...).

Thankfully I never have dataloss issue with bookmarks in UnGoogled-Chromium, compared to Mozilla Firefox. But I didn't use UnGoogled-Chromium that much in past.

(In reply to Marco Bonardo [::mak] (Away 7-25 Aug) from comment #11)

We have users with really low memory availability to support, we ship in more than 70 locales.

I have even less than minimal Firefox System Requirements,
which is "2GB of RAM for the 64-bit version" ( https://www.mozilla.org/en-US/firefox/68.0.2/system-requirements/ )
and I have 1GB of RAM in Windows 7 (64-bit) in my old laptop which is HP Mini 210-1020SW
and I didn't encounter any problems and issues in UnGoogled-Chromium with bookmarks tasks and my bookmark database,
I saw only performance speed advantages.

So in my opinion based on performance tests, UnGoogled-Chromium has much more superior database,
and lately more and more I'm seeing it compared to Mozilla Firefox,
as simple (from user perspective) tasks like deleting or moving bookmarks or folders are done nearly instant in UnGoogled-Chromium, but in Mozilla Firefox it takes seconds or even minutes (in bad scenario even hours).

So I think that there is the advantage to copy something from UnGoogled-Chromium to Mozilla Firefox.

Main problem in this bug is that bookmarks tasks hangs/locks/freezes Firefox GUI completely for that time.

Comment 23

[Timea Cernea [:tbabos][inactive]]

Hi Virtual_ManPL,
Are you still able to reproduce this issue on the latest Firefox versions? If you don't have the time to look into this, it would be great if you could upload a bookmarks.json with that many bookmarks and I will give it a try. Thanks!

Comment 24

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

(In reply to Timea Cernea [:tbabos] from comment #23)

Hi Virtual_ManPL,
Are you still able to reproduce this issue on the latest Firefox versions? If you don't have the time to look into this, it would be great if you could upload a bookmarks.json with that many bookmarks and I will give it a try. Thanks!

Yes. The issue still occurs in latest Mozilla Firefox Nightly 97.0a1 (2021-12-20).

Comment 25

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Moving open bugs with topperf keyword to triage queue so they can be reassessed for performance priority.

Comment 26

[Julien Wajsberg [:julienw]]

It looks like an issue with the code handling bookmarks.

Could you please capture a performance profile as mentioned here and post it here? https://firefox-source-docs.mozilla.org/performance/reporting_a_performance_problem.html

It would help us a lot to identify the problem.

Thanks!

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Comment 29

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:mak, since the bug doesn't have a severity set, could you please set the severity or close the bug?

For more information, please visit BugBot documentation.