408745 - Download Manager clobbers the UI thread and is slow to respond when updating its list

Status: VERIFIED FIXED

(Toolkit :: Downloads API, defect, P3)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

This is a spinoff of bug 404006; although that bug addressed the time it takes to open a larger-sized downloads.sqlite (but certainly not an abnormal one at all), we still have major performance issues with seemingly clobbering the UI thread.

This bug is meant to address--at the least--the fact that populating the DM list is still slow, and also that, in general, we stomp the UI thread, making things like:
* context menu
* window resizing
* window-focus switching, etc., slow...

We really should do as much as we can to address this.

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

(In particular, see https://bugzilla.mozilla.org/show_bug.cgi?id=404006#c19.)

Comment 2

[Shawn Wilsher :sdwilsh]

Yeah, this really hits me with my 280+ downloads file :(

Comment 3

[Mike Beltzner [:beltzner, not reading bugmail]]

Do we have any strategies for success here? Until we do, I don't know if we can block on this.

Possible strategies:
 - going back to a "Show All" UI
 - reducing the bunching size some more
 - building the list in delayed startup? (that one might not be a good idea)

Comment 4

[Shawn Wilsher :sdwilsh]

I think there is a lot that we can still try.

Comment 5

[Ed Lee :Mardak]

What does it mean to be 'generally slow'? Is it still slow to initially open the download manager?

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #5)
> What does it mean to be 'generally slow'? Is it still slow to initially open
> the download manager?

I've updated the summary from "...is still generally slow" to be "...is slow to respond when updating its list"; to answer your second question, specifically, no, it's not.  It's all the building of the DM items that takes the time and makes the window/Firefox fairly non-responsive.

Comment 7

[Ed Lee :Mardak]

Does this help any?

https://build.mozilla.org/tryserver-builds/2007-12-19_20:[e-mail]-show0/

All that does is make it clear the list then add one item then add in chunks. (Currently it clears the list *and* adds one item then adds in chunks, so it separates the first part.)

This is kinda a separate issue from what's described in the bug.. this would be the beachball when opening the list. (This bug is describing using the UI while it's populating the list.)

Comment 8

[Ed Lee :Mardak]

Attachment: v1

Just an idea.. it's building in the tryserver as show0.500.5

Delay up to 500ms per batch and do only 5 per batch still keeping with the 'add items faster' for the first several batches.

5 items: 50ms delay
10 items: 100ms delay
15: 150
20: 200
25: 250
30: 300
35: 350
40: 400
45: 450
50: 500
55: 500
...

The first 50 will take at least 3 seconds to show and each 2*5 items after that takes a second at least. So 200 items will take the 3 seconds to show the first 50, 5 for 50-100, 5 more for 100-150, and 5 for 150-200.

"We want to show the first W downloads within X seconds and with Y downloads showing up every Z seconds." ?? What should W X Y Z be?

W = 50
X = 3
Y = 10
Z = 1

Comment 9

[Ed Lee :Mardak]

I suppose another knob is how many updates there should be every second.

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #7)
> Does this help any?
> 
> https://build.mozilla.org/tryserver-builds/2007-12-19_20:[e-mail]-show0/
> 
> All that does is make it clear the list then add one item then add in chunks.
> (Currently it clears the list *and* adds one item then adds in chunks, so it
> separates the first part.)
> 
> This is kinda a separate issue from what's described in the bug.. this would be
> the beachball when opening the list. (This bug is describing using the UI while
> it's populating the list.)

This build helps immensely in one area in particular--the spinning beachball (times it takes to open the download manager)--but also slightly helps with the 'slow-to-open-context menus' problem.

The Download Manager now opens sub-second (instantaneously); while there are improvements we can make to the 2nd issue (that, once it's open we still poll/clobber the UI thread a bit too much), the way it chunks seems much-improved.

Additionally, it seems to cut in 1/2 the time it takes to fully display the same DM profile with which I've been testing; 8 seconds as compared to 16 on the trunk...

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

I also wanted to report that https://build.mozilla.org/tryserver-builds/2007-12-19_23:[e-mail]-show0.500.5/ is even better than https://build.mozilla.org/tryserver-builds/2007-12-19_20:[e-mail]-show0/, especially for UI/app responsiveness, for things mentioned in comment 0; it's the best patch/try build, by far...

Comment 12

[Ed Lee :Mardak]

Attachment: v1.1

Split off the beach ball stuff and try out 300ms delay between adding 3 items. (instead of 500 and 5)

stephend: Can you try these builds on mac and windows?

https://build.mozilla.org/tryserver-builds/2007-12-20_19:[e-mail]/

It's got..

Bug 408696 - Don't show leading 0 for download dates
Bug 409311 - I see beach ball instead of download manager
Bug 408745 - Download Manager clobbers the UI thread and is slow to respond when updating its list
Bug 409105 - Just finished download shows start time instead of completed time
Bug 409314 - Default download manager size should be golden (485x300)
Bug 409317 - Download date/time should show the full date and time as a tooltip
Bug 409326 - Complement eTLD+1 status with a full host tooltip
Bug 394516 - Figure out a remaining-time rounding scheme for minutes -> hours/days
Bug 406857 - Don't clear referrer when restarting downloads
Bug 401316 - Open-with downloads are readonly

Bug 395735 - Instrument the location bar auto-complete to report accuracy
Bug 406359 - Unify the logic for url bar searches and drop down items
Bug 406358 - Location bar drop down ignores frequency of visits
Bug 395739 - adaptive learning (match entered text to selected item) in url bar autocomplete
Bug 393678 - location bar autocomplete should take word boundaries in account
Bug 406422 - Globally decay adaptive input history to allow for new entries
Bug 406425 - Limit how many adaptive input history entries to track
Bug 406427 - Penalize skipped results to allow adaptive learning of shifting preferences
Bug 352923 - Autocomplete should prefer typed URLs to URLs that were merely visited
Bug 394066 - use "if bookmark" to weight url autocomplete results
Bug 409271 - Weight the main website address higher than sub-pages

Comment 13

[Shawn Wilsher :sdwilsh]

Comment on attachment 294164 [details] [diff] [review]
v1.1

r=sdwilsh

Comment 14

[Ed Lee :Mardak]

Checking in toolkit/mozapps/downloads/content/downloads.js;
/cvsroot/mozilla/toolkit/mozapps/downloads/content/downloads.js,v  <--  downloads.js
new revision: 1.125; previous revision: 1.124
done

Comment 15

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED with Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b3pre) Gecko/2008010204 Minefield/3.0b3pre; this is much, much improved.  Invoking the Download Manager window is instantaneous, and building the list in chunks ameliorates the "clobbering-of-the-UI-thread" issue.  As I mentioned in bug 404006, I'd like to see us cache the "Downloads" window's list, so I'll log a separate bug for that.

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #15)

<snip>

> As I mentioned in
> bug 404006, I'd like to see us cache the "Downloads" window's list, so I'll log
> a separate bug for that.

I filed 410538.

Comment 17

[Stephen Donner [:stephend] Not actively reading bugmail]

https://litmus.mozilla.org/show_test.cgi?id=5143

in-litmus+