404006 - Opening download manager takes way too long

Status: VERIFIED FIXED

(Toolkit :: Downloads API, defect, P3)

Description

[Jesse Ruderman]

Opening the Download Manager (with Cmd+J) takes 7 to 9 seconds in the 2007-11-15 nightly.  In the 2007-11-11 nightly, it only takes 1 second with the same profile.

Comment 1

[Shawn Wilsher :sdwilsh]

TLD information?

Comment 2

[Ed Lee :Mardak]

How many entries do you have showing up? Is it just opening or is it also searching?

If I search for 'a', it returns all 150 downloads in about 1-1.5 seconds for me.

Opening is almost instantaneous for me, but that only shows 8 entries.

Comment 3

[Jesse Ruderman]

About 100 entries show up when I open download manager.  I don't think it's searching, just showing the most recent downloads.

Searching is also slow, even when only a few entries match.

What is "TLD information"?

Comment 4

[Ed Lee :Mardak]

We recently started showing the eTLD+1 for finished download.

The new designs seem to point to showing all downloads by default, so I'm guessing whatever slowdowns you see now for the 7 days of downloads will only get worse.

But I'm still not sure what would be causing the issue.. Do you have a lot of downloads whose corresponding file is deleted?

You're running on a macbook pro?

Comment 5

[Jesse Ruderman]

I have deleted or moved most of the files I have downloaded.  Many of them have identical filenames.

I'm running Tiger on a MacBook Pro.

Comment 6

[Mike Schroepfer]

Edward can you take this?

Comment 7

[Ed Lee :Mardak]

This will probably be addressed with bug 392097 which will make all the downloads show up instead of just 7 days, so the problem will only get worse.

Comment 8

[Ed Lee :Mardak]

Actually.. perhaps part of the overhead is initializing the download manager backend -- not just displaying the items.

So if we figure out something for bug 399838, it might help as well.

Comment 9

[Ed Lee :Mardak]

Jesse, do these builds take a long time to open the download manager as well?

https://build.mozilla.org/tryserver-builds/2007-12-03_23:[e-mail]-fastdlmgr/

First time opening vs second time?

Comment 10

[Ed Lee :Mardak]

Jesse: Here's a build that shows 1 item then 10 at a time. Is the download manager still slow to open?

https://build.mozilla.org/tryserver-builds/2007-12-04_09:[e-mail]-chunk10dlmgr/

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

I can corroborate Jesse's findings; the 11th build is still slow, but doesn't appear to clobber the UI thread (does that still run on |main()|?); with recent trunk builds, and my downloads.sqlite file (https://bugzilla.mozilla.org/attachment.cgi?id=292568) it takes quite a bit of time, and on its first opening attempt we beach-ball for over 6 seconds (on my machine, at least) before even showing any portion of the DM window at all.

I'll try the tryserver builds tomorrow with this downloads.sqlite file and get some wall-clock timings for us.  

Comment 12

[Ed Lee :Mardak]

If you want a build to try..

https://build.mozilla.org/tryserver-builds/2007-12-07_09:[e-mail]/

Comment 13

[Stephen Donner [:stephend] Not actively reading bugmail]

Not scientific by any means, but I tested each build:

Today's 2007-12-13-19 vs. Mardak's build in comment 12, above, with the following scenarios:

1) Invoke Tools | Downloads right after an identical Firefox start (same profile, obviously)
2) Took note of the "first-open" time
3) Invoked Tools |Download 9 successive times, taking note of each time

* Results:

Build | First-open time | Successive Open Average (9 opens)
-----------------------------------------------------------
Trunk |     ~16 sec     |            ~14-15 sec           |
Mardak|      ~2 sec     |               ~16 sec*          |
-----------------------------------------------------------

* Even though the TOTAL average time to _fully_ load and render the Download Manager UI takes this long, it's -far- more usable than the current, non-patched trunk builds; here, one can scroll at 3 or 4-second intervals (when whatever's grinding the UI thread decides to give CPU cycles back to the DM window); on the trunk, we block the UI thread completely--spinning beach ball and all--for the ~16-second duration.

(Do we want my downloads.sqlite posted here?)

Comment 14

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #13)

System spec: Mac OS X 10.4, MacBook Pro, 2.2 GHz Intel Core Duo, with 2 GB of RAM.  I'll try to find time to test Windows tomorrow; do we want XP or Vista, or both?

Comment 15

[Ed Lee :Mardak]

I'm kinda surprised at the first-open time. Do you mean first opening when first starting Firefox? My iBook G4 1.33GHz PowerPC shows the download manager window with 1 download in less than ~.5 sec even when compiling trunk at the same time.

There's 3 interesting times:
1) First open of dlmgr after starting firefox (how long until the window appears)
2) Non-first opening of dlmgr (how long until the window appears)
3) Finished loading all items (how long until the window stops adding stuff)

Most important time is #2 as #1 will be dealt with in another bug and #3 isn't really the common use case.

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

Sigh; my table is way misleading; sorry about that.

For Mardak's builds, the time should be represented as such:

Total time to open the DM window and finish rendering is around ~16-17 seconds; of those, it takes us--again on average--2 seconds to open the DM window, which leaves 14-15 seconds to completely finish populating (I take the time it takes to completely finish setting the relative vertical size of the scrollbar, as the indication of time-to-render the DM lists).

This means that the overall net time to completely finish opening the DM is around 16 seconds on both builds; the perception and user experience is much-improved in Mardak's build, however we still have a larger performance issue at stake.

Comment 17

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: downloads.sqlite with 175 entries

Comment 18

[Shawn Wilsher :sdwilsh]

Attachment: downloads.sqlite with 277 entries

Just because I can, here's one with 277 entries.  This one has been through a lot (basically since we first landed the new DM).

Comment 19

[Shawn Wilsher :sdwilsh]

So - we can improve UI performance by adding more delay to our setTimeout call I'd think.  Perhaps we should play around with this?

Comment 20

[Ed Lee :Mardak]

Both bug 392097 and bug 399838 are fixed, so the time for the download manager to show up when pressing ctrl-j or starting a download with auto-open should be really fast.

Comment 21

[Stephen Donner [:stephend] Not actively reading bugmail]

I spun-off bug 408745; I hope to get a chance to verify this bug in a bit.

Comment 22

[Stephen Donner [:stephend] Not actively reading bugmail]

I'm going to claim victory on this particular bug, as it's now nearly instantaneous to actually *open* the Download Manager window on trunk; building the list is a different story, but is still much-improved due to the way we chunk its populating.

I think we could and should still do some caching, such that subsequent DM-window opens are faster, but that'll be a separate bug.

Verified FIXED using Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b3pre) Gecko/2008010204 Minefield/3.0b3pre

Comment 23

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #22)

> I think we could and should still do some caching, such that subsequent
> DM-window opens are faster, but that'll be a separate bug.

I filed bug 410538.