21986 - Display estimated download time more realistically

Status: RESOLVED WORKSFORME

(Core Graveyard :: File Handling, defect, P3)

Description

[mahjongg]

When startin a download process using mozilla seamonkey (observed with Buid ID
1999121612 ) an estimate is displayed of the time it will theoretically take to
finish the download process.
If you observe this estimate you will notice that this estimate behaves strange.
Instead of a nice and systematical decrementation of this value you will
probably notice that the value is updated eratically. The value will even
increase somtimes goinf from for example 4.30 minutes to 4.35 minutes.
Actually the reason for this is quite clear if you think about it. The displayed
figures are an _estimate_ calculated from parameters of the percentage of the
file downlaoded and the duration of the download so far. This means that the
total estimated time the complete download will take is, especially in the
beginning of the download, very uncertain and so can change during the download.
Because the total estimated duration can change the "counter" can also run
backwards.
For an unsophisticated user this can be very confusing.
Therefore Ill make the following proposal.
change the display algorithm so that:
During the first few seconds no estimate is displayed of the estimated download
time, so the algorithm can make an estimate of the expected transfer rate upon
which a rough estimate in units of minutes can be calculated of the download time.
Then this estimate can be displayed, in units of a minute. If the estimated time
is expected to be less than a minute, simply display a message to that intent.
The purpose of this is that for short downloads knowing that it will take less
than a minute is enough information. For downloads with an estimated time
between 1 and say five minutes a slightly more accurate notition in units of
minuted and ten seconds can be displayed with enough certanty to be accurate and
usefull. For downloads that are estimated to take much longer than five minutes
the value should be displayed in units of a minute only.
After a download period of 2 minutes or so, the displayed accuracy can be
increased from units of one minute to units of one minute and ten seconds.

The resuling user experience will be much more logically correct and intuitive.
Other advantages are that the inherent unreliability of the estimate that is
greatest during the initial phase of the download will be masked.
The user will not be confronted with unintuitive behaviour of the system so it
will gain more faith in it.

Comment 1

[Eli Goldberg]

Moving to correct component.

[e-mail], please note this component for future download progress bug
reports, so that you can report them directly to the right place. Thanks!

Comment 2

[Sean Richardson]

Smoothing the estimated download time is a very good idea - I can confirm that
some users do get confused and ask "why?" (usually the ones that have no
intention of listening carefully enough to have any chance of understanding
a clear explanation!).  And estimates based on receipt of just one or two
packets can be very unrealistic, if the latency is high or the user is using
a modem link with other traffic.

But I am concerned that the proposed change would present the estimate too
coarsely to be realistic in two common circumstances: very short files, and
download rates that get much slower during the download.

For very short files, an estimate of 1 minute or less gives the user no useful
information at all. If, after 3 seconds, 7K of 10k has been downloaded,
the appropriate unit for estimated time remaining (ETR) is seconds, and the
user will normally just wait. If, after 3 seconds, 7K of 100k has been
downloaded, a rough estimate of 1 minute is as good as any, and the user may
prefer to check e-mail or view another webpage while waiting. Not everyone
is patient - and those who are least patient hate (yes, hate) getting no
feedback (and even those who are normally patient can become inpatient faced
with no feedback for 10 seconds or more).

For this, I'd propose taking into account both download speed and file size
rather than presenting the first estimate after a fixed number of seconds.
If at all possible, no estimate should be given until at least 8 packets
are received (to smooth out the initial latency at least a little), or at least
10 percent of the file is recieved, whichever comes first - but in any case
the first estimate should appear within 3-5 seconds. Regardless of anything
else, the ETR should be updated after another 8 packets at the latest,
to smooth out the effects of the TCP slow-start somewhat.

Also, if the download rate for a longer download suddenly gets much slower,
the proposed algorithm would give the user no feedback at all for over a minute.
If the user is downloading, over a 28.8 kbps modem, a large file from a server
on another continent at a rate of 1k per second with a time remaining of 30
minutes, and then starts a download from a local server that stabilizes at a
rate of 2.2k per second, the first can easily be reduced to 0.5k per second,
so that it takes a full hour longer, not 30 minutes. This kind of slowdown
can also happen if the user starts actively websurfing while the download
continues.

With 4.x, the ETR starts climbing immediately, but slowly, as the reported
download speed very slowly decreases. This at least gives the user some
immediate indication that the first download will now take longer. It
appears that the instantaneous download speed is heavily smoothed,
and the ETR is not smoothed at all.

Implementing smoothing based on proportions on the ETR would allow the ETR
to be presented much more realistically if the download speed varies,
while not confusing the user with uninformative jitter. I'd propose decreasing
the ETR presented to the user if it improves by as little as 0.2 percent from
the last update (or 5 seconds less, whichever comes sooner), but not presenting
a longer ETR until it is at least 1 percent longer than the previous reported
ETR (or 60 seconds longer, whichever comes first). With this smoothing in
place, the download speed could be smoothed *less*. The result of all this
would be a serious reduction in meaningless reporting of changed ETR,
but at the same time an improved and more realistic reporting of ETR when
download rates suddenly change meaningfully.

Another issue is the maximum rate at which updates should be displayed.
Once per second makes sense for downloads with ETRs under 60 seconds,
but once per 5 seconds may be sufficient for downloads with longer ETRs
(this would revert to once per second in the last 60 seconds).

Comment 3

[mahjongg]

Yes I totally agree with your comments.
The protocol I described was an explanation and was not intended to be exact.
I am glad you seem to understand exactly what is involved and you have
obviously more experience with TCP-IP network behaviour than I do ( I Must
confess that most of my experience with this is 10 years old and with BBS
systems).
The gist of my proposal is:
Only give information to an accuracy that is consistent with the statistical
uncertainty of this information.
Avoid seemingly irrational behaviour.

Remember that people often in their mind confuse between observed reliability
of the information a system gives them and the relibility of the system itself,
at least I do :-) .

Comment 4

[Eric Vaughan]

Looks like a job for xpapps

Comment 5

[don]

Hmmmm ...

Comment 6

[don]

Move to "Future" milestone.

Comment 7

[Blake Ross]

Changing all Progress Window components to XP Apps: GUI Features.  The Progress 
Window component will be retired shortly.

Comment 8

[Viswanath Ramachandran]

Since Don has left, Vishy is taking his bugs in bulk, pending reassignment.
thanks,
	Vishy

Comment 9

[sairuh (rarely reading bugmail)]

bill/mscott, has this been fixed yet?

am clearing the milestone and adding nsbeta1 just in case this is still an
issue.

if this isn't [or, there are more specific bugs for this], feel free to resolve
this puppy. thx!

Comment 10

[Bill Law]

Each byte is like a little winged heavenly body.  Your hard disk is like the
dull terminus of a small, sharp sewing implement typically used to termporarily
bind fabrics together.

In that context, this bug indeed degenerates into an argument about how many
angels can dance on the head of a pin (well, to be more precise, how many are
estimated to be dancing there in the future).

Just pointing that out.

I do have a patch that adds an asterisk next to the "time remaining" and a bit
of text at the bottom of the dialog explaining that "this value is an estimate;
no guarantee is expressed or implied; Mozilla will not be held liable for any
damages, consequential or otherwise, if any estimate turns out to be off by
hours, or minutes, nay, even *seconds*."

As soon as the lawyers give their super-review, I'll be checking that in.

Sorry; I'm just feeling ironic today.

Comment 11

[timeless]

damages, _monetary,_ consequential or otherwise :)

please check this in

Comment 12

[Paul Chen]

nav triage team:

Not counting angels on the head of a pin here, marking nsbeta1-

Comment 13

[Viswanath Ramachandran]

nav triage: this is a nsCatFood issue since it impacts the feedback to the user 
of a browser operation. 

Comment 14

[Paul Chen]

nav triage team:

Pushing out to mozilla0.9.3

Comment 15

[Paul Chen]

nav triage team:

Not a mozilla0.9.3 stopper, pushing out to mozilla1.0

Comment 16

[sairuh (rarely reading bugmail)]

spam: over to File Handling.

Comment 17

[Mitsu Hadeishi]

It seems to me that the download time algorithm right now suffers from another
problem: it appears to be calculating download speed from the time you first
click on the link, rather than the time you actually begin downloading.  If you
have entered in a download location, this means the initial estimates of
download speed are usually way low, and slowly creep up to asymptotically
approach the actual value as the download proceeds.

Whatever the reason for it, it seems to me that download speed should at the
very least be calculated from the time packets are first received, rather than
some earlier time.  In addition, it might be reasonable to dynamically move the
window used to calculate the speed, using just recently-received packets to
calculate current speed.  Regardless, however, the asymptotically-approaching
speed issue ought to be fixed --- looks like an easy fix.

Comment 18

[sairuh (rarely reading bugmail)]

setting to future/helpwanted.

Comment 19

[Clement Cherlin]

When I first started testing Mozilla, lo unto many milestones ago, I was very
pleased that it displayed the current download speed, and not a decaying average
like Netscape 4. The decaying average in NS4 was always wrong, crept up slowly,
and never reached the actual download speed. Mozilla now seems to use the same
kind of decaying average, and I would vastly prefer the earlier behavior. When I
started downloading at 100KB/s I knew it, I didn't have to wait five minutes.
Now, I have to guess based on how fast the KB Received is going up, just like in
NS4. Regardless of how ETA is calculated, Current Speed should be current.

Comment 20

[:Hb]

Current time display is sufficient.

Comment 21

[Pulsebot]

Pushed by ffxbld-merge:
https://hg.mozilla.org/releases/mozilla-beta/rev/2a2866cbf8bd
[fenix] For https://github.com/mozilla-mobile/fenix/issues/21986 Confirmation when people set tabs to auto-close