Closed Bug 291255 Opened 19 years ago Closed 15 years ago

calICalendar getItems progress

Categories

(Calendar :: Internal Components, enhancement)

Product:
Calendar
Component:
Internal Components
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 472483

People

(Reporter: gekacheka, Unassigned)

References

(
URL
)

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
Build Identifier: trunk

Currently the only progress information available while loading a
calendar, say with calICalendar getItems, is that getItems calls are
being made.
http://lxr.mozilla.org/seamonkey/source/calendar/base/public/calICalendar.idl#208

In a future version it would be nice to to be able to show some sort
of progress bar.

* ICS calendar providers might be able to provide size of the file
  loaded so far and the full file size, if the remote server provides a
  file size (or estimated from previous file size).

* Database providers might be able to provide the number of items
  returned so far and the number of items in the table or query
  result.

* Composite providers might show some composite progress, such as
  the average of the percentages of each of its component calendars.


Reproducible: Always

Steps to Reproduce:




(One possible approach might be to add this information to the aDetail
parameter of the onGetResult method of calIOperationListener.)
http://lxr.mozilla.org/seamonkey/source/calendar/base/public/calICalendar.idl#364

(An alterative interface change might be to add version of getItems which 
passes something like nsIProgressEventSink.  But it's not clear what to pass
as the aRequest and aContext parameters to the nsiProgressEventSink.onProgress
method.)
http://lxr.mozilla.org/seamonkey/source/netwerk/base/public/nsIProgressEventSink.idl#43
I'm not sure we have any providers that can tell how many items there are in a
set until they've processed them, or have effectively retrieved them from the
data store.  In the mozStorage case, we could select the count(*) as well, but
we'd get that when the first row came in, and would basically be pumping back
calls immediately after that; all the waiting is over by then.  Similarly with
CalDAV, which won't get a result from the server until the op is completed, and
ICS, which only knows how many items are in a VCALENDAR when it hits the END:
block, by which point it's done.

If there are more expensive operations than retrieval being performed, the
operating code should know how many are being operated on, and could perform the
book-keeping itself.  Because asynchronous notifications could in theory be
re-ordered, putting some ordinal in the calls might also be misleading, and I
don't think providers should be responsible for tracking that for the sake of a
possible future caller.

I might be missing a use case, though: can you described what sort of operation
would want these progress notifications, and is such that the provider can
inform the caller usefully?
USE CASE: A progress bar is needed particularly when the data is
on a remote server.  In that case, the bottleneck is the network
rather than the disk.  Someone who opens a large event calendar on a
slow link (say, dialup or slow wireless) is going to want to see how
fast it is making progress.

(People will open large calendars.  Some Sunbird users have
tried opening or importing calendars with a thousand or more of events
in them and complained Sunbird was unusable.  A calendar may be large
because
- the calendar covers a long period (such as an active calendar
  in daily use for years), or
- the calendar covers parallel tracks (such as schedules for large
  conferences or an olympiad), or 
- the calendar contains detailed descriptions (such as talk abstracts
  or meeting agendas).)

EVENT COUNT NOT NECESSARY FOR PROGRESS FRACTION: For a progress bar,
just need a fraction (a number so far)/(total expected number).  The
numbers need not count events provided: if it is easier or more
informative to provide a different number, such as the byte count, or
rows scanned, or something else, it may.  As mentioned, a remote ICS
file provider might use the http content-size and bytes so far rather
than the event count.

Maybe aDetail could provide one or two numbers which are encouraged
but not required for providers:
 - progressFraction: float fraction provided so far if known (else -1)
[- expectTotal: a total number of events if known (else -1).]

Only the progressFraction is needed for a progress bar.  [ExpectTotal
might be useful to some callers for preallocating an array to store
the events, but it is not as important as progressFraction.
ExpectTotal may be dropped if it is not useful enough.]

(An alternative would be to providing a number so far and a total
number, which are not necessarily event counts.  But because some
providers provide event counts, someone might erroneously come to
expect that they are always event counts.  By using progressFraction
and expectTotal, it is clear when eventTotal is -1 the
progressFraction was computed on a different basis than the event
count.)

ORDER: Currently the getItems interface says (1) the events are
returned in occurrence order, and (2) it may be replaced by something
more SQL-like (which I take to mean it might include sorting).  The
order matters already, so it does not appear to be an additional
burden.  (For the first case, the composite calendar could do a
date-ordered merge on the ordered component result streams and still
provide results somewhat incrementally.)

http://lxr.mozilla.org/seamonkey/source/calendar/base/public/calICalendar.idl#208


(In reply to comment #2)
> As mentioned, a remote ICS
> file provider might use the http content-size and bytes so far rather
> than the event count.

Just using the download progress isn't enough. After downloading, the file needs
to be parsed. This might take some time as well. So it isn't that simple.
Tracking the download progress should be possible already, since that's a
network function and not a calendar processing one (and we don't do ICS-fetching
from getItems in the general case).

We don't know how far libical is into processing the string buffer we provide it
until it's done.

The storage provider doesn't know how many rows have been scanned until the
query returns its first row, and since we return in order, that means that all
rows have of necessity been scanned.

I guess what I meant to ask was: if you were to write a patch that added some
progressFraction to the callback, can you point out to me where our providers
would calculate that fraction meaningfully?  I can't think of any where we know
a fraction that's interesting before we've got most of the hard work done, and
it's up to the callback to determine what expensive operation it's going to
perform on the results.  I think that in pretty much every case there's going to
be a "still doing something" indeterminate-progress for most of the operation,
then a very fast counting up from 0% to 100%.  We talked about progress
notification when we were first designing these interfaces, and didn't come up
with anything workable, which is why I'm interested in hearing concrete
data-flow examples here.

I think the reason that people thought Sunbird was broken before, on large
calendars, was that it just hung the UI while processing data.  Eliminating that
is in large part why we have the async-callback model of the post-0.2 trunk.

It's possible that we should kill the "return in order" piece of the getItems
contract.  That was put there just to make initial porting of the old UI code
easy, because the old UI code depended on it.

I think the one case where this could be useful is for WebDAV ICS calendars,
since the percentage could meaningfully be calculated from bytes received/total
bytes.

Note that the up-and-coming calICachingCalendar is likely to make this
functionality even less used.
Unfortunately, I do not have cycles to work on Calendar stuff these days (just as it's getting to the good part!), so I am a bad owner for these bugs.  To delete the tragically-large chunk of bugspam, search for gregorianabdication.
Assignee: shaver → nobody
This should be part of the following bug:
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.