Sends too big query to server if there are many events (over 1000) causing all events to disappear

RESOLVED FIXED in 1.0b2

Status

Calendar
Provider: CalDAV
--
major
RESOLVED FIXED
10 years ago
8 years ago

People

(Reporter: Keigo IMAI, Assigned: Simon Vaillancourt)

Tracking

unspecified
1.0b2

Details

Attachments

(1 attachment)

(Reporter)

Description

10 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_3; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.1 Safari/525.20
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; ja-JP-mac; rv:1.8.1.13pre) Gecko/20080331 Sunbird/0.8

CalDav provider can also work with iCal server (which is bundled with Leopard server).
However, if you make a big amount of events -- over 1000 events -- then all events disappear from the calendar window.

In the server log (/var/log/caldavd/error.log):
2008-05-14 17:32:06+0900 [-] [caldav-8009] [-] 'Too many matching components in calendar-query report'

The size of query exceeds the specified limit of the iCal server. If I increase the limit (in max_number_of_results = 1000 in /usr/share/caldavd/lib/python/twistedcaldav/method/report_calquery.py), the error does not occur and all events reappear.

This limit is specified in the CalDav specification, accoding to this post:
http://discussions.apple.com/thread.jspa?messageID=7128137#7128137

So please fix the Sunbird to split a big query into a few queries with reasonable size.



Reproducible: Always

Steps to Reproduce:
1. Add iCal server caldav calendar in Sunbird. (http://www.jonathansaggau.com/blog/2007/11/leopard_server_caldav_and_mozi.html)
2. Create over 1000 events in the calendar.
3. All events will disappear.


Actual Results:  
All events disappear from the calendar window.

Expected Results:  
Fix the Sunbird to split a big query into a few queries with reasonable size.

Comment 1

9 years ago
I can confirm this. It is also reproduceable on Windows with an iCal Server running under Linux.
I wonder if this also happens on a PROPFIND, in light of bug 455262. I suspect so, therefore confirming this bug. shouldn't be too hard to split the queries into more than one. Setting dependancy since we want to land the PROPFIND code first.
Status: UNCONFIRMED → NEW
Depends on: 455262
Ever confirmed: true
OS: Mac OS X → All
Hardware: Macintosh → All

Comment 3

9 years ago
This is pretty serious because it is easy to have a calendar with more than 1000 events in a short time if one uses the calendar daily and with various items per day (I got caught by this error and my only workaround is to delete old items which is not very desirable).
Any chance this could be fixed for Lightning 0.10 or we must wait for the 1.0 release?
Thanks.

Updated

9 years ago
Severity: normal → major
Summary: Sends too big query to server if there are many events (over 1000) -- at least iCal server (bundled with Leopard server) can't accept it. All events disappear from the calendar. → Sends too big query to server if there are many events (over 1000) causing all events to disappear
Is there some way the client can find out how many events the max is? This might vary depending on server. Also, we need to know for what type of requests we need to watch for this limit.
simon_at_orcl noted that most clients split the requests into batches of 100. I think we should do this too, maybe we can make the number of events per request a hidden pref.
(Assignee)

Updated

8 years ago
Assignee: nobody → simon.at.orcl
Status: NEW → ASSIGNED
(Assignee)

Comment 6

8 years ago
Created attachment 436082 [details] [diff] [review]
multiget should now be done in batches of 100, configurable with the calendar.caldav.multiget.batch.size hidden preference.

Did some basic tests to make sure the batch retrieval works but it will need to be tested on an apple server with +1000 events.
(Assignee)

Updated

8 years ago
Attachment #436082 - Flags: review?(philipp)
Comment on attachment 436082 [details] [diff] [review]
multiget should now be done in batches of 100, configurable with the calendar.caldav.multiget.batch.size hidden preference.

>+        let batchSize = cal.getPrefSafe("calendar.caldav.multiget.batch.size", 100);
I'd suggest calling it caldendar.caldav.multigetBatchSize, or do you think its likely we will have more multiget-related or multiget.batch-related parameters?


Otherwise r=philipp, I'll leave the final decision on the prefname to you.
Go ahead and check this in any time :-)
Attachment #436082 - Flags: review?(philipp) → review+
(Assignee)

Comment 8

8 years ago
pushed to comm-central with "caldendar.caldav.multigetBatchSize" as the preference name: 
http://hg.mozilla.org/comm-central/rev/e66f19ee877e
(Assignee)

Updated

8 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED

Updated

8 years ago
Target Milestone: --- → 1.0b2
You need to log in before you can comment on or make changes to this bug.