Offline cache should synchronize in the background (hang every 4 minutes)

RESOLVED FIXED in 1.0b1

Status

Calendar
General
--
major
RESOLVED FIXED
10 years ago
8 years ago

People

(Reporter: klint, Assigned: Ludovic Marcotte)

Tracking

(Blocks: 2 bugs)

unspecified
1.0b1
Dependency tree / graph

Details

Attachments

(1 attachment)

(Reporter)

Description

10 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 Creative ZENcast v2.00.14
Build Identifier: Lightning 0.9 20080917 + Gcal Provider 0.5 20080917

I have 7 GCal-based calendars in Lightning, all cached, and Thunderbird hangs for 30s every 4 minutes, with 100% CPU. Which makes it pretty hard to use at my office. 
I have set the calendar.invitations.autorefresh.enabled to false.
I have set the options so that the remote calendars are not refreshed regularly.

What other event in Lightning/Gcal can happen every 4 minutes (by default, I suppose, as I don't remember setting anything to "4 minutes" in the config)?


Reproducible: Always

Steps to Reproduce:
1.
2.
3.
The cached calendars sync every 4 minutes. I remember there being an advanced pref to change this interval, not sure off hand.

Comment 2

10 years ago
It's a calendar property "cache.updateTimer", so you can't change it in the pref.
I am wondering more about why replaying changes (your calendars seem to support changelog) takes so long. This ought to be cheap on average.
(Reporter)

Comment 3

10 years ago
My calendars have plenty of repeting events, occuring every working day. I'll try to deactivate the richest one and check.
But I have a question though: if I set the calendars not to be refreshed automatically(in the options), why would a cached calendar still refreshing/synchronizing every 4 minutes ?
Or maybe the concept of synchronization is not that clear to me :)!
(Reporter)

Comment 4

10 years ago
So, I have deactivated the most complex calendar, and there is still a 100% CPU every 4 minutes, but only for 15s.

Comment 5

10 years ago
Klint, I think it shouldn't. That's one bug for offline-cache imho. Another bug I guess is that reloading calendars shouldn't hang TB. To prevent this the reloading process should be run seperate from TB. This has been mentioned a couple of times in the past.
(Reporter)

Comment 6

10 years ago
Based on all your comments, I have created bug 456208 for the synchro frequency issue.
And for the CPU consumption, wouldn't it be yet another duplicate of bug 456208, y any chance? 
Thanks
I think Bas is more talking about better concurrency, and thus better UI responsiveness while loading/parsing/... is happening. While in general the API and providers work asynchronously, it seems that UI is blocked quite a lot. I am yet not sure how we could improve that.
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows XP → All
Hardware: PC → All
Summary: TB+Lightning+GCal hangs for 30s every 4 minutes → Offline cache should synchronize in the background (hang every 4 minutes)
Duplicate of this bug: 457176
(Assignee)

Comment 9

9 years ago
What's the point of the cache.updateTimer pref and the associated timer to refresh the cache?

We already have the calendar.autorefresh.enabled pref and it'll also take care of refreshing cached calendars.

I think we should wipe this chunk of code from calCachedCalendar.js :

    if (this.supportsChangeLog) {
        var updateTimer = this.getProperty("cache.updateTimer");
        if (updateTimer === null) {
            updateTimer = 4; // override for changelog based providers
        }
        var timerCallback = {
            mCalendar: this,
            notify: function(timer) {
                LOG("[calCachedCalendar] replay timer");
                if (!this.mCalendar.getProperty("disabled")) {
                    this.mCalendar.refresh();
                }
            }
        };
        this.mReplayTimer = Components.classes["@mozilla.org/timer;1"]
                                      .createInstance(Components.interfaces.nsITimer);
        this.mReplayTimer.initWithCallback(timerCallback,
                                           updateTimer * 60 * 1000,
                                           Components.interfaces.nsITimer.TYPE_REPEATING_SLACK);
    }
We assume that replayChangeOn is quite cheap on average (thus the short interval). If caldav cannot determine a smaller changeset, it could override that property, e.g. for instance let caldav's getProperty('cache.updateTimer') return getPrefSafe("ccalendar.autorefresh.timeout").
(Assignee)

Comment 11

9 years ago
We discussed this over IRC but it's not that replayChangeOn() is expensive for CalDAV calendars (especially for those that support ctag:s) but rather because it'll trigger an onLoad() call for each refreshed calendar which will trigger a plethora of databased calls (for cached CalDAV calendar) as mentionned in https://bugzilla.mozilla.org/show_bug.cgi?id=469691 and that is slow.

It's not dependent on the number of items in a CalDAV calendar but rather in the number of subscribed CalDAV calendars one has.

Updated

9 years ago
Blocks: 441710
(Assignee)

Comment 12

9 years ago
Any updates on the discussion we had? I'm willing to make a patch for this bug (either we remove that code entirely or we do what Daniel suggested in comment #10).
Reading the recent comments: Does my suggestion from comment #10 do some cure at all then?
(Assignee)

Comment 14

9 years ago
For CalDAV, yes.
(Assignee)

Comment 15

9 years ago
Created attachment 376229 [details] [diff] [review]
Proposed fix for the CalDAV part
Attachment #376229 - Flags: review?(philipp)
Comment on attachment 376229 [details] [diff] [review]
Proposed fix for the CalDAV part

>--- calDavCalendar.js.orig	2009-05-01 05:41:38.000000000 -0400
>+++ calDavCalendar.js	2009-05-07 08:38:46.000000000 -0400
>@@ -360,6 +360,8 @@
>                 break;
>             case "organizerCN":
>                 return null; // xxx todo
>+	    case "cache.updateTimer":
>+	        return getPrefSafe("calendar.autorefresh.timeout");
>             case "itip.transport":
>                 if (this.hasAutoScheduling) {
>                     return null;

This will certainly postpone the problem a bit, but it will not fix it. Why is the synchronize action so expensive? Is there some way we could make it less expensive instead? What about other providers? Do we have similar problems there? Maybe we should just generally increase the update counter? I'd rather leave this to Daniel to decide, since he initially created the update Timer.
Attachment #376229 - Flags: review?(philipp) → review?(dbo.moz)
Attachment #376229 - Flags: review?(dbo.moz) → review+
Comment on attachment 376229 [details] [diff] [review]
Proposed fix for the CalDAV part

As I was told this is due to the load of database calls. r=philipp for the quick fix, we shouldn't forget about the onLoad issues though!
Keywords: checkin-needed
I don't see a problem increasing the updateTimer to 30 minutes, even coupling it to the auto-refresh pref, because users could customize it.

Pushed to comm-central <http://hg.mozilla.org/comm-central/rev/8d08a1568e8c>

-> FIXED
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → 1.0
This bug has been marked FIXED, but some comments sound like there will also be fixes needed for other calendar providers. Philipp, Daniel?
I think this problem currently only concerns caldav.

Comment 21

9 years ago
I concur with comment #11 above. Note that I have suggest to Philipp a small change to the onLoad event, which would enable changelog-based calendars to declare whether the data has changed or not. But it doesn't solve the fundamental issue of the superfluous refresh in the caldav code.

Comment 22

8 years ago
Here's a bit more input on this issue. I'm using Thunderbird 2.0.0.23 with the Lightning 0.9 and "Provider for Google Calendar" 0.5.4 add-ons to access 10 Google Calendars from Thunderbird. It works, but every 4 minutes Thunderbird stops responding for about 5 seconds. I've tried turning off both "calendar.autorefresh.enabled" and "calendar.invitations.autorefresh.enabled", but the problem continues. I have caching turned on for all the calendars, because without it they kept disappearing, apparently while being reloaded. I was hoping that caching would enable Lightning to avoid this reloading and just continuously display all the calendar data - but it isn't working as smoothly as I had hoped. 

The freeze-up that happens every 4 minutes appears to be caused by the same cache update operation that you all have been discussing here, which has no user-accessible controls. I wonder if it might be worthwhile to revisit comment #9 and consider removing that chunk of code.

Comment 23

8 years ago
Just testing... to see if the system will send me an email this time

Comment 24

8 years ago
I looked into this some more and realized that I needed a newer version of Lightning before I could know whether the fix of comment 18 had already taken care of my Google Calendar issues. And it looks like this fix went into the code for Thunderbird 3, which doesn't yet have a release version. But I found a fixed version that does work with Thunderbird 2. This is Lightning 0.9.6 (Inverse Edition), which incorporates this bug fix along with a number of others. Here's a link to the download site: http://www.scalableogo.org/english/downloads/frontends.html. I find that "Provider for Google Calendar" 0.5.4 also includes an adjustment that allows it to work with this newer version of Lightning.

I've installed Lightning 0.9.6 and played around with it a bit. 
Observations:

- My Google calendars are displaying just fine
- Thunderbird isn't freezing up every 4 minutes now
- As an experiment I set Calendar.autorefresh.enabled to "True" and set the timeout to just 1 minute, expecting that now every minute I'd get the Thunderbird freeze-up that I previously got every 4 minutes. But the refresh is no longer causing any noticable slowdown.

Thank you for the good repair work you all did.

I'm really glad to have found this. The cache update every 4 minutes had apparently been impacting the Google server as well my computer. I had begun getting locked out of my Google Group with the server complaining that my computer was sending automated requests that violated my service agreement. I don't know for sure that it was related to this Lightning issue but suspect so, because the Google Group issue went away when I disabled Lightning for a period of time last week.
Assignee: nobody → lmarcotte
Target Milestone: 1.0 → 1.0b1
Blocks: 462277
You need to log in before you can comment on or make changes to this bug.