732393 - Cannot edit or delete task in cached CalDAV calendar [Error: this.mItemInfoCache[aNewItem.id] is undefined]

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect)

Description

[TGOS]

When selecting the task and trying to delete it, the error console prints:

Error: this.mItemInfoCache[aItem.id] is undefined
Source File: file:///Users/xxx/Library/Thunderbird/Profiles/lujdvccv.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 869

When I try to edit the task in any way, I get:

Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///Users/xxx/Library/Thunderbird/Profiles/lujdvccv.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 696

Comment 1

[Stefan Sitter [:ssitter]]

Did it work in previous releases? Does it work in most recent beta or nightly test builds?

Please provide more detailed information that can help developers to reproduce the problem in a fresh profile, including other information like type and version of CalDAV server, etc.

In addition you could go to Thunderbirds advanced preference editor and enable the calendar.debug... preferences. This should provide more information from CalDAV provider in Error Console.

Comment 2

[TGOS]

I never had a task to delete in any previous release and no, I cannot downgrade this system to test that (this is a production environment). The most recent beta or nightly doesn't work with my version of Thunderbird any longer and when being used with a nightly build of Thunderbird as well, they behave absolutely identical in really every aspect (including this bug).

No, I cannot create a fresh profile (again, this is a production environment, especially the CalDAV server, which is crucial for the company I work at) and use it with this CalDav Server, which is an iCal Server, version unknown, OS X Version unknown (Snow Leopard or Lion, I suppose).

There is no option calendar.debug, however there is calendar.debug.log and calendar.debug.log.verbose and after setting both to true, and even restarting Thunderbird, the console log output looks exactly as in my initial report.

-----------

Was anything above really helpful? I don't think so. Sorry, but I think the information in my initial report were more than adequate to do something about this bug. This is a very simple bug: Don't access something that may be undefined, verify before accessing it and if undefined, either define it or work without that information. Form the perspective of a software developer, I'm very disappointed by the Lightning team so far. Every bug I report only leads to plenty of usually pointless questions, I'm getting pushed around to try old versions, new versions, beta versions, and try out other obscure pseudo-fixes or provide information, totally unrelated to the bug in question. I'm a long term bug reporter, my first bug on bugzilla.mozilla.org dates back to 2002-07-09(!) and yes, I'm a little bit pissed that Lighting bug reports only receive replies, that are intended to pass back the ball to the reporter, instead of ever showing any progress on the actual issue. I saw this happening with several other Lighting bug reports, too. This is not the professional level of bug tracking I'm used from Firefox.

Comment 3

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Mohit, seems this bug is back :-(

TGOS: Stefan was suggesting you create a new profile to see if it happens with a fresh profile too. Creating a new profile will not harm your production profile and you can still use it.

Fixing this bug is unfortunately not that easy. Of course we could make the error message go away, but this won't fix the bug. Other things will break.

If you want to make this go away, please delete the cache.sqlite in your profile/calendar-data. This will make the cache items resynchronize and the error will go away.

I can't guarantee this bug won't reappear afterwards, but if you have been using subsequent versions of Lightning, the initial bug may have happened with a previous release. We fixed something that might make this not reappear, but it would be nice to find out about that.

So in short, specifically for your situation

1. delete you cache
2. use your calendar
3. if it shows up again, let us know.

--

I'm sorry this bug has bitten you and you have had such a negative experience with filing bugs. On the other hand, you have to understand us. We do the same for each bug reporter and confirming bugs is not something we can do alone without spending less time fixing bugs and more time confirming them.

If the user is using an older version, it may have been fixed in a newer build. If the user is using the newest version it would help very much to find the regression range, which is why we might ask to test an older version. As a software developer, I would have hoped you know this situation.

The more information we get, the better we can reproduce the bug. For example, when I repeat the steps you posted in comment 0, I don't get those error messages. Without getting further information, how do you expect me to get rid of the bug?

We are not asking these questions to be annoying, but to find out what situation causes the bug.

Comment 4

[Mohit Kanwal [:redDragon]]

Hi TGOS,

Phillip had tried to fix this issue in a previous version via bug 700637. You can follow the discussion over there where it was usually occurring during the processing of recurring events. 

Looks like it is happening now with tasks :(

Could you provide information, or just a scenario, where we can isolate the issue? In the case of bug 700637 we had managed to reproduce the issue by fiddling around with the system clock. 

Maybe it happens only with recurring tasks. or something like that. I am, in the meanwhile trying to test out a new build to see if I can reproduce this issue with tasks the way I managed to do it in bug 700637.

Comment 5

[TGOS]

@Phillip:

Okay, I quit Thunderbird, deleted cache.sqlite (not local.sqlite), and started Thunderbird again. After a short time, the task came back, so this task was in fact never deleted on the calendar server. I selected it and deleted it, this time without an error message. I restarted Thunderbird to make sure it won't come back and it didn't, so it was deleted locally and on the server. This makes me think, the issue is only related to a local cache "corruption"; though corruption is maybe a bit "harsh", it might just be that the local cache is out of date.

For testing purposes, I created three other tasks: One in the future, one in the past, one without any date; all three for the same calendar on the same calendar server that caused the problem before. I restarted Thunderbird again and tried to delete all three tasks. I was able to delete all three without an issue. So this is not a reproducible issue and thus most likely not related to my calendar server.

As a developer I see two bugs here:

  1. The bug that causes cache database corruption. I have no idea what causes this bug but fixing this bug will fix the "real issue". The problem is that once the database is already corrupted, there is no way to find out which past action caused this corruption. I understand that this is no easy bug to fix. It could be a cache update issue, like the cache not being updated correctly or only partially being updated, causing an inconsistent view on the data in question.

  2. The bug that once the database is corrupted, there is no way to recover from that state. You said making the error message go away will break other things but what if you just catch this error and handle it by "invalidating" the cached data for this "object"? In that case the data is re-fetched on next sync and the corruption will most likely go away, just as it did when I deleted the cache database. This does not solve the "real issue", it's just a pain killer for the symptoms, however, sometimes such pain killers are necessary, since if you have some headache, it's often much more important to get rid of it than it is to find out what the real cause of the headache is. Looking at the fix of bug 700637, it does exactly what I suggested: It verifies the value is defined and otherwise uses a fallback "method" if it isn't, as well as catching an exception where necessary. So the fix for 700637 does not really "solve" the problem (which may be server side and thus unsolvable), it also just works around the broken state.

I'm sorry about my harsh comment before, I know how "painful" bug tracking can be, especially in situations, where a bug keeps reoccurring every now and then but there is no way to reproduce it at will. However, I'm a software developer and I'm also a part time software support agent, and if I was telling a customer to go back to a previous version or try the latest beta version, I knew that I'm basically just pushing him around to buy us some more time, since the chances that the latest beta fixes this bug and I'm unaware of that fact are close to nonexistent and the chances that the bug was introduced with the last release are almost equally small. Also having the user jump between versions often causes more harm than it does any good, since internal data formats, database or file layouts may have changed and data is converted forward and backward, and in the end the state is even more corrupt than it used to be. 

That you cannot reproduce the bug is not very surprising to me. I guess if this would happen anytime for everyone, I could find thousands of people reporting this bug here. And if I could repeat this bug every time, I had certainly mentioned that in my initial bug report. I don't know CalDAV/iCalendar very well and to be honest, I think these two standards are very poorly designed, considering how relatively "new" they are, one could have done much better. But reading bug 700637 I get the impression the problem is related to each item (Event/Task) having an ID and this ID is used as an identifier in the SQL database, however, this ID might change on the server (for whatever reason) and this simply causes things to break. And if the standard allows the ID to change and Lightning is not taking this into account, assuming it never changes unless Lightning itself changes it, then this is a serious design flaw.

----

@Mohit:

The task was not reoccurring, it had no start or due date, no category, no location, no reminder, no description, and the status was not specified either. Basically I created that task a while ago, just giving it a title and a couple of weeks later I wanted to delete it. The task was in a private calendar, so no other user could have edited it, and I'm only accessing this calendar from one single computer. That means unless the server has "modified it on its own", only my local copy of Thunderbird could have modified it.

Comment 6

[Tom Louwrier]

This looks very much like the issue I'm experiencing. It may be related to 769118, I posted a comment there too.
It's not confined to recurring events, but also dismissing alarms that pop up, both for missed events and current ones.

I get this about 23 times in the log after I click 'dismiss all' once.

Timestamp: 11-12-12 10:51:13
Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///usr/lib/mozilla/extensions/%7B3550f703-e582-4d05-9a08-453d09bdfdc6%7D/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 660


Running Tbird 17.0 / Lightning 1.9 on Ubuntu 12.04 amd64.
My CalDAV server is a Sogo one, current version. Using the server's web interface to dismiss, delete or edit the events does not solve the situation in Tbird/Lightning.

Switching off local calendar caching, restarting Tbird and enabling caching again (need to be able to work offline) clears the hanging events. After a while one comes up that can not be dismissed and then all following events will join the queue of missed events that can not be dismissed.


cheers
Tom

Comment 7

[enzomas]

Same symptoms for me since I upgraded to lightning 1.9 (with TB 17.0).

This bug affect my calendars on different servers (some on MacOSX server, some others on Sogo).

Steps to reproduce (tested with a brand new installation/profile) :
- set a new calendar on a caldav server (offline mode disabled)
- create a new event
- enable offline support for this calendar
- refresh calendar
- restart TB
- trying to edit or delete the previous event is impossible :
(Error : this.mItemInfoCache[aNewItem.id] is undefined
in file:///C:/Users/xxxxxxx/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Ligne : 660)

Disabling offline support (or deleting cache.sqlite) is the only way to modify/delete the event again.

Comment 8

[Mohit Kanwal [:redDragon]]

Hi enzomas, 

I suspect this might be happening due to bug 742528. If you are able to get your hands dirty with code, can you try editing 

~/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js 

and attempt to add the lines that were removed in the patch for bug 742528. 

Check if this solves the problem.

Comment 9

[enzomas]

(In reply to Mohit Kanwal [:redDragon] from comment #8)
> Hi enzomas, 
> 
> I suspect this might be happening due to bug 742528. If you are able to get
> your hands dirty with code, can you try editing 
> 
> ~/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-
> 4020-b5ad-a41df1933103%7D/components/calDavCalendar.js 
> 
> and attempt to add the lines that were removed in the patch for bug 742528. 
> 
> Check if this solves the problem.

In Lightning 1.9, these two lines are still present in calDavCalendar.js.

Comment 10

[Christoph Kaulich]

This bug is still true for lightning 2.1a2.

The two lines mentioned in comment #8 are still there, but setting a task to completed works only for online calendar for me.

Also resetting the cache from the right mouse button leads to a crash. Maybe this is connected.

Comment 11

[Greg K.]

I encountered this condition attempting to edit an event. It occurred after Thunderbird crashed for another, unrelated reason (sending a message). Resetting the calendar cache via the contextual menu on the calendar itself resolved it.

Does the cache maintain itself in a way that an application crash won't corrupt it?

Should the cache be automatically reset after a Thunderbird crash? Is it?

Comment 12

[ingo.goeppert]

Same problem on my installation, TB 17 ESR with lightning 1.9 and zarafa as server.

I stopped TB and deleted the cache.sqlite. Then I can create new events and edit the old events. After I restarted TB several times and created a new event in Outlook, the problem happens again.

Lightning is unusable for me at the moment, because reminders pop up again and again. I have to switch off and on the cached mode. Very frustrating.

What can I do to help to fix the bug? Generate logfiles? How?

Comment 13

[Chris Davies]

TB 17.0.2, Lightning 1.9, Davmail 4.1.0-2042 talking to Exchange. All running on Debian Linux but with the standard Mozilla installation packages.

I do not get this problem for events that I have created myself. I regularly get this error for events created by others to which I have been invited. The Davmail debug log shows no attempt by Lightning to talk CalDAV to it.

TB console shows this:

Timestamp: 09/01/13 11:26:41
Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///home/chris/.thunderbird/0bm31s7m.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 660

My calendar is not read-only. Usually the calendar is marked as available for offline support. The problem appears to go away when I untick the offline support flag.

Going back into the calendar properties I notice it's switched on again (possibly a separate fault?); at this stage I cannot determine whether it is the flag display or the flag setting that is incorrect.

Comment 14

[marc]

I have developed a Sabre based Caldav/Carddav Server 
which supports webdav-sync(RFC6578) AND getctag(caldav-ctag-02)! 

I can confirm the bug for TB 17-ESR/Lighning 1.9/SOGo Connector 17.0.2!

I discovered that lightning uses a combination of getctag and webdavsync if the server supports webdav-sync!

If I DISABLE the getctag Property on the Server the bug is gone ...
every refresh is done by webdav-sync 
and the cache-data seems to be updated and stable after Restart/Reboot!

Comment 15

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Marc, could you check if this bug still occurs with the patch for bug 827078 applied?

Comment 16

[marc]

After serveral days of testing I can confirm:

the patch for bug 827078 solves the problem!


Great Job - Thanx!!!!

Comment 17

[Stefan Sitter [:ssitter]]

Thanks for testing. I'll resolve this problem as fixed for 1.9.1 by Bug 827078.