Open Bug 1683321 Opened 5 years ago Updated 2 years ago

Loss of events and tasks w/reminder

Categories

(Calendar :: General, defect)

Product:
Calendar
Component:
General
Version:
Thunderbird 78
Type:
defect

Tracking

(Not tracked)

Status:
UNCONFIRMED

People

(Reporter: MozBug20, Unassigned)

References

Details

Attachments

(5 files)

reminders window as of 2021-10-14 14:01 (i.e. 2:01 p.m.)
85.38 KB, image/png
Details
reminders window as of 2021-10-14 14:01 (i.e. 2:01 p.m.) tasks marked in orange
102.62 KB, image/png
Details
TB Cal has no reminders - MS cal does (same source)
327.23 KB, image/jpeg
Details
Event retained as of November 06th, 2021
83.71 KB, image/png
Details
Animation.gif
1.61 MB, image/gif
Details

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0

Steps to reproduce:

Nothing (now).
Created events and tasks earlier (sometimes years earlier).

Actual results:

Events and tasks, which are due, are deleted from calender resp. task tab after some time. I don't know exactly when.
This occurs both for non-recurring as well as for recurring events and tasks.
I already lost tens of tasks and events.
Problem probably first occurred with Thunderbird 78.5.0 or 78.5.1.

Example:
A few days ago i created an event for yesterday 5 pm with reminder 1 week before using Thunderbird 78.5.1. I never did close the notification manually. Today event is deleted from calender as if it never had existed.
Possible trigger for deleting may have been quitting Thunderbird and restarting today.

I have a backup of Thunderbird profile (.zip) from some weeks ago. Is there a way to figure out diffs of events and tasks from then to today?

Expected results:

Events and tasks should never be deleted automatically

(In reply to MozBug20 from comment #0)

A few days ago i created an event for yesterday 5 pm with reminder 1 week before using Thunderbird 78.5.1.
Added quotes and comma to make more clear:
A few days ago i created an event for yesterday 5 pm with reminder "1 week before", using Thunderbird 78.5.1.

I'm using german version, therefore actual wording of reminder may be a bit different.

Likely fixed in bug 1664731 (will likely be in 78.7)

Bug reporter, can you please check the behavior with the most recent release, Thunderbird 78.9.0?

Flags: needinfo?(MozBug20)

(In reply to Martin Schröder [:mschroeder] from comment #3)

Bug reporter, can you please check the behavior with the most recent release, Thunderbird 78.9.0?

Last loss of events and tasks occurred with TB 78.8.0.
Typically i keep Thunderbird open for days, using Windows hibernate in between.
Yesterday i did quit Thunderbird and reboot the machine.
Today i did launch TB78.8.0: Five reminders where shown (one for today's evening, 4 which were due in the past).
Created screenshot of reminders.
Did not touch (i.e. close or defer) any of the reminders.
Quit TB78.8.0 again.
Lauch TB78.8.0 again:
No reminders where shown anymore.
Did check events and task of sceenshot above and found, that all of them have been deleted.
Some of them had existed since years (recurring once a month), some of them had been created a few days ago.
Both recurring as well as non recurring events and tasks have been deleted.

Then did update to TB 78.9.0.
Could not yet find any additional loss of data, probably because there are no reminders.

It seems, that loss of events and tasks is triggered either by quitting Thunderbird or by launching Thunderbird.

Flags: needinfo?(MozBug20)

(In reply to Martin Schröder [:mschroeder] from comment #3)

Bug reporter, can you please check the behavior with the most recent release, Thunderbird 78.9.0?

Once again lost multiple events and tasks with TB78.10.0
Updated from TB78.9.0 to TB78.10.0 on April, 20th 2021.
Update did not corrupt anything.
7 reminders where shown on April, 23rd.
Created screenshot of reminders.
Quit TB and reboot the machine on April, 24th.
Start Thunderbird on April, 25th.
Only 1 reminder shown.
Checked tasks and events for missing 6 events compared to screenshot above.
6 events and tasks have disappeared, i.e. have been deleted.

In my opinion this is a prio 1 bug.

I have the same problem reported in "Bug 1702982 Calendar losing events". I use Ubuntu 18.04 LTS and I reboot daily.
I agree, this is a very serious bug.

People in Support Forum also reporting this problem.
Several people reported same issue : https://support.cdn.mozilla.net/de/questions/1326232
https://support.mozilla.org/de/questions/1333078
https://support.mozilla.org/en-US/questions/1317218

People seem to say that when they get a pop up reminder and they select to dismiss, it is deleting the event
OR closing the pop up reminder window and not select anything , it is deleting the event
Or getting the pop up reminder not selecting anything nor closing pop up, but exited Thunderbird and event is deleted.
This is quite specific and seems related to that pop up reminder.
Early reports say it started in version 78.5.0 and it is still occuring in 78.10.0

I'll try to hunt down other reports to gather confirmation and see if this deleting issue is related to the pop up reminder.

Hi, I have the same problem and I wrote already some information here:
https://support.mozilla.org/en-US/questions/1333078

The last time, the problem occurred was on 2021-04-25. At that date, I used the Thunderbird version 78.10.0. Today I use 78.10.1 and the problem did not occur with this version yet. The last deleted events are:
Day of deletion: 2021-04-25
Event: 19. April
Event: 22. April
Event: 22. April
Event: 27. April

I use this system:
Thunderbird version: 78.10.1
Portable: yes
OS: Windows 7
Language: German
Time zone: Berlin
Location of calendar: local

My installation of Thunderbird-Portable is in a VeraCrypt container. Everything is encrypted, but I think this is not the problem. I used TrueCrypt and now VeraCrypt this since several years without any problems.

I think, the problem is not related to the reminder popup or the actions I do with the events in the reminder popup. Sometimes I did nothing and the events are gone the next time I start Thunderbird. I do usually this: start the computer, start Thunderbird, look at the reminder popup, leave it visible, if all events are in the future, I do nothing, if an event is obsolete, I click close on this event. At the end, I close Thunderbird and the reminder popup and shut down my computer.

I never use snooze. My wife has the same problem and almost the same system (e.g. Win7). She never used snooze.

The bug reporter wrote he is using the german version like me. Perhaps this is an idea. But here are some screenshots looking like an english version:
https://support.cdn.mozilla.net/de/questions/1326232

I think, this is a very serious bug if events are deleted with no way back.

(In reply to MozBug20 from comment #5)

On April 27th, i did the following:

  • Quit TB 78.10.0
  • Backup folder calender-data from TB profile
  • Delete folder calender-data from TB profile
  • Restore folder calender-data from backup as of April 20th
  • Launch TB 78.10.0

6 events and tasks, which where lost in comment#5 above, are back again.
Reminders for these 6 events and task are shown in popup window.
Obviously these reminders have not been deleted again during launch of TB this time.
Then did set calendar containing these events and tasks to read-only to avoid losing them again.

After quitting and launching TB again, reminders for read-only calender are not shown, see bug 1707558.

On May 7th did quit TB78.10.0. (I always use menu File->Quit, which is also closing reminders popup window; i never close reminders popup window manually and never use dismiss all.)
At this time 5 reminders had been show in reminders popup window. (created sceenshot)

When launching TB78.10.0 on May 8th, 3 reminders are shown in reminders popup window. (created screenshot)
One of them had not been shown earlier. Thus 3 reminders have disappeared.
Used screenshots to checking events and tasks for missing reminders.
Once again 3 events/tasks have been lost, as if they never had existed before.

I have created several 7z archives of TB profile folder for backup purposes.
Over time i lost lots of reminders for e.g. birthdays, initially without notice.
Is there some way to figure out, which events/tasks have been lost?
Obviously restoring older calender-data as in comment#9 above would result in loss of events and tasks created or modified since then.
So a merge/diff would be needed.

Is there an official way to backup/restore calendar data?

Since i'm aware of data losses, i regularly created export of calendars to .ics and .csv.
However diff of export data shows tons of differences, probably due to internal updates of some time fields.
Also it seems ordering of events/tasks in export files varies.
So this does not help either.

Hi MozBug20,

to backup and diff my events I do this:
create a folder like yyyy-mm-dd_hh-mi-ss
copy the file local.sqlite as backup to this folder
export the events to a csv file with this command:
sqlite3.exe -header -csv local.sqlite "select cal_id, id, title, datetime(event_start/1000/1000, 'unixepoch') as ev_start, event_start_tz from cal_events order by cal_id, title, ev_start, id;" > export.csv
compare the new csv file with the last one.

You can find the file sqlite3.exe in the Thunderbird folder or in the web.

Perhaps this is a solution for you.

Not sure if this is the same bug. I have a problem with reminders that fail to reappear after snooze.

I'm on archlinux, fluxbox window manager, thunderbird 78.8.1

An example of this bug occurred this morning:

  1. A reminder dialog appeared with three events listed
  2. I selected "snooze all" for 2 hours
  3. 2 hours later the reminder dialog reappeared, but with only two of the previous events listed

In this case, 2 of the events were recurring and one was not. The reminder that failed after snooze was one of the recurring events.

I don't shut down thunderbird, or my computer on a daily basis. I restart thunderbird once a week or so, since it seems to leak memory.

Calendars and reminders are because people need help remembering things. If the computer forgets, it sort of defeats the purpose. Please focus the Mozilla debug effort on functional issues (like this one, or not downloading IMAP folder contents) instead of UI modifications or framework migrations.

johnea

(In reply to johnea from comment #13)
From my point of view, the essential question is: What happened to the third event?
a) Has it been deleted from calendar? Then this probably is the same bug.
or
b) Is the event still there in calendar, but no reminder is appearing anymore? Then this probably is a different problem.

(In reply to MozBug20 from comment #14)

(In reply to johnea from comment #13)
From my point of view, the essential question is: What happened to the third event?
a) Has it been deleted from calendar? Then this probably is the same bug.

No, the event is still there. This specific example was a weekly recurring event. The remaining events in the reminder have been dismissed, or continued to be snoozed, without issue. While the lost reminder has still not recurred.

b) Is the event still there in calendar, but no reminder is appearing anymore? Then this probably is a different problem.

In other occurrences of this issue, the lost reminder does not affect the next weeks reminder.

In other occurrences of this issue, these lost reminders will sometimes pop up at some unexpected time in the future.

It seems a tool to browse/inspect the event/reminder database would be helpful. Perhaps an enhancement to the field in each event that displays it's reminder status, e.g.: reminder 1 day in advance, currently snoozed until [exact date/time].

(In reply to johnea from comment #15)

(In reply to MozBug20 from comment #14)

(In reply to johnea from comment #13)
From my point of view, the essential question is: What happened to the third event?
a) Has it been deleted from calendar? Then this probably is the same bug.

No, the event is still there. This specific example was a weekly recurring event. The remaining events in the reminder have been dismissed, or continued to be snoozed, without issue. While the lost reminder has still not recurred.

b) Is the event still there in calendar, but no reminder is appearing anymore? Then this probably is a different problem.

In other occurrences of this issue, the lost reminder does not affect the next weeks reminder.

In other occurrences of this issue, these lost reminders will sometimes pop up at some unexpected time in the future.

It seems a tool to browse/inspect the event/reminder database would be helpful. Perhaps an enhancement to the field in each event that displays it's reminder status, e.g.: reminder 1 day in advance, currently snoozed until [exact date/time].

The bold in #15 was unintentional, I just tried to enter some dashes to separate my response to teh reply from additional comments.

Sorry to jump in so late but I just ran across this thread while trying to figure out how to reopen a closed bug report. I opened bug 1659250 in bugzilla sometime back and we thought this had been corrected so bug was closed. I am running Win 10, TB 78.12.0. I have been pushing all tasks into the future each night then shutting down TB due to the problems I have been having. I close TB each night using File-Exit and without closing the reminder window. The tasks mentioned below are regular tasks rather than recurring. Here is what I am seeing in my testing since updating to latest release:

7/18/2021: Updated TB to 78.12.0.

07/19/2021: Added Test task #1 then shut down TB for night.

07/20/2021: Started TB and Test task #1 still there and shows in reminder window. Added second task backdated to 6/20/2021 with reminder and it immediately shows in reminder window.

07/21/2021: Started TB and both tasks still there but the backdated task does not show in the task reminder window even though it shows green in the task tab. Changed the date on it to 6/21/2021, saved it and it immediately popped up in the reminder window.

07/22/2021: Started TB and the same thing happened. The 6/21 task shows in green in task list but doesn't show up in reminder window. The 7/19 task shows in the reminder window. 30 days appears to be the point at which it quits showing in the task reminder window?

07/24/2021: Did not update the backdated task (6/21) the past two days and it continues to show in the task list as green but doesn't show in reminder window. The task from 7/19 continues to show in reminder window.

07/29/2021: Started TB this morning and the task from 6/20/21 still shows in the task list and not in the reminder window. However, The task from
7/19/2021 is gone completely! Not in reminder window and not in task list. I had been leaving the 7/19 task in the reminder window each evening when I shut down and until this morning it has worked correctly. So much for my 30 day theory.

Like others, I have no idea whether we are losing the tasks on shutdown or on startup but my gut feeling is startup based on some of the prior testing I did. You can refer to bug 1659250 for a detailed description of the testing I've done since this problem first appeared for me at the end of 2020 and beginning of 2021.

I'm done. This is the last time I get hosed with T'Bird Calendar. I've moved on to a more reliable application.

Issue also happens with CALDAV calendars where other clients (I am trying Windows 10 Calendar) work OK, TB calendar losses the reminders, usually when restarted.

Refreshing the calendar data from the CALDAV server doesn't help.
Same issue when using TBSync with Provider for CalDav and CardDav Extensions.

This issue seems to be at the heart of TB calendar rather than any specific method in creating calendar entries or data sources

How do we get the TB Calendar team to look at this? (I'd be happy to share screen grabs of my CALDAV server data, Windows calendar and TB for the same event if needed)

I echo graham's comment about being happy to provide any information and/or testing to help the TB Calendar team solve this issue. It really is a nuisance since I must continue to reset unfinished tasks to a future date just to make sure they don't disappear. Let us know what we can do to help!

Once again lost 13 events and tasks.
Attachment shows reminder window as of October, 14th 2021 at 02:01 p.m.
(Texts removed for privacy reasons.)
Current TB version is 78.14.0.
As you can see, there are reminders which are due in the past as well as in the future.

What i did:
Quit Thunderbird at about 04:00 p.m. using menu item File->Quit.
At about 06:02 p.m. did launch Thunderbird again.

What happened:
None of the reminders did appear again.
Checking tasks and events shown in screenshot showed, that all of these events and tasks have been removed from calender as if it never had existed.

Nobody taking care of data loss?
Calendar effectively is unusable.
I wish integration of calendar would be rolled back to TB version 68.
As long as calendar had been an add in, it worked fine!

MozBug20 could you clarify something for me? Which ones above were tasks and which ones were calendar events? I've not had any problems with calendar events (that i remember) but the disappearing task issue has been an ongoing problem. I have continued my testing and haven't lost any tasks since my reporting from July of this year. I may need to leave an open calendar reminder to see if that has any impact. I keep hoping I can make this fail on a regular basis but so far no luck.

Same reminders window as before, however tasks are marked in orange.
Rest of the entries are calendar events.

Maybe also of interest:
I selected "Erinnerung: 1 Woche vorher" (english probably: "Reminder: 1 week before") for all of the events and tasks shown in this reminder window.

First task on screenshot has
Start: Samstag, 25. September 2021 17:00
(not visible on screenshot as first row is selected and therefore output format is different)

Did manually recreate lost events and tasks.
Quit Thunderbird on October 18th, 2021 at about 6:55 p.m.
Launched Thunderbird again at about 9:10 p.m.

Once again all events and tasks, which had been shown in reminders window are lost.

(In reply to MozBug20 from comment #25)

Did manually recreate lost events and tasks.
Quit Thunderbird on October 18th, 2021 at about 6:55 p.m.
Launched Thunderbird again at about 9:10 p.m.

Once again all events and tasks, which had been shown in reminders window are lost.

For some people after an update to version 91*, they suddenly experienced an empty Home default TB calendar and when they tried to add events, they would remain visible until they exited Thunderbird. Upon restart all were gone. However, the 'local.sqlite' file still had a decent file size, so this was a read file and display issue and not a loss of data.
People have been testing a fix, most successfully where it was discovered that a null file was causing a problem.

As you seem to be displaying the same sort of problem, just in an earlier version, maybe the same fix could be tested. You may be one of the few experiencing this issue prior to 91*.
Please read information at this link and report back on the results.
https://support.mozilla.org/en-US/questions/1352939

Flags: needinfo?(MozBug20)

SELECT * from cal_events WHERE id IS NULL LIMIT 100
displays
Empty

BTW, there are 6019 events and 3421 alams in my local.sqlite file.

Flags: needinfo?(MozBug20)

Side by side screen garb of TB calendar event vs MS calendar event. Data source is CalDav Synology Calendar server.

MS always works fine, but TB randomly does not show reminders

(In reply to MozBug20 from comment #25)

Did manually recreate lost events and tasks.
Quit Thunderbird on October 18th, 2021 at about 6:55 p.m.
Launched Thunderbird again at about 9:10 p.m.

Once again all events and tasks, which had been shown in reminders window are lost.

(In reply to MozBug20 from comment #24)

Created attachment 9246304 [details]
reminders window as of 2021-10-14 14:01 (i.e. 2:01 p.m.) tasks marked in orange

Same reminders window as before, however tasks are marked in orange.
Rest of the entries are calendar events.

Maybe also of interest:
I selected "Erinnerung: 1 Woche vorher" (english probably: "Reminder: 1 week before") for all of the events and tasks shown in this reminder window.

First task on screenshot has
Start: Samstag, 25. September 2021 17:00
(not visible on screenshot as first row is selected and therefore output format is different)

Thank you. I will add some events into my testing and see if I can make them fail.

(In reply to MozBug20 from comment #25)

Did manually recreate lost events and tasks.
Quit Thunderbird on October 18th, 2021 at about 6:55 p.m.
Launched Thunderbird again at about 9:10 p.m.

Once again all events and tasks, which had been shown in reminders window are lost.

After that, i once again manually recreated lost events and tasks.
This time no more tasks and events have been lost for about two weeks.

To be continued...

On Nov 6th, 2021 used menu item File->Quit, when about 13 reminders where shown in reminders window.
Launched TB 78.14.0 again, some hours later.

All events and tasks, which had been shown in reminders window before, are lost (as usual), except for one task, see screenshot.
What is special about this task: I had assigned an end date.
Usally i never assign an end date.

Hello,

I have a specific scenario here where i can reproduce the problem very consistently:

Thunderbird 91.6.0 Portable
Windows 10 Pro
CalDAV calendar accessing an Office 365 calendar via DavMail Gateway and EWS protocol.

  1. Start Thunderbird
  2. Open Calendar and observe the event "Conversando" (see attached gif) is present on both Outlook Web and Thunderbird (ok, synchronized).
  3. Right click on calendar and execute "Synchronize Calendars"
  4. After the execution of this command, the events flashes (seems "normal") but then suddently the event "Conversando" gets deleted.
  5. Close and start Thunderbird again. After the startup sync (auto), the event is added back on the calendar.
  6. If "Synchronize Calendars" is executed again, the event gets deleted again.

During all this process, the event remained untouched on Outlook Web.

I have attached a gif (screen recording).

What kind of information (logs) could i provide for you to investigate this problem?

Attached image Animation.gif —

Screen recording showing the event being deleted after a calendar sync.

(In reply to Felipe from comment #33)

Created attachment 9266313 [details]
Animation.gif

Screen recording showing the event being deleted after a calendar sync.

From my point of view, this is a different problem. Therefore i suggest you create a separate bug report.

This bug is about ultimately losing tasks or events, having a reminder shown.
Task or event typically is lost when quiting TB und starting again.

None of these is true for comments #32 and #33.

(In reply to MozBug20 from comment #34)

(In reply to Felipe from comment #33)

Created attachment 9266313 [details]
Animation.gif

Screen recording showing the event being deleted after a calendar sync.

From my point of view, this is a different problem. Therefore i suggest you create a separate bug report.

This bug is about ultimately losing tasks or events, having a reminder shown.
Task or event typically is lost when quiting TB und starting again.

None of these is true for comments #32 and #33.

Ok, actually i was just following the instructions given by the contributor Toad-Hall on this thread:

https://support.mozilla.org/en-US/questions/1326232

He asked to provide information if the user notices any kind of event loss (event is spontaneously deleted) after a reminder popup... which is exactly the case on this screen recording.

It might be a different bug, but we need some direction from the support/developers to collect and provide the proper information for investigation.

(In reply to Felipe from comment #35)

(In reply to MozBug20 from comment #34)

(In reply to Felipe from comment #33)

Created attachment 9266313 [details]
Animation.gif

Screen recording showing the event being deleted after a calendar sync.

From my point of view, this is a different problem. Therefore i suggest you create a separate bug report.

This bug is about ultimately losing tasks or events, having a reminder shown.
Task or event typically is lost when quiting TB und starting again.

None of these is true for comments #32 and #33.

Ok, actually i was just following the instructions given by the contributor Toad-Hall on this thread:

https://support.mozilla.org/en-US/questions/1326232

He asked to provide information if the user notices any kind of event loss (event is spontaneously deleted) after a reminder popup... which is exactly the case on this screen recording.

It might be a different bug, but we need some direction from the support/developers to collect and provide the proper information for investigation.

The link in forum says:

If the Event seems to get deleted if one of the following occurs:
You get a pop up reminder and you select to dismiss
You get pop up reminder and you just close pop up without selecting anything
You get a pop up reminder which you ignore, leave visible and exit Thunderbird.
If you get a pop up reminder and make any other selection like snooze for a day and it deletes the event...
If you have any information, please report what OS you use, what version of Thunderbird, whether it is portable version or not.
What you selected with that pop up reminder.
Whether this is just Lightning home calendar or whether you are using CalDav etc.
All help in nailing down this specific bug would be appreciated.

I would say it was very specific about the Reminder pop up must occur and loss of event.
When you create event, you can set a reminder to occur and it appears as a pop up which you can choose various options like to remind you in x number of minutes or dismiss etc. I do not see any pop up reminder about event in your example.
You appear to have an anomally occurring with synchronisation with outlook office calendar which is not the same problem.
TRy asking a question in the Thunderbird Support Forum to see if someone has already come across this issue and can offer help.

Ok, i was finally able to isolate the cause of the problem i reported on Comment 32. In fact it seems to be a different issue. I will report in a new bug.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: