Closed Bug 1666871 Opened 6 years ago Closed 4 years ago

MODIFICATION_FAILED and DAV_NOT_DAV errors accessing remote calendar

Categories

(Calendar :: Provider: CalDAV, defect)

Product:
Calendar
Component:
Provider: CalDAV
Version:
Thunderbird 78
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1750655

People

(Reporter: psychonaut, Unassigned)

References

(Regressed 1 open bug)

Details

(Keywords: regression)

I use Thunderbird to access a remote calendar on a Radicale server. This recently stopped working; whenever I try to sync to/from the calendar, Thunderbird pops up an error dialog with the message MODIFICATION_FAILED. Below is an anonymized version of what is printed to Thunderbird's error console when calendar.debug.log and calendar.debug.log.verbose are enabled.

The problem may be connected to a recent upgrade to Thunderbird 78.2.2. I don't remember the problem occurring before upgrading yesterday. I have two other machines, one running Thunderbird 78.2.1 and the other running Thunderbird 68.11.0 with Lightning; both of those installations are configured to connect to the exact same remote calendar, and they don't have any problems reading and writing to it.

The affected machine is running openSUSE 15.2 (x86-64). The problem occurs with both the openSUSE 15.2 Thunderbird package and the official Thunderbird binaries, and also in safe mode.

Lightning: [calCachedCalendar] Performing playback operation add on 0 items to psy calCachedCalendar.js:647
Lightning: [calCachedCalendar] Performing playback operation modify on 0 items to psy calCachedCalendar.js:647
Lightning: [calCachedCalendar] Performing playback operation delete on 0 items to psy calCachedCalendar.js:647
Lightning: [calCachedCalendar] Doing changelog based sync for calendar https://calendar.example.com:5232/psy/psy.ics/ calCachedCalendar.js:340
Lightning: CalDAV: send (PROPFIND https://calendar.example.com:5232/psy/psy.ics/): <?xml version="1.0" encoding="UTF-8"?>
<D:propfind xmlns:D='DAV:' xmlns:C='urn:ietf:params:xml:ns:caldav' xmlns:CS='http://calendarserver.org/ns/'><D:prop><D:resourcetype/><D:owner/><D:current-user-principal/><D:supported-report-set/><C:supported-calendar-component-set/><CS:getctag/></D:prop></D:propfind> CalDavRequest.jsm:121
PROPFINDhttps://calendar.example.com:5232/psy/psy.ics/
TypeError: impl is null
6 network-response-listener.js:104:7
Lightning: CalDAV: Error during initial PROPFIND for calendar psy: [object Object] CalDavCalendar.jsm:1590
Lightning: There has been an error reading data for calendar: psy. However, this error is believed to be minor, so the program will attempt to continue. Error code: DAV_NOT_DAV. Description: The resource at https://calendar.example.com:5232/psy/psy.ics/ is either not a DAV collection or not available CalCalendarManager.jsm:824
Lightning: There has been an error reading data for calendar: psy. However, this error is believed to be minor, so the program will attempt to continue. Error code: READ_FAILED. Description: CalCalendarManager.jsm:824
Lightning: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.example.com:5232/psy/psy.ics/, result=2147500037, operation=[xpconnect wrapped calIOperation] calCachedCalendar.js:346
Lightning: [calCachedCalendar] replayChangesOn finished. calCachedCalendar.js:357
Lightning: [calCachedCalendar] sync queue empty. calCachedCalendar.js:329

Component: General → Provider: CalDAV

This looks like a network error, and a number of things could be causing that.

  • Is the server software looking for the word "Lightning" in the HTTP user-agent string? It's no longer there.
  • Is TLS 1.0 or 1.1 being used? Those are disabled by default now.
  • Does the server have a self-signed certificate? I know there's been problems with those but can't recall exactly what they are off the top of my head.
Flags: needinfo?(psychonaut)

No, the server isn't looking for "Lightning" in any HTTP header. The server offers only TLS 1.2. And yes, I'm using a self-signed certificate. Is the problem you're referring to Bug 1590474?

Flags: needinfo?(psychonaut)

Also, is it possible the present issue is related to Bug 1660859, which is about Thunderbird not accepting SSL certificates from Radicale? Note though that I'm not getting the security-related error dialogs reported in that issue. Also note that Bug 1660859 is reproducible with both self-signed certificates and certificates bought from a vendor.

I doubt it's bug 1590474, as the calendar uses different code for that which apparently was fixed for the same problem in bug 1590472.

Kai, perhaps you have some insight. If it is something that changed between 78.2.1 and 78.2.2 as mentioned in comment 0, I can't see it.

Flags: needinfo?(kaie)

Tristan, have you been using a self-signed certificate all the time, and it worked with TB 78.2.1 and earlier versions?
If yes, I wonder why it had worked.

I don't have other good ideas, the modified bad certificate handling code seems the most likely reason.

Flags: needinfo?(kaie)

Yes, I have been using the same self-signed certificate for years.

From looking at my update history, I see that I updated from Thunderbird 68.11.0 to 78.2.1 on 12 September 2020, and then to Thunderbird 78.2.2 on 22 September 2020. I'm absolutely certain that the calendar was working flawlessly with 68.11.0. I'm not completely sure whether or not it was working with 78.2.1, but that's only because I might not have used the calendar for then ten days it was installed. (I don't remember.) But it was definitely occurring with 78.2.2.

But as I mentioned, I do have a different computer running 78.2.1 that has no problem with the same calendar.

I suppose I could try removing the calendar from Thunderbird and then re-adding it. Maybe something got borked during the process of updating from 68.11.0 or 78.2.1, and so simply re-adding the calendar from scratch will fix things.

I see that adding an override for a self signed certificate is supposed to work with latest 78.x, apparently since 78.1.0: bug 1590472

Maybe bug 1650925 will help here as well.

I am continuing to troubleshoot and can offer two further pieces of information:

  1. Subscribing to the same calendar using other CalDAV clients on the same machine (such as KOrganizer) works fine. So I don't think the problem has anything to do with my network connection (firewall, etc.).

  2. I have tried subscribing to the same calendar using a fresh Thunderbird profile. As expected, I get warned about the self-signed certificate, and add a permanent exception. But after this the problematic behaviour returns -- the calendar does not get synchronized and the error console shows the same DAV_NOT_DAV error.

I could see if I get the same error when subscribing to a calendar on a completely different kind of CalDAV server provided I can find a free CalDAV service to test. (I heard that Google might offer such a service, so I'll look into that.) This might determine whether the problem is specific to how Thunderbird interacts with Radicale. I could also try changing my own Radicale server's certificate from a self-signed one to a Let's Encrypt one, which might determine whether the problem is with how Thunderbird handles self-signed certificates.

OK, I changed the certificate on my Radicale CalDAV server from a self-signed one to a Let's Encrypt one. Now when I try to subscribe to the calendar with a fresh profile, I am not warned about the certificate, and instead a dialog pops up asking me for the calendar's username and password. So far so good—I never got as far as this prompt when using the self-signed certificate! However, once I enter the correct username and password, the dialog disappears and the calendar is shown as disabled. Trying to enable it results in the following messages in the Error Console, which are similar to those described in Bug 1667067:

TypeError: impl is null
7 network-response-listener.js:104:7
_forwardNotification resource://devtools/server/actors/network-monitor/network-response-listener.js:104
onStatus resource://devtools/server/actors/network-monitor/network-response-listener.js:395
TypeError: impl is null
network-response-listener.js:104:7
_forwardNotification resource://devtools/server/actors/network-monitor/network-response-listener.js:104
onProgress resource://devtools/server/actors/network-monitor/network-response-listener.js:391
TypeError: impl is null
2 network-response-listener.js:104:7
_forwardNotification resource://devtools/server/actors/network-monitor/network-response-listener.js:104
onStatus resource://devtools/server/actors/network-monitor/network-response-listener.js:395
TypeError: impl is null
network-response-listener.js:104:7
_forwardNotification resource://devtools/server/actors/network-monitor/network-response-listener.js:104
onProgress resource://devtools/server/actors/network-monitor/network-response-listener.js:391
Lightning: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.example.com:5232/psy/pys.ics/, result=2147500037, operation=[xpconnect wrapped calIOperation] calCachedCalendar.js:346

I was not successful in figuring out how to access my Google Calendar via CalDAV. Following the official instructions doesn't work, as I always end up getting username/password errors; this may have something to do with the fact that I don't have a GMail account. If anyone thinks it's worthwhile for me to test a non-Radicale CalDAV server, please recommend a free service (other than Google) that has clear instructions on accessing the calendar from a CalDAV client.

The errors from comment 10 are bug 1650925 which should be fixed in the next 78 release. (Not 100% sure about the last line, but I think it's the same problem.)

OK, but I would prefer to continue using my own self-signed certificate, for which I'm getting completely different errors.

Sorry, I forgot to mention that I upgraded to Thunderbird 78.3.1 a few days ago (and saw no change in the behaviour described in my original report). All my comments from today apply to testing with 78.3.1, not 78.2.2.

I've done some bisection on the official Thunderbird releases. The earliest version of Thunderbird with which I can reproduce this bug is 78.0.1—subscribing to my calendar using a fresh profile with this release always results in the DAV_NOT_DAV error message. Trying to do the same thing with the previous version, 68.12.1, works fine.

Keywords: regression

Can you try nightly? This may have been a symptom of bug 1664731 which was now fixed.

Looks good. I made an update from a working 68.12.1 (64-bit) to a 84.0b3 (32-bit) and caldav calender still works (https://bugzilla.mozilla.org/show_bug.cgi?id=1669068).

Ok, well bug 1664731 only landed in today's nightly so far. I guess something else fixed it.

(In reply to Magnus Melin [:mkmelin] from comment #17)

Can you try nightly? This may have been a symptom of bug 1664731 which was now fixed.

I am trying Thunderbird 86.0a1 (2020-12-16) (64-bit) in two different scenarios, neither of which allows me to use my calendar, but both of which show different behaviour than in my original bug report above:

  1. I tried using my existing profile from 78.5.1, where my calendar is already configured (but not working, per my original report). Trying to sync the calendar fails, with the following error message output to the Error Console:

    Lightning: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.example.com:5232/psy/psy.ics/, result=2147500037, operation=[xpconnect wrapped calIOperation]

  2. I created a new profile and attempted to add my calendar. Upon pressing the Find Calendars button in the Create New Calendar dialog, seven(!) separate Add Security Exception dialogs appear in rapid succession. If I press the View… button, a new about:certificate tab appears in the main Thunderbird window, with a busy throbber as the tab icon, but it stays busy and nothing appears in the tab. If I press the Confirm Security Exception button in each dialog, then after the last one disappears I'm taken back to the Create New Calendar dialog; there's no way of proceeding other than to press the Find Calendars button again (which has the same effect as described) or pressing the Cancel button. If I press the Cancel button, then the about:certificate tab finally gets renamed to "Certificate for example.com" and populated with my certificate details. But my calendar never gets added to Thunderbird.

It's been a few months, so I've tried testing again with the latest stable (78.9.1) and nightly (89.0a1) releases:

  1. If I use Thunderbird 78.9.1 in my existing profile, where my calendar is already configured but not working, then I see the same behaviour as documented in point #1 of Comment 20 above.

  2. If I use Thunderbird 89.0a1 with a new profile and try to add my calendar, I get the same behaviour as documented in point #2 of Comment 20 above.

One further point of information: the problem happens only on my openSUSE Leap 15.2 systems -- even on a fresh install of Leap 15.2. I'm running the latest stable version of Thunderbird on two openSUSE Tumbleweed systems, and they have no trouble connecting to the very same calendar. Is it possible that there's some OS-level dependency that is at fault here?

If there's anything else I can do to help troubleshoot, please let me know.

I've tried both Thunderbird 78.9.1 and Thunderbird 89.0a1 on a completely fresh install of openSUSE Tumbleweed (in a virtual machine) and I get the same behaviour reported in Comment 21 above. So I guess the problem is not specific to openSUSE Leap. Strangely, my two physical machines running Tumbleweed are still able to connect to the calendar (which I had added to Thunderbird many years ago).

Problem also occurs with Thunderbird 90.0a1 on a fresh install of CentOS, so this problem is nothing specific to the openSUSE family of OSes.

Is there any way of manually adding the certificate override (i.e., bypassing the broken GUI altogether)? I tried copying the line in cert_override.txt for my calendar server from one of the Thunderbird instances that can still connect to it, but this had no effect.

Hi Tristan.
I ran into the same error from Comment 20 #1 with TB 78.11.0 and Ubuntu 20.04, freshly installed.
I also use radicale
I first tested it without certificate in a private network.

Have you found a solution, yet?

Yes, my workaround was to copy the cert9.db and cert_override.txt files from a working Thunderbird profile (i.e., one that can already connect to my calendar). I suppose if you don't already have a Thunderbird profile that can connect to your calendar, you could try downloading an older version of Thunderbird (say, 68.11.0), use it to create a new profile and connect to your calendar, and then copy the cert9.db and cert_override.txt files from that 6.8.11.0 profile to the profile you normally use with Thunderbird 78.11.0.

Thank you so much for your Answer Tristan! I had such an older version of TB, but beforehand I took a closer look into my calendar-files. Guess what: Even though I copied my calendar 1:1 from the old system, something went wrong in that procedure! The old calendar was kinda not existing or just some parts of it (luser confirmed) ... -.-
If anyone is as mindless as I was, I did the following:
0. saved the fresh dates on my android-device directly on the device (just in case something goes wrong),

  1. created a new calendar in Radicale,
  2. copied all the new files from the new subfolder into the old one (where all the devices are directed to),
  3. deleted the new calendar and
  4. synced everything.
    Sooo easy O.O
    and thx again Tristan. I guess without you I would not have found out about my own mistake.

I believe I have finally discovered the cause of this bug: when confirming a security exception for a CalDAV calendar, Lightning (or Thunderbird) writes the correct domain name but the wrong port number (443 instead of whatever port number the user specified for the calendar—the CalDAV default is 5232) to the cert_override.txt file. Manually editing that file to give the exception the correct port number works around all the problems described in this bug report (i.e., the MODIFICATION_FAILED error for existing calendars, and the impassable "Add Security exception" dialogs when adding new calendars).

Here is a detailed set of steps for working around the problem when adding a new calendar:

  1. From a fresh or existing profile, launch the command to create a new calendar. A "Create New Calendar" dialog appears.
  2. Select "On the Network" and press "Next".
  3. Enter the location for the calendar (e.g., https://calendar.example.com:5232/foo/bar.ics/) and press "Find Calendars".
  4. Seven separate "Add Security Exception" modal dialogs appear. In the seventh, press "Confirm Security Exception". Dismiss the remaining dialogs either by pressing "Confirm Security Exception" or "Cancel" -- it doesn't matter which button is pressed, though it may be necessary to dismiss the dialogs in the reverse order that they appeared. You will be returned to the original "Create New Calendar" dialog.
  5. Press "Cancel".
  6. Exit Thunderbird.
  7. Using a text editor, open the cert_override.txt file in the Thunderbird profile directory.
  8. Find the entry for the domain name of the certificate whose exception you confirmed in Step 4. This will have the same domain name but the port will be 443 instead of the port specified in Step 3 (e.g., calendar.example.com:443).
  9. Change 443 to whatever port was specified in Step 3 (e.g., 5232) and save the file.
  10. Start Thunderbird again and redo Steps 1 to 3. This time the calendar will be successfully added.

If Lightning stopped being able to connect to an existing calendar (which happened to me following an upgrade from Thunderbird 68.x to Thunderbird 78.x), then it's only necessary to follow Steps 6 to 9 above.

Great, in that case, bug 1750655.

Status: UNCONFIRMED → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE

(In reply to Tristan Miller from comment #27)

I believe I have finally discovered the cause of this bug: when confirming a security exception for a CalDAV calendar, Lightning (or Thunderbird) writes the correct domain name but the wrong port number (443 instead of whatever port number the user specified for the calendar—the CalDAV default is 5232) to the cert_override.txt file. Manually editing that file to give the exception the correct port number works around all the problems described in this bug report (i.e., the MODIFICATION_FAILED error for existing calendars, and the impassable "Add Security exception" dialogs when adding new calendars).

Here is a detailed set of steps for working around the problem when adding a new calendar:

  1. From a fresh or existing profile, launch the command to create a new calendar. A "Create New Calendar" dialog appears.
  2. Select "On the Network" and press "Next".
  3. Enter the location for the calendar (e.g., https://calendar.example.com:5232/foo/bar.ics/) and press "Find Calendars".
  4. Seven separate "Add Security Exception" modal dialogs appear. In the seventh, press "Confirm Security Exception". Dismiss the remaining dialogs either by pressing "Confirm Security Exception" or "Cancel" -- it doesn't matter which button is pressed, though it may be necessary to dismiss the dialogs in the reverse order that they appeared. You will be returned to the original "Create New Calendar" dialog.
  5. Press "Cancel".
  6. Exit Thunderbird.
  7. Using a text editor, open the cert_override.txt file in the Thunderbird profile directory.
  8. Find the entry for the domain name of the certificate whose exception you confirmed in Step 4. This will have the same domain name but the port will be 443 instead of the port specified in Step 3 (e.g., calendar.example.com:443).
  9. Change 443 to whatever port was specified in Step 3 (e.g., 5232) and save the file.
  10. Start Thunderbird again and redo Steps 1 to 3. This time the calendar will be successfully added.

If Lightning stopped being able to connect to an existing calendar (which happened to me following an upgrade from Thunderbird 68.x to Thunderbird 78.x), then it's only necessary to follow Steps 6 to 9 above.

Regressions: 1792470
You need to log in before you can comment on or make changes to this bug.