The latest thunderbird and lightning no longer work with chandler projects server

RESOLVED FIXED in 1.3

Status

defect
--
major
RESOLVED FIXED
9 years ago
7 years ago

People

(Reporter: urkle, Assigned: sfleiter)

Tracking

({regression})

Details

(Whiteboard: [CalDAV server: Chandler], )

Attachments

(2 attachments, 6 obsolete attachments)

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.8) Gecko/20100802 Lightning/1.0b2 Thunderbird/3.1.2

After upgrading to thunderbird 3.1 / lightning 1.0b2 I can no longer access my calendars from the chandler hub.  (upgrade from Fedora 12 - Fedora 13 x86_64).. Also tested on a Mac OS X machine as well.



Reproducible: Always

Steps to Reproduce:
1. Add a new caldav calendar into lightning
2. Use the caldav URL from chandler hub (ie.. https://hub.chandlerproject.org/dav/collection/some-long-hash-code-here)
3. Login when prompted by lightning.
Actual Results:  
Nothing happens.. no calendar events are pulled in.

Expected Results:  
The calendar data to be pulled in as previous versions have (1.0b1 and thunderbird 3.0 worked perfectly)

Bug ID posted on chandler's bugzilla.. 
https://bugzilla.osafoundation.org/show_bug.cgi?id=12980
I can confirm this for TB 3.1.4 / Lightning 1.0b2 on Windows
This appears to be a 1.0b1 vs 1.0b2 issue.  Using the same version of Thunderbird (for example, 3.1.4) 1.0b1 works while 1.0b2 fails.
I tried several Lightning nightlies on Ubuntu Linux, Tb 3.1b1pre with Chandler.

Lightning nightly 1.0b2pre from 20100312 (Build ID: 2010031235150) WORKS.

Lightning nightly 1.0b2pre from 20100313 FAILS. With logging enabled the error console shows:
CalDAV: send: OPTIONS https://hub.chandlerproject.org/dav/collection/
CalDAV: Unexpected status 404 while querying options Chandler

Maybe a regression caused by one of the checkins for Bug 435854?
We should look into this, thanks for the regression range!
Flags: blocking-calendar1.0?
Keywords: regression
If I comment out the "return;" statement (around line 1531) that was introduced to calDavCalendar.js with the patch from Bug 435854 [1], my Chandler calendar is working again, as far as I can tell.
The throbber stopped his endless spinning and I successfully created, moved, modified and deleted an event and a task.

I don't know what exactly this "return;" is needed for and if commenting it out helps with Chandler while breaking other things. But maybe it's a hint.

Some of the messages in the error console:
[...]
Adding supported item: VTODO for calendar: Chandler
Adding supported item: VEVENT for calendar: Chandler
CalDAV: send: OPTIONS https://hub.chandlerproject.org/dav/collection/
CalDAV: Unexpected status 404 while querying options Chandler
CalDAV: Error getting DAV header for Chandler, status 404, data: <?xml version='1.0' encoding='UTF-8'?><D:error xmlns:D="DAV:" xmlns:cosmo="http://osafoundation.org/cosmo/DAV"><cosmo:not-found /></D:error>
CalDAV: Server does not support CalDAV scheduling.
[... calendar data following ...]

[1] http://hg.mozilla.org/comm-central/diff/112ba8740afe/calendar/providers/caldav/calDavCalendar.js
I turned on debug logging (via the calendar.debug.log key in about:config) and did a force refresh of my chandler hub calendar and it seems the lines in question are triggering because a 404 was returned. 

I'm going to further investigate.
OK.. the URL being requested at that point is https://hub.chandlerproject.org/dav/collection/ which for chander hub isn't valid.  (the url for the calendar is a specific GUID under that)..

So "skipping" that check allows lightning to continue checking the REAL collection https://hub.chandlerproject.org/dav/collection/4d7a8c30-fa1f-11dd-9b65-c7db12e33c2d

Updating the chandler hub bug with this new found information.. 

Thanks Robert for narrowing this down.
Whiteboard: [CalDAV server: Chandler]
Version: unspecified → Lightning 1.0b2
Now that the cause has been identified, is there any hope of a fix in sight?
I'll be looking into caldav bugs after I have fixed the remaining [needed-beta] bugs, no promises though.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Same issue on:
Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10
I am using tickets to access Chandler Hub.
Error log with Lightnening is:
CalDAV: send: <D:propfind xmlns:D="DAV:" xmlns:CS="http://calendarserver.org/ns/" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <D:resourcetype/>
    <D:owner/>
    <D:supported-report-set/>
    <C:supported-calendar-component-set/>
    <CS:getctag/>
  </D:prop>
</D:propfind>
CalDAV: Status 207 on initial PROPFIND for calendar Roo
CalDAV: Authentication scheme for Roo is Ticket
CalDAV: recv: <?xml version="1.0" encoding="UTF-8"?>
<D:multistatus xmlns:D="DAV:">
    <D:response>
        <D:href>/dav/collection/b5e1a7d0-a104-11df-8372-8abcbc580eb3/</D:href>
        <D:propstat>
            <D:prop>
                <D:owner>
                    <D:href>/dav/users/roo/</D:href>
                </D:owner>
                <C:supported-calendar-component-set xmlns:C="urn:ietf:params:xml:ns:caldav">
                    <C:comp name="VFREEBUSY"/>
                    <C:comp name="VTODO"/>
                    <C:comp name="VAVAILABILITY"/>
                    <C:comp name="VEVENT"/>
                    <C:comp name="VJOURNAL"/>
                </C:supported-calendar-component-set>
                <CS:getctag xmlns:CS="http://calendarserver.org/ns/">RsDluofNZV3jTDo6BCXIb9HwfyA=</CS:getctag>
                <D:supported-report-set>
                    <D:supported-report>
                        <D:report>
                            <D:principal-property-search/>
                        </D:report>
                    </D:supported-report>
                    <D:supported-report>
                        <D:report>
                            <C:calendar-multiget xmlns:C="urn:ietf:params:xml:ns:caldav"/>
                        </D:report>
                    </D:supported-report>
                    <D:supported-report>
                        <D:report>
                            <C:calendar-query xmlns:C="urn:ietf:params:xml:ns:caldav"/>
                        </D:report>
                    </D:supported-report>
                    <D:supported-report>
                        <D:report>
                            <D:principal-match/>
                        </D:report>
                    </D:supported-report>
                    <D:supported-report>
                        <D:report>
                            <C:free-busy-query xmlns:C="urn:ietf:params:xml:ns:caldav"/>
                        </D:report>
                    </D:supported-report>
                </D:supported-report-set>
                <D:resourcetype>
                    <C:calendar xmlns:C="urn:ietf:params:xml:ns:caldav"/>
                    <D:collection/>
                </D:resourcetype>
            </D:prop>
            <D:status>HTTP/1.1 200 OK</D:status>
        </D:propstat>
    </D:response>
</D:multistatus>

CalDAV: initial ctag RsDluofNZV3jTDo6BCXIb9HwfyA= for calendar Roo
Adding supported item: VTODO for calendar: Roo
Adding supported item: VEVENT for calendar: Roo
CalDAV: send: OPTIONS https://hub.chandlerproject.org/dav/collection/?ticket=a17c1f50
CalDAV: Unexpected status 404 while querying options Roo


However, SunBird [1.0b1
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.5) Gecko/20091211 ]
accessing same ticket gives:
CalDAV: send(https://hub.chandlerproject.org/dav/collection/b5e1a7d0-a104-11df-8372-8abcbc580eb3/?ticket=a17c1f50): <D:propfind xmlns:D="DAV:" xmlns:CS="http://calendarserver.org/ns/">
  <D:prop>
    <CS:getctag/>
  </D:prop>
</D:propfind>
CalDAV: Status 207 checking ctag for calendar Roo
CalDAV: recv: <?xml version="1.0" encoding="UTF-8"?>
<D:multistatus xmlns:D="DAV:">
    <D:response>
        <D:href>/dav/collection/b5e1a7d0-a104-11df-8372-8abcbc580eb3/</D:href>
        <D:propstat>
            <D:prop>
                <CS:getctag xmlns:CS="http://calendarserver.org/ns/">RsDluofNZV3jTDo6BCXIb9HwfyA=</CS:getctag>
            </D:prop>
            <D:status>HTTP/1.1 200 OK</D:status>
        </D:propstat>
    </D:response>
</D:multistatus>

CalDAV: ctag matches, no need to fetch data for calendar Roo


calDavCalendar.js is exactly the same for Sunbird and Lightening and removing the 'Return' does not help in my case.

I use Sunbird because it works, but would prefer Lightening.
Version: Lightning 1.0b2 → Lightning 1.0b4
Thunderbird 5.0 and lightning 1.0b4 simply commented out line 1560 (return;) and if fixed it still..

1555             if (request.responseStatus != 200) {
1556                 cal.LOG("CalDAV: Unexpected status " + request.responseStatus +
1557                         " while querying options " + thisCalendar.name);
1558                 thisCalendar.completeCheckServerInfo(aChangeLogListener,
1559                                                      Components.results.NS_ERROR_FAILURE);
1560 //                return;
1561             }
Have just tried this work around with Thunderbird5.0 and Lightning 1.0b4 (Linux 2.6.40-4.fc15.i686).
Now works! Excellent.
Edward, Roo, could you find out which responseStatus is happening at that point? It sounds kind of wrong to me to ignore the error there and continue?
It is returning a 404 error, since at that point lightning is trying to access https://hub.chandlerproject.org/dav/collection/ which is not valid on Chandler Server apparently.

It seems lightning is trying to query the parent folder of the collection it is querying which fails on Chandler.    I've been running this "patch" for nearly a year without any issues thus far.  I'm sure there is cleaner code that can be written there to properly handle this situation, as it was explicitly changes in 1.0b2 to bail out in the != 200 situation where 1.0b1 did not.
In that case I'd prefer to make sure we are querying the correct URL.
This is what I get from https://hub.chandlerproject.org/dav/collection/ :

HTTP/1.1 404 Not Found
Date: Mon, 12 Sep 2011 17:09:17 GMT
Server: Apache-Coyote/1.1
X-Cosmo-Version: 1.1-SNAPSHOT
Content-Type: text/xml;charset=UTF-8
Via: 1.1 hub.chandlerproject.org
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 135
Keep-Alive: timeout=15, max=500
Connection: Keep-Alive

<?xml version='1.0' encoding='UTF-8'?><D:error xmlns:D="DAV:" xmlns:cosmo="http://osafoundation.org/cosmo/DAV"><cosmo:not-found /></D:error>

A very stupidly simple way to implement Philipp's suggestion (as I understand it) would be: If
1. "X-Cosmo-Version" HTTP response header is present, and
2. the last path component is "collection",
go on and don't return prematurely.
I can confirm that this problem exists with Seamonkey 2.4.1 and lightning 1.0b7.
(In reply to Paul M. Dubuc from comment #17)
> I can confirm that this problem exists with Seamonkey 2.4.1 and lightning
> 1.0b7.

I also tried Seamonkey 2.5b4 with Lightning 1.0. and the problem is there also.  This is for Mac OS X 10.6.8
You don't even need to rebuild or repackage the add-on to try this patch. Simply apply it to [profile_folder]/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/components/calDavCalendar.js or wherever it is installed.
Attachment #574882 - Flags: review?(philipp)
(In reply to David Foerster from comment #20)
The patch did not seem to help me on the machine with which I tried it.  This is a Fedora 15 laptop with Thunderbird 7 and Lightning 1.0b7.  Applying the patch (manually, as the affected area is in a different location in the file), I now get:

An error was encountered preparing the calendar located at <location> for use. It will not be available.

Error Code 0x80570015

[Exception... "Component returned failure code: 0x80570015 (NS_ERROR_XPC_CI_RETURNED_FAILURE) [nsIJSCID.createInstance]"  nsresult: "0x80570015 (NS_ERROR_XPC_CI_RETURNED_FAILURE)"  location: "JS frame :: resource://calendar/modules/calUtils.jsm -> file:///usr/lib/mozilla/extensions/%7B3550f703-e582-4d05-9a08-453d09bdfdc6%7D/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/calendar-js/calCalendarManager.js :: cmgr_createCalendar :: line 623"  data: no]
(In reply to David Foerster from comment #20)
> You don't even need to rebuild or repackage the add-on to try this patch.
> Simply apply it to
> [profile_folder]/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/
> components/calDavCalendar.js or wherever it is installed.

The patch works with Seamonkey 2.4.1 and Lightning 1.0b7. Thank you!
I missed the 2nd condition (see comment 16) in the first patch.
Attachment #574882 - Attachment is obsolete: true
Attachment #574882 - Flags: review?(philipp)
(In reply to Don Levey from comment #21)
> (In reply to David Foerster from comment #20)
> The patch did not seem to help me on the machine with which I tried it. 
> This is a Fedora 15 laptop with Thunderbird 7 and Lightning 1.0b7.  Applying
> the patch (manually, as the affected area is in a different location in the
> file), I now get:
> 
> An error was encountered preparing the calendar located at <location> for
> use. It will not be available.

The patch was meant against v1.0. I probably should have stated that in its description (fixed in the new patch).

What does your patched calDavCalendar.js look like now, if the patch didn't apply cleanly?
(In reply to David Foerster from comment #24)
> (In reply to Don Levey from comment #21)
> > (In reply to David Foerster from comment #20)
> 
> > An error was encountered preparing the calendar located at <location> for
> > use. It will not be available.
> 
> The patch was meant against v1.0. I probably should have stated that in its
> description (fixed in the new patch).
> 
> What does your patched calDavCalendar.js look like now, if the patch didn't
> apply cleanly?

That'll teach me not to trust 'patch' (I had noticed that the line numbers didn't match what was in the file).  Just applying the fix using 'patch' did apply cleanly, and allow me to start Thunderbird/Lightning without the error I previously reported.

However, it still isn't working.  I don't see any calendar events (whereas I still have, on another machine, a working copy of Sunbird which displays these events).  What's more, while I have six calendars defined, I can only enable the first one.  The other five are greyed out and even selecting "switch this calendar on" in its properties doesn't work.

The following two lines show "reload remote calendars" activity in the server access log:

192.168.1.131 - sandra [21/Nov/2011:08:24:55 -0500] "PROPFIND /chandler/dav/collection/b5d3bdf7-c083-11dd-d382-cafa7eb0e437/ HTTP/1.1" 207 2367 "-" "Mozilla/5.0 (X11; Linux i686; rv:7.0) Gecko/20110927 Thunderbird/7.0 Lightning/1.0b7"
192.168.1.131 - sandra [21/Nov/2011:08:24:55 -0500] "OPTIONS /chandler/dav/collection/ HTTP/1.1" 404 140 "-" "Mozilla/5.0 (X11; Linux i686; rv:7.0) Gecko/20110927 Thunderbird/7.0 Lightning/1.0b7"


And this shows an attempt to enable one of the disabled calendars:
192.168.1.131 - sandra [21/Nov/2011:08:25:58 -0500] "PROPFIND /chandler/dav/collection/c7986d2e-c080-11dd-c372-cafa7eb0e437/ HTTP/1.1" 403 303 "-" "Mozilla/5.0 (X11; Linux i686; rv:7.0) Gecko/20110927 Thunderbird/7.0 Lightning/1.0b7"

I'm not an expert in the Chandler server, but it looks like I'm still seeing the same 404 errors.
If the patch applied without fuzz, it was most likely applied correctly. Don't mind non-matching line numbers (under normal circumstances).

The 2nd request error log entry seems like the one, the patch is about. Can you post its exact response like I did in comment 16? Maybe they are different and the patch doesn't check for "the right thing(TM)" in your case.
(In reply to David Foerster from comment #26)
> If the patch applied without fuzz, it was most likely applied correctly.
> Don't mind non-matching line numbers (under normal circumstances).
> 
> The 2nd request error log entry seems like the one, the patch is about. Can
> you post its exact response like I did in comment 16? Maybe they are
> different and the patch doesn't check for "the right thing(TM)" in your case.

Ah, there *was* fuzz:
"$ sudo patch < calDavCalendar.patch
patching file calDavCalendar.js
Hunk #1 succeeded at 1679 with fuzz 1 (offset -14 lines)."

I'll have to check for the response to the Chandler project URL when I get home...
Ugh - how are you capturing that response?
(In reply to Don Levey from comment #28)
> Ugh - how are you capturing that response?

Open a raw/telnet connection to the Chandler server and issue a HTTP GET request for the resource in question and record the output. In your case that would probably be sth like
$ echo -ne 'GET /chandler/dav/collection/ HTTP/1.1\r\n\r\n' | telnet 192.168.1.131 80 | tee logfile

Another way would be a tool like Firebug that can display the raw response.


(In reply to Don Levey from comment #27)
> Ah, there *was* fuzz:
> "$ sudo patch < calDavCalendar.patch
> patching file calDavCalendar.js
> Hunk #1 succeeded at 1679 with fuzz 1 (offset -14 lines)."

As for the fuzz, to make absolutely shure you should check manually if the resulting file looks like it should in the patched area(s). Though fuzz 1 is usually no problem.
Bug 610087 has a better description.
This bug is not only related to one special server (Chandler/Cosmo) but is a violation of the WebDAV RFCs.
As mentioned in bug 610087 users of DAViCal (and probably other servers) also are not able to use Lightning because of this bug.
Since the problem is not Cosmo/Chandler specific the proposed patch in attachment 575795 [details] [diff] [review] is not right.

The OPTIONS request has to be done against the Calendar Collection URL and not against the parent.

For that I changed the following in calDavCalendar.js of Lightning 1.0:
- In function caldav_setCalHomeSet() I replaced
        let split2 = baseUrl.split('/');
        split2.pop();
        calUri.spec = split2.join('/') + '/';
  by
	calUri.spec = baseUrl;
  and thereby stopped the stripping of the last path segment.

I do not understand why the last path segment should be stripped
off this.mCalHomeSet.
Maybe somebody knowing the code a bit can help there.

With the above change Lightning does work against Cosmo/Chandler 1.2.0.
For people wanting to test the fix described in comment 31.
Patch is against Lightning 1.0.

These is my first Mozilla patch ever, so please be gentle. :-)
Unfortunately, David, using a raw telnet connection fails because I'm not passing the proper authentication along with the request.  I don't currently have any of the tools available to peer into the connection the Lightning client is making.

Stefan, I tried your patch (as well as David's patch) with similar results.  On my wife's laptop - my test machine for this - I am able to see the first calendar I added, though not always (it doesn't always completely refresh properly if I change the view multiple times).  When the calendar itself doesn't refresh properly, the "coming events" section above the calendar does show the list of events.  The "today" pane to the right of the main T-Bird view only shows what is listed for today, no "soon" tasks.  These are all for the first calDAV calendar I add.

For all subsequent calendars, I create them using the same procedure (right-clicking, selecting "new calendar", filling in the dialogs), but the calendars are immediately disabled and thus I cannot view any events.  If I right-click on the calendar in the list, choose "properties" and enable the calendar, it immediately disables itself again.  Unfortunately, I can't find any sort of error console or other output to tell me why this happens.

I can't tell if this is related to the main error, or is simply co-incident to it.  However, I am reluctant to believe that the server itself is the problem not only because Lightning doesn't currently work against multiple back-end servers but because it *used* to work in prior versions.  Unfortunately, I am unable to use those versions on all of the affected systems.
@Don:
I assume you hit another bug and propose you open a new one.
The bug in the original description prevents Thunderbird to get calendar data at all.

Debugging calendar issues with Thunderbird is not that difficult, simply follow
the following steps:
    Open Tools/Options
    Click on Advanced Button
    Choose tab General
    Click on Button Config Editor
    set the following options to boolean true
        calendar.debug.log
        calendar.debug.log.verbose 
    restart Thunderbird 

    see debug spew in Tools/Error Console

But make a new bug please and do not paste the information here.
Thanx.
Cleaned up version of last patch, no functional difference.
Attachment #578252 - Attachment is obsolete: true
My apologies - I wasn't sure if it belonged here as it was clearly a change in behaviour based upon the patch application.  New bug is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=707138 including debug information.  Thanks!
After dealing with auth info I can say that this works well on the (first) Linux machine on which I tested.  Now - how can I apply this patch to my Windows installation?  I can't find the calDavCalendar.js file on my system, except in the Sunbird installation (1.0b1) that I'm using to compensate.
In Thunderbird
Help->Troubleshooting Information
Next To "Profile Directory" click on "Open Containing Folder"
Step into extensions
One of the subfolders contains a "calendar-js" directory, that is Lightnings dir.
Apply the patch from there.
If you can not create Events with Lightning and Cosmo and probably other calendar servers, that might be bug 707231.
Stefan, please forgive my ignorance.  While I was able to retrieve and install a Windows port of the patch util, it is not finding the proper file - perhaps because a calDavCalendar.js file doesn't exist in that directory.  I am indeed doing this within the calendar-js directory.  Do I need to edit the patch text to specify a particular file?

patch < caldav-patch.txt
can't find file to patch at input line 3
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|--- components\calDavCalendar.js.orig  2011-12-01 15:07:00.844786100 +0100
|+++ components\calDavCalendar.js       2011-12-02 13:54:34.650812800 +0100
--------------------------
File to patch:
Skip this patch? [y]
Skipping patch.
patch unexpectedly ends in middle of line
1 out of 1 hunk ignored
No, you should do that in the directory which includes the calendar-js dir, not in the calendar-js dir itself.
That directory should have a components dir and in the a file called calDavCalendar.js as you can see in the patch file.
If you have further problems do not add to this bug, but send me an email.
I won't be near a computer for most of the weekend, so do not expect fast answers. :-)
Comment on attachment 578548 [details] [diff] [review]
Don't remove last path segment from mCalHomeSet (cleaned up)

@Fallen:
After reading of comment 9 I realized you might be interested in this.
Could you give your opinion on the patch?
I do not know why the last path element should be stripped, 
according to
https://tools.ietf.org/html/rfc4791#section-3.1
one must not assume that the parent of a CalDAV collection
supports CalDAV features:
   Therefore, a repository confining calendar data to the /calendar/
   collection would only need to support the CalDAV required features
   within that collection.

Looking at the diff again the original code seemed to try to make sure
the url ends in a single '/'.
I can update the patch accordingly if needed.

Thx.
Attachment #578548 - Flags: feedback?(philipp)
Patch for Lightning 1.1b1 from
https://addons.mozilla.org/de/thunderbird/addon/lightning/#install-beta
which is the current version for Thunderbird Beta 9.

This time it is made sure that the URL ends with a single /.

I had a look over the code and found no reason the last path segment
was stripped before.
Attachment #578548 - Flags: feedback?(philipp)
Attachment #579383 - Flags: feedback?(philipp)
Comment on attachment 579383 [details] [diff] [review]
Don't remove last path segment from mCalHomeSet with one / at end (Lightning 1.1b1)

I dug out this comment from back when the original code [1] was written:

    // we need to be able to locate the principal-URL of the calendar           
    // in order to get certain properties, but there currently is no reliable   
    // way to do this programatically that works with different server          
    // implementations. So provisionally we assume the 99% case, where the      
    // calendar's principal-URL is the immediate parent of the calendar itself  

In bug 398975 first the principal uri was first set this way [1], then a followup patch changed the naming to the home set [2]. Afterwards the comment is as follows:

    // we need to be able to locate the principal-URL of the calendar           
    // in order to get certain properties, but there currently is no reliable   
    // way to do this programatically that works with different server          
    // implementations. So provisionally we assume the 99% case, where the  
    // calendar's calendar-home-set is the immediate parent of the calendar itself

If you change the home set as you are in the patch, then it means you assume that all servers home set matches the calendar uri. I don't think that is the case for most calendar servers. I'm going to have to deny feedback based on this.

We are doing some whacky stuff to detect all properties that we need and to be honest I don't see through it. I was recently told we should rather be issuing an authenticated current-user-principal query to get the principal instead, and then we should be able to get the home set from there. This is a much larger change though and I don't know how widely supported the needed server queries are.

Now we need to figure out where to go from here, you've put some thought into this and I don't want to just say no. 

Are you willing to invest some time into fixing the caldav provider on a higher level? This means figuring out what queries we do in what order (and if there are alternatives, when they are taken) and also what properties are retrieved from which urls. Such an overview would be marvelous, it would help fixing and extending the provider in the future. From there we can find out if there is a way to use different queries and less guessing.

If you don't want to go so far, I think we could get away with a workaround for cosmo, or possibly even more servers. If the code finds out that the guessed home set doesn't work, it could retry with the full calendar uri maybe? I'm open to other suggestions, as you may be more into the code.

If you need ad-hoc help, you can reach me on irc.mozilla.org #calendar, my nick is Fallen. If you want to go in deeper then we can also do a phonecall, I speak German.



[1] http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/calendar/providers/caldav/calDavCalendar.js&rev=1.106&root=/cvsroot
[2] http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/calendar/providers/caldav/calDavCalendar.js&rev=1.110&root=/cvsroot
Attachment #579383 - Flags: feedback?(philipp) → feedback-
Finding calendars is described here:
https://tools.ietf.org/html/rfc4791#section-8.4

The normal case seen here is that the client is directly configured with a
calendar location.
That's exactly the case I expected from Lightning.

So the next step for Lightning IMO would be to check whether the URL
already points to a calendar collection and if so do not remove the
last path element.

The full discovery process could be implemented next.
From IRC discussion for further reference:

[22:07] <stfl> No, I think I got it wrong.
[22:07] <stfl> Lightning currently awaits the principal URL to be the parent URL of the collection.
[22:08] <stfl> Lightning wants to do some queries which can be done on the principal URL according to spec.
[22:08] <stfl> The right idea would probably be to do those queries directly on the provided URLs if they are not successful on the URLs parent.
[22:09] <stfl> Detect the error and do the same again on the original URL.
[22:09] <stfl> The other proposed solution was a thinko.
[22:10] <stfl> For me it sounds more helpful than detecting Cosmo and doing something specific.
[22:11] <stfl> Does that sound right?
[22:17] <Fallen|mac> stfl: that sounds like a reasonable workaround, yes


[22:37] <Fallen|mac> stfl: do you know if cosmo supports current-user-principal?
[22:37] <Fallen|mac> stfl: what we could do is try plugging in a current-user-principal query into the current process
[22:38] <Fallen|mac> if it succeeds, use that as the principal uri
[22:38] <Fallen|mac> and then let the other parts continue
[22:40] <stfl> Fallen|mac: I do not think so. I am at the wrong machine to check that but can do that tomorrow.
[22:40] <karora> You can ask for the <owner> property on the collection to find the owner's principal-URL.  Of course that might not be the principal-URL for the current user.
[22:43] <stfl> karora: That depends on which principal-URL is needed in which case.
[22:43] <karora> Yeah.  I know there are some problems that Lightning has when it goes for the owner of the collection and is denied access.
[22:44] <karora> People frequently configure DAViCal to share just the collection, so a non-owner accessing that collection should be able to see it, but it can fail in Lightning at that point.
[22:45] <stfl> IMO the main problem is that Lightning asks for a CalDAV URL without describing what kind of URL that should be.
[22:48] <Fallen|mac> its always the collection url right now, but I agree we should detect the other way around. Lenni is working on that, but he is swamped with university work atm
[22:49] <stfl> Expecting the parent URL of a collection URL to be available is against the spec as we all know.
[22:49] <Fallen|mac> right
[22:50] <Fallen|mac> I don't think the auto-detection will be read in the next few months, so a workaround with current-user-principal would be nice
[22:52] <Fallen|mac> doing auto-detection right causes so many changes. If we auto-detect, there should be one calendar account that contains the subcalendars. This alone is a very substantial change. We've already gone the simple route with the current auto-detection implementation (not in core yet)
[22:53] <stfl> Fallen|mac: But this workaround will only work for new Cosmo versions if I rember correctly and Cosmo does not support current-user-principal yet.
[22:54] <Fallen|mac> ok, in that case we can go the route you originally suggested, if it fails on the parent then do it on the calendar uri
Cosmo does not support current-user-principal.
Fix bug as described in comment 46, includes fix for bug 605378.

I can create a separate patch if desired, but since both patches are to the same file the order of applying them would have to be chosen.
Otherwise the generated patch won't apply cleanly (without offsets).
Attachment #582017 - Flags: review?(philipp)
Assignee: nobody → stefan.fleiter
Status: NEW → ASSIGNED
Duplicate of this bug: 610087
Duplicate of this bug: 697524
Possible duplicates:
bug 599635 and bug 658960
From duplicates this bug hinders DAViCal and possibly jmail (bug 658960) from working, too.
So this is not chandler/cosmo specific.
Comment on attachment 582017 [details] [diff] [review]
Fall back to use calendar URL if parent URL does return 404

Review of attachment 582017 [details] [diff] [review]:
-----------------------------------------------------------------

Going in the right direction here!
Nevertheless, r- for now to fix the below comments and get a new patch.

::: calendar/providers/caldav/calDavCalendar.js
@@ +1612,5 @@
>                  thisCalendar.mSupportedItemTypes.length = 0;
>                  for each (let sc in supportedComponentsXml.C::comp) {
> +                    // accept name attribute from all namespaces to workaround Cosmo bug
> +                    // see https://bugzilla.mozilla.org/show_bug.cgi?id=605378#c6
> +                    let comp = sc.@*::name.toString();

Lets keep this in the other bug, as I'm about to push that one.

@@ +1699,5 @@
> +                    // try again with calendar URL, see https://bugzilla.mozilla.org/show_bug.cgi?id=588799
> +                    cal.LOG("CalDAV: calendar homeset was not found at parent url of calendar URL" +
> +                            " while querying options " + thisCalendar.name + ", will try calendar URL itself now");
> +                    thisCalendar.setCalHomeSet(false);
> +                    thisCalendar.checkServerCaps(aChangeLogListener);

What happens if both URLs are 404? This could cause an endless loop of queries. Please make sure that an error is returned if both URLs return a 404.
Attachment #582017 - Flags: review?(philipp) → review-
Attachment #575795 - Attachment is obsolete: true
Attachment #578548 - Attachment is obsolete: true
Attachment #579383 - Attachment is obsolete: true
(In reply to Philipp Kewisch [:Fallen] from comment #53)

> Review of attachment 582017 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Going in the right direction here!

Thx!

> Lets keep this in the other bug, as I'm about to push that one.

No problem, will do.

> 
> @@ +1699,5 @@
> > +                    // try again with calendar URL, see https://bugzilla.mozilla.org/show_bug.cgi?id=588799
> > +                    cal.LOG("CalDAV: calendar homeset was not found at parent url of calendar URL" +
> > +                            " while querying options " + thisCalendar.name + ", will try calendar URL itself now");
> > +                    thisCalendar.setCalHomeSet(false);
> > +                    thisCalendar.checkServerCaps(aChangeLogListener);
> 
> What happens if both URLs are 404? This could cause an endless loop of
> queries. 

I do not know how that should happen, but probably I misunderstand
the code because I am new to Javascript programming.

As I understand it a 404 in caldav_checkDavResourceType 
stops processing and delegates to 
completeCheckServerInfo:

 if (!str || request.responseStatus == 404) {
                // No response, or the calendar no longer exists.
                cal.LOG("CalDAV: Failed to determine resource type for" +
                        thisCalendar.name);
                thisCalendar.completeCheckServerInfo(aChangeLogListener,
                            Components.interfaces.calIErrors.DAV_NOT_DAV);
                return;
 }

So caldav_checkServerCaps should not be reached in that case.

What am I missing?
So this is what I understand what could happen:


* checkServerCaps gets called with the stripped uri
* the server responds with a 404
* checkServerCaps onStreamComplete finds out and changes the homeset
    -> checkServerCaps is called again from the beginning
* The server could now again respond with a 404, for whatever reason.
* checkServerCaps onStreamComplete finds out, but has alredy changed the home set.
    -> Nevertheless, checkServerCaps is called again from the beginning
* Endless recursion

Please correct me if I am wrong though
Next version of patch.
This time with recursion stop even in case of a malicious server.
I changed setCalHomeSet to make clear calendarUri is used
in the retry case.

Would be nice if this could go in aurora, too.
Attachment #582017 - Attachment is obsolete: true
Attachment #584186 - Flags: review?(philipp)
Comment on attachment 584186 [details] [diff] [review]
Fall back to use calendar URL if parent URL does return 404

Review of attachment 584186 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, r=philipp and approval for aurora. I'll take care of the checkin in a moment.

::: calendar/providers/caldav/calDavCalendar.js
@@ +1739,5 @@
> +                    // try again with calendar URL, see https://bugzilla.mozilla.org/show_bug.cgi?id=588799
> +                    cal.LOG("CalDAV: calendar homeset was not found at parent url of calendar URL" +
> +                            " while querying options " + thisCalendar.name + ", will try calendar URL itself now");
> +                    thisCalendar.setCalHomeSet(false);
> +                    thisCalendar.checkServerCaps(aChangeLogListener, ++calHomeSetUrlTryCount);

We could get rid of the above if() by doing

thisCalendar.checkServerCaps(aChangeLogListener, (calHomeSetUrlTryCount || 0) + 1);
Attachment #584186 - Flags: review?(philipp) → review+
Pushed to comm-central changeset e866151d44a0
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → 1.4
Backported to releases/comm-aurora changeset 56a69133e5f0
Target Milestone: 1.4 → 1.3
Flags: blocking-calendar1.0?
I am sorry but the modifications from comment 57 prevents the fallback mechanism from working.

In checkServerCaps the fallback is activated when |calHomeSetUrlTryCount == 1|
but checkDavResourceType calls checkServerCaps without the calHomeSetUrlTryCount parameter so calHomeSetUrlTryCount is undefined.

So we could either check in the patch in attachment 584186 [details] [diff] [review] or add the parameter in
the call of checkServerCaps in checkDavResourceType.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
This patch fixes the current central and aurora branches, see comment 60.
Without this patch the fallback mechanism is inactive.
Attachment #585527 - Flags: review?(philipp)
Attachment #585527 - Flags: approval-mozilla-aurora?
Comment on attachment 585527 [details] [diff] [review]
Repair of Fall back to use calendar URL if parent URL does return 404

Review of attachment 585527 [details] [diff] [review]:
-----------------------------------------------------------------

r=philipp and approval for aurora (I can't set the approval flags yet)
Attachment #585527 - Flags: review?(philipp)
Attachment #585527 - Flags: review+
Attachment #585527 - Flags: approval-mozilla-aurora?
Pushed to comm-central changeset 6ad18d15c741
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Target Milestone: 1.3 → 1.4
Backported to releases/comm-aurora changeset 883966da7ec7
Target Milestone: 1.4 → 1.3
Duplicate of this bug: 658960
I have a DAViCal server and some users share their calendars with other ones.

Imagine that 'user2' gives read/read+write privileges for a calendar of his own (i.e. https://caldav.server/caldav.php/user2/calendarX) to 'user1'. If 'user1' uses Lightning 1.5 to access that calendar it will just fail after sending an OPTIONS request to 'user2' principal URL (i.e. https://caldav.server/caldav.php/user2/), because 'user2' didn't give access permissions to his own principal URL, so server answers with a 403 code.

Does it make sense to you to also use the configured URL instead of the parent path when server returns a 403 code?
Any updates on this? Lightning 1.8 keeps sending an OPTIONS request to parent URL and loads no events at all because of the first query returning a 403.
Jorge, please try the latest nightly builds, I have fixed a few issues regarding shared calendars and URLs lately. If the issue persists, please open a new bug.
You need to log in before you can comment on or make changes to this bug.