1468912 - CalDAV does not synchronize with Nextcloud if network.cookie.same-site.enabled is true

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect)

Description

[Florian]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180605171542

Steps to reproduce:

install Thunderbird 60 Beta 7 (lightning 6.2)
start Thunderbird
cancel mailconfig
open calendar-tab
in sidebar - rightclick new calendar 
* on network, -> next
* caldav and entered URL, -> next
* entered name, -> next, -> ready
* entered credentials, -> ok



Actual results:

calendar on the sidebar is created
no calendar-entrys from caldav were syncronised
no error-message is shown



Expected results:

calendar-entrys from caldav should be syncronised (works in TB 52.8.0)
in error-case show error-message

Comment 1

[Florian]

caldav calendar does not synchronize in Thunderbird 60 Beta 8 (lightning 6.2)

Comment 2

[Danny Qwertz]

I think this also could be linked to bug 1468755

Error console is giving me a Status 503, on tb-stable the same caldav and carddav-links (as in bug 1468755) are working without any problems.

Comment 4

[Norbert]

Bug still remains on latest upcoming Beta release
http://ftp.mozilla.org/pub/thunderbird/releases/60.0b9/

Comment 5

[Florian]

Furthermore I tested the import of an ics calendar:

https://www.ferienwiki.de/exports/feiertage/2018/de/niedersachsen

On TB 52 everything syncs, on TB 60 nothing happens after creating the new network-based calendar.

This seems to be a general import-problem.

Comment 6

[Norbert]

CardDAV has the same problem.
Trying to create a new addressbook under CardBook results in the following error messages.
Sorry that these are in German, but I think you will understand them anyway.

2018.06.27 09:48:59:134 : Validation module: Überprüfe ohne Suchauftrag auf https://nextcloudurl/remote.php/dav/addressbooks/users/username/contacts/ …
2018.06.27 09:48:59:252 : Validation module: Synchronisation fehlgeschlagen (Schritt: validateWithoutDiscovery, URL: https://nextcloudurl/remote.php/dav/addressbooks/users/username/contacts/, Status: 503)
2018.06.27 09:49:00:157 : Validation module: Discovery-Phase 1 auf https://nextcloudurl/remote.php/dav/addressbooks/users/username/contacts/.well-known/carddav …
2018.06.27 09:49:00:237 : Validation module: Synchronisation fehlgeschlagen (Schritt: discoverPhase1, URL: https://nextcloudurl/remote.php/dav/addressbooks/users/username/contacts/.well-known/carddav, Status: 503)
2018.06.27 09:49:01:146 : Validation module: Überprüfe ohne Suchauftrag auf https://nextcloudurl/remote.php/dav/addressbooks/username …
2018.06.27 09:49:01:227 : Validation module: Synchronisation fehlgeschlagen (Schritt: validateWithoutDiscovery, URL: https://nextcloudurl/remote.php/dav/addressbooks/username, Status: 503)

As in Lightning using a caldav calendar, error 503 is thrown.

I don't know if it is helpful, but I'm using a let's encrypt certificate on our Nextcloud Server.
Perhaps this might have something to do with this bug?

Comment 7

[Norbert]

No, seems not to be SSL related.
Tried the same without SSL and the error stays the same.

Comment 8

[David Ross]

I can not reproduce this - Thunderbird Beta 60.0b9 - Lightning 6.2 - New User Profile
Took longer than expected to sync, but it eventually did for me.

Comment 9

[Norbert]

Attachment: TB60-profile-bug1468912.7z

Profile for Thunderbird 60.0 beta 9

The 7zip archive contains the whole profile to test the things described in this bug report.
It also contains the file Nexcloud.login.txt which includes the URL and login credentials to our server for review of one of the developers

The archive is password protected.
I will give you the password if you send an email to my email account.

Comment 10

[Norbert]

Florian, could you please change the fields product and component to something that better describes the bug.
In fact this bug is not only related to lightning as you can see from my test with cardbook addon.
It must have something to do with a core component in Thunderbird itself that deals with network connections.

So, perhaps you can change the field product to Thunderbird and the field component to something that is network related?

Comment 11

[Florian]

David Ross: I tried it again, now I can import an ics calendar.
I can't say exactly what it was.

Unfortunately I cannot edit comment 5.


egal8888: There are no network related entries there, so I choosed general.

Comment 12

[Norbert]

(In reply to Florian from comment #11)
> David Ross: I tried it again, now I can import an ics calendar.
> I can't say exactly what it was.

I can confirm that importing an ics calendar works in Lightning.
But, as already stated, the imported events will not be synchronized to nextcloud server.

> egal8888: There are no network related entries there, so I choosed general.

Thanks anyway for changing.

Comment 13

[Norbert]

Apache log gives the following results while starting Thunderbird:

for Thunderbird 52.8.0:
46.82.42.227 - - [28/Jun/2018:14:31:42 +0200] "PROPFIND /remote.php/dav/calendars/test60/personal/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "PROPFIND /remote.php/dav/calendars/test60/personal/ HTTP/1.1" 207 1740 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "OPTIONS /remote.php/dav/calendars/test60/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "OPTIONS /remote.php/dav/calendars/test60/ HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "PROPFIND /remote.php/dav/principals/users/test60/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "PROPFIND /remote.php/dav/principals/users/test60/ HTTP/1.1" 207 926 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"
46.82.42.227 - - [28/Jun/2018:14:31:43 +0200] "REPORT /remote.php/dav/calendars/test60/personal/ HTTP/1.1" 207 312 "-" "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 Lightning/5.4.8"

As you can see, the server first answers with status code 401 and therefor sends an WWW-Authenticate header field to the client. The client sends the required credentials back to the server and logs in while getting response code 207.

for Thunderbird 60.0b9:

46.82.42.227 - - [28/Jun/2018:14:40:56 +0200] "PROPFIND /remote.php/dav/calendars/test60/personal/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Thunderbird/60.0 Lightning/6.2"
46.82.42.227 - - [28/Jun/2018:14:40:56 +0200] "PROPFIND /remote.php/dav/calendars/test60/personal/ HTTP/1.1" 503 - "-" "Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Thunderbird/60.0 Lightning/6.2"

As in section above the server first answers with status code 401 and therefor also sends the WWW-Auhtenticate header field to the client. But different to Thunderbird release version 52.8.0 the beta version seems to misinterprete the WWW-Authenticate header field and does not send the right credentials or otherwise does not send any credentials, or it sends another false answer flag, that makes the server responding with code 503.

The error console in Thunderbird 60.0b9 throws the following error:

Lightning:[calCachedCalendar] replay action failed: null, uri=https://test60:password@nextcloudurl/remote.php/dav/calendars/test60/personal/, result=2147500037, operation=[xpconnect wrapped calIOperation]
calCachedCalendar.js:330

Comment 14

[Florian]

(In reply to David Ross from comment #8)
> I can not reproduce this - Thunderbird Beta 60.0b9 - Lightning 6.2 - New
> User Profile
> Took longer than expected to sync, but it eventually did for me.

Did your comment refer to the ics- or the caldav sync?
(ics-sync is actualy working, caldav- respectively carddav-sync are problematic)

Comment 15

[Norbert]

Owncloud 7.0.4 does not run into the described issue.

That's why it might be, that the bug is not Thunderbird related.

I opened bug report for Nextcloud: https://github.com/nextcloud/server/issues/10134

Comment 16

[Florian]

I looked at some other versions of (own/next)clouds:

I can confirm a working ownCloud 10.0.2.
I can confirm a not working Nextcloud 11.0.3 and nextcloud 12.0.9.

Comment 17

[Sisim Biva]

We might be investigating the same issue here: https://gitlab.com/CardBook/CardBook/issues/306 and https://bugzilla.mozilla.org/show_bug.cgi?id=1468755

Could you confirm that TB60 beta 4 did not have the issue, while beta 5 and following have it? (you can find these versions here: https://filehippo.com/fr/download_thunderbird/history/

Comment 18

[Florian]

I can confirm:
- Beta 4 syncs caldav (works)
- Beta 5 does not sync (-> actual issue)

Comment 19

[Florian]

(In reply to Sisim Biva from comment #17)
> We might be investigating the same issue here:
> https://gitlab.com/CardBook/CardBook/issues/306 and
> https://bugzilla.mozilla.org/show_bug.cgi?id=1468755

Should a bug be closed as a duplicate?
(the other should be adjusted accordingly to both)

Comment 20

[Jorg K (CEST = GMT+2)]

Anything we can investigate in TB here?

Comment 21

[Florian]

(In reply to Jorg K (GMT+2) from comment #20)
> Anything we can investigate in TB here?

From Thunderbird 60 Beta 5 the DAV sync with nextcloud is broken.
(Bug 1468755 for CardDAV, this one for CalDAV)

Where can I find the Changelog Beta 4 -> Beta 5?

Comment 22

[Florian]

ok found it in bug 1468755:
https://hg.mozilla.org/releases/comm-beta/pushloghtml?fromchange=f51daa333cb3&tochange=ec17f661d2d4

Comment 23

[Jorg K (CEST = GMT+2)]

Sadly that's not right. TB is also built from the mozilla-beta repository and that will have had changes between those two betas. I tried to answer the question in bug 1468755 comment #6:
https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?changeset=481fea2011e6
https://hg.mozilla.org/releases/mozilla-beta/rev/3b208a6419c5

Comment 24

[Florian]

I have tested several (own|next)cloud versions: 
From the release of Nextcloud, with the first version 9.0.50, Thunderbird 60 Beta 5+ no longer synchronizes.
The first 9er Owncloud version 9.0.2 synchronizes with Thunderbird 60 Beta 5 (installation problems with 9.0.0 and 9.0.1) 

I don't know which exact version of owncloud has been forked.

The only perspective seems to be a detailed logging in Apache with.... 
- Thunderbird 60 Beta 4 
- Thunderbird 60 Beta 5 

with owncloud and nextcloud.

Comment 25

[Sisim Biva]

(In reply to Florian from comment #19)

> Should a bug be closed as a duplicate?

@admins: please decide where you prefer to follow up this bug, I don't really care ;) and unfortunately, I'm not skilled enough to help more here :(

Comment 26

[Florian]

Attachment: nextcloud-sync - Apache2 error.log with loglevel trace8

Comment 27

[Florian]

Attachment: owncloud-sync - Apache2 error.log with loglevel trace8

Comment 28

[Florian]

(In reply to Norbert from comment #13)
> Apache log gives the following results while starting Thunderbird:

I added apache2 log files with loglevel trace8, each for:
- nextcloud sync (not working)
- owncloud sync (working)

I also lack background knowledge about the process of the webdav-protocol.


(In reply to Sisim Biva from comment #25)
> @admins: please decide where you prefer to follow up this bug, I don't
> really care ;) and unfortunately, I'm not skilled enough to help more here :(
I do not care, too.

Comment 30

[Sisim Biva]

(In reply to Jorg K (GMT+2) from comment #20)
> Anything we can investigate in TB here?

Did you make progress on this bug? Isn't it preventing the release of TB60?

Comment 31

[Jorg K (CEST = GMT+2)]

The person working on Calendar is on leave and will return next week.

Comment 32

[Christian Rütgers]

@Philipp Kewisch
Issue is not only related to CalDav but also to CardDav as mentioned in the subject of this issue.

Since Thunderbird Beta 60.5 I can no longer sync Calendars (Lightning) and Adressbooks (CardBook) with my Nextcloud server. The problem description "PROPFIND" results in 503 is the same in both cases.

Comment 33

[[:MakeMyDay]]

This is not a general issue, but seems limited to nextcloud, since it was already reported to work with owncloud 10 and it does as well with my davical test server and there seem to be no duplicates for other calendar servers atm.

And this is not a Lightning code issue - it seems to be a cookie handling issue of the network code or the nextcloud server. For the following, please also have a look at the already attached log file in attachment 8991262 [details].

The nextcloud server sets different cookies in the response to the first (Set-Cookie header), unauthorized propfind request in the log file. TB (resp. the network code) now sets the Cookie header mentioning these cookies in the second request along with the Authorization header. With this setup, the request fails and you receive the observed 503 response.

If you resend this request with just the Cookie header dropped (you can use devtool to do so, cURL or whatever you like to send it manually), it is succesful and you receive a 207 response code.

Since you can control the cookie management in TB, there is also a workaround for this issue: disable accpting cookies at all (not sure whther this might cause oauth problems) or at least and preferably for the nextcloud server (data protection setting in TB options). After doing so, a resync of the calendar will be immediately successful.

For everybody who wants to test/verify it, you can abtain a temporary instance of nextcloud at https://demo.nextcloud.com - I used that also to verify the above.

That said, it's probalbly a good idea to point the nextcloud guys to this, so they can check/fix their cookie handling. Norbert/Florian, since you already filed a bug there, can you do this?

And maybe there have been some changes in the Mozilla network code, too, since the above report mentioned this to be a non-issue before b5 (which I can confirm at least for ESR52), although the described handling doesn't look wrong to me. Philipp/Jörg/Geoff, do you have somebody from the network team who could take a look?

Comment 34

[Jorg K (CEST = GMT+2)]

Honza Bambas (:mayhemer) sometimes helps with TB issues.

Comment 35

[Sisim Biva]

Nextcloud issue updated and support asked to NC's developer: https://github.com/nextcloud/server/issues/10134#issuecomment-404960656

Comment 36

[Geoff Lankow (:darktrojan)]

(In reply to [:MakeMyDay] from comment #33)
> And maybe there have been some changes in the Mozilla network code, too,
> since the above report mentioned this to be a non-issue before b5 (which I
> can confirm at least for ESR52), although the described handling doesn't
> look wrong to me. Philipp/Jörg/Geoff, do you have somebody from the network
> team who could take a look?

Could you run the same test in 52/60b4 and see if/how it handles the cookies differently?

Comment 37

[Geoff Lankow (:darktrojan)]

Oops, wrong person.

Comment 38

[Florian]

Attachment: TB52 nextcloud-sync - Apache2 error.log with loglevel trace8

Comment 39

[Florian]

Attachment: TB52 owncloud-sync - Apache2 error.log with loglevel trace8

Comment 40

[Florian]

Attachment: TB60B4 nextcloud-sync - Apache2 error.log with loglevel trace8

Comment 41

[Florian]

Attachment: 60B4 owncloud-Sync apache2 error.log with loglevel trace8

Comment 42

[[:MakeMyDay]]

Thanks for providing the additional logs.

From the header of the 401 response to the inital request in b5

> Date: Tue, 10 Jul 2018 13:10:18 GMT
> Server: Apache/2.4.18 (Ubuntu)
> Set-Cookie: oc21jzsxmqti=tt2e6o21i5jjsto9b62jpkg8v0; path=/nextcloud; HttpOnly
> Expires: Thu, 19 Nov 1981 08:52:00 GMT
> Cache-Control: no-store, no-cache, must-revalidate
> Pragma: no-cache
> Set-Cookie: oc_sessionPassphrase=wcVw6Ewhlzza1KwZmiV2YR0uNd68ZwfRYGWqofRJg5evnfHTtI4ZWeoVI7US9JAIS0jxylG7HEUoygtRNjqvt8StxqV2Ln%2FjLCbgd6Ijya8WGxt%2Fh9KKqgEOHg9ganx%2F; path=/nextcloud; HttpOnly
> Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *
> Set-Cookie: nc_sameSiteCookielax=true; path=/nextcloud; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
> Set-Cookie: nc_sameSiteCookiestrict=true; path=/nextcloud; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
> WWW-Authenticate: Basic realm=\\"sabre/dav\\"
> WWW-Authenticate: Basic realm=\\"sabre/dav\\"
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> X-Robots-Tag: none
> X-Frame-Options: SAMEORIGIN
> X-Download-Options: noopen
> X-Permitted-Cross-Domain-Policies: none
> Content-Length: 414
> Keep-Alive: timeout=5, max=100
> Connection: Keep-Alive
> Content-Type: application/xml; charset=utf-8

the two nc_sameSiteCookie* cookies are not submitted when doing the follow-up request

> Host: 192.168.179.126
> User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.0 Lightning/6.2b5
> Accept: text/xml
> Accept-Language: en-US,en;q=0.5
> Accept-Encoding: gzip, deflate
> Accept-Charset: utf-8,*;q=0.1
> Content-Length: 306
> Content-Type: text/xml; charset=utf-8
> Depth: 0
> Connection: keep-alive
> Pragma: no-cache
> Cache-Control: no-cache
> Authorization: Basic YWRtaW46UGEkJHcwcmQ=
> Cookie: oc21jzsxmqti=tt2e6o21i5jjsto9b62jpkg8v0; oc_sessionPassphrase=wcVw6Ewhlzza1KwZmiV2YR0uNd68ZwfRYGWqofRJg5evnfHTtI4ZWeoVI7US9JAIS0jxylG7HEUoygtRNjqvt8StxqV2Ln%2FjLCbgd6Ijya8WGxt%2Fh9KKqgEOHg9ganx%2F

but were with 52 or b4:

> Cookie: oc21jzsxmqti=f6r055r9oljbelvs6l7ftjqdt1; oc_sessionPassphrase=%2FiSnxtYRIBxMVMfxyR1Mz5da12fRXozVzsWez01JWUXbosuKNNR06RV06O0RBVhneqdBinjWQr1RANhq8KwWVe0S%2FufbR7sV%2BhB8A3lW%2Fj8Wktl8sJm6KPrZEjoG89Nl; nc_sameSiteCookielax=true; nc_sameSiteCookiestrict=true

(the tokens of the first two cookies are of cause expecetdly different here)

This may produce an unexpected situation for the server.

@mayhemer: do have an idea why this could happen?

Comment 43

[Geoff Lankow (:darktrojan)]

You could also try flipping the pref network.cookie.same-site.enabled to false. I'm not sure of the state of this feature when it landed on our beta branch, so hopefully just disabling it will improve things.

Comment 44

[Florian]

Attachment: TB60B10 - nextcloud-sync - pref network.cookie.same-site.enabled set to false - apache2 error.log with loglevel trace8

Comment 45

[Florian]

I can confirm a working TB60B10 with pref network.cookie.same-site.enabled set to false, see log-file.

Comment 46

[Jorg K (CEST = GMT+2)]

For the record:
Pref network.cookie.same-site.enabled was introduced at https://hg.mozilla.org/mozilla-central/rev/38a419def0a7#l5.12 in bug 1452699. That bug landed on mozilla61 and was uplifted/backported to mozilla60.

We merged it into the Thunderbird branch at https://hg.mozilla.org/releases/mozilla-beta/rev/3b208a6419c5#l397.31 for TB 60 beta 5.

You'd have to ask the author/reviewer of bug 1452699 for details:
Christoph and Valentin, can you please tell us something about that pref. It's causing some cookie problems in our CalDav implementation when talking to certain servers (at least that's what I understand).

Comment 47

[Jorg K (CEST = GMT+2)]

This shouldn't be an issue in TB 60 ESR (being released soon) since bug 1452699 (so far) wasn't uplifted to mozilla-esr60.

Comment 48

[Jorg K (CEST = GMT+2)]

Oops, ignore that comment.

Comment 49

[Philipp Kewisch [:Fallen][📱📆]]

(In reply to Christian Rütgers from comment #32)
> @Philipp Kewisch
> Issue is not only related to CalDav but also to CardDav as mentioned in the
> subject of this issue.
Cardbook is an add-on, therefore you will have to report the issue to the Cardbook author who will fix it separately from this bug. I'm modifying this bug to only handle the caldav part in Lightning.

Comment 50

[Sisim Biva]

I confirm that Cardbook works with TB60B10 with pref network.cookie.same-site.enabled set to false

Comment 51

[Christoph Kerschbaumer [:ckerschb, back on Sept 9th]]

(In reply to Jorg K (GMT+2) from comment #46)
> You'd have to ask the author/reviewer of bug 1452699 for details:
> Christoph and Valentin, can you please tell us something about that pref.
> It's causing some cookie problems in our CalDav implementation when talking
> to certain servers (at least that's what I understand).

What do you want to know in particular? We wrote a blogpost (including various links) which might be helpful to get started:
https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/

Comment 52

[Sisim Biva]

Any update on this issue? Do you want to solve it before TB60 release?

Comment 53

[Jorg K (CEST = GMT+2)]

Isn't the solution to set the preference like we already mention in the release notes:
https://www-stage.thunderbird.net/en-US/thunderbird/60.0beta/releasenotes/

We could switch it off by default. MMD, do you have an opinion? Can you explain the issue to me please. What's special about the NextCloud cookie handling and why does it run into "same site" issues in the first place?

Comment 54

[[:MakeMyDay]]

Since this is a security feature, we should not disable it by default even though the attack vector for caldav/carddav use is limited, but people may use other addons to browse with TB.

If TB/ the network code worked here flawlessly, this can only be an issue of the server to not hanlde it correctly, so we should relnote about the known issue for Nexcloud in the versions mentioned in this bug and advise the user to (only) flip the pref if he/she is seeing the issue.

There is no report yet for any other caldav/dcarddav server and we have reports for some products that it works. However, this doesn't mean there would be no issue with the network code, since these products probably don't use the samesite feature, but since samesite is available in FF since version 60 there should have been bug reports already if something doesn't work (and I used FF to check with demo.nextcloud.com and got no warnings/problems when doing so).

Comment 55

[[:MakeMyDay]]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #51)
> What do you want to know in particular? We wrote a blogpost (including
> various links) which might be helpful to get started:
> https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/

Christoph, thank you for pointing to the blog post and offering your assistance. In the findings wrapped up in comment 42, the requests were fired against the same ip address, so no domain name involved. Can you check the server response there and the TB response wether these are appropriate with respect to the rfc so we can exclude at least one side to be responsible for the behaviour - from what I read there, I'm not sure whether it's ok to send strict and lax at the same time?

On the other hand, that should have been an issue for FF as well. Is there a special hanlding for exotic http/dav methods like PROPFIND, OPTIONS or REPORT?

Another thought: since we send our request from javascript (both Lightning and cardbook are technically addons, even Lightning is shipped bundled with TB), could that prevent the cookies from being set by the network code correctly?

Comment 56

[Christoph Kerschbaumer [:ckerschb, back on Sept 9th]]

(In reply to [:MakeMyDay] from comment #55)
> (In reply to Christoph Kerschbaumer [:ckerschb] from comment #51)
> > What do you want to know in particular? We wrote a blogpost (including
> > various links) which might be helpful to get started:
> > https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/
> 
> Christoph, thank you for pointing to the blog post and offering your
> assistance. In the findings wrapped up in comment 42, the requests were
> fired against the same ip address, so no domain name involved. Can you check
> the server response there and the TB response wether these are appropriate
> with respect to the rfc so we can exclude at least one side to be
> responsible for the behaviour - from what I read there, I'm not sure whether
> it's ok to send strict and lax at the same time?
> 
> On the other hand, that should have been an issue for FF as well. Is there a
> special hanlding for exotic http/dav methods like PROPFIND, OPTIONS or
> REPORT?
> 
> Another thought: since we send our request from javascript (both Lightning
> and cardbook are technically addons, even Lightning is shipped bundled with
> TB), could that prevent the cookies from being set by the network code
> correctly?

Mark, is the author of the spec; 302 to him on your questions above.

Comment 58

[[:MakeMyDay]]

Mark, any chance that you can comment? We're about to release TB 60 and would like to take this to the release notes.

Comment 59

[Sisim Biva]

Maaaaaaaaaaark, we neeeeeeeed youuuuuuuuuuuu :)

Comment 60

[Sisim Biva]

Hum, TB60 has been released and this is the single "Known Issues" in the release notes https://www.thunderbird.net/en-US/thunderbird/60.0/releasenotes/

What a pity :( Any chance to fix that soon?

Comment 61

[[:MakeMyDay]]

At this point, it's not clear whether there is something to fix at our end at all. And until this changes, relnoting the issue with a workaround recommendation for affected users is an absolutely valid approach to deal with the situation.

And if you're looking over to the NC issue, with not much of activity visible to analyse the issue on the NC side, the answer to your question is probably no. But let's see how things are going with TB60 being releaseed now.

Comment 62

[Sisim Biva]

You're right @MakeMyDay and I understand your strategy to release TB. I'm just saying it's a pity that we could not solve this issue, while it has been reported 2 months ago. If it had been solved, you could have released a "clean" TB ;)

I hope NC developers will have a look soon (I messaged them again this morning). On your side, are we still waiting for a comment from Mark ([e-mail])?

Comment 63

[Mark Goodwin [:mgoodwin]]

(In reply to [:MakeMyDay] from comment #58)
> Mark, any chance that you can comment? We're about to release TB 60 and
> would like to take this to the release notes.

Apologies for the delay, I've been off work for a while following an accident.

From the logs, it looks like TB *should* be sending those cookies. I'll get TB60 running so I can try to track down what's up.

(leaving the ni? so I don't forget to follow up)

Comment 64

[Martin Fink]

Hey,

we also encountered this Problem after upgrading TB to 60.0 but only on Windows Maschines.
we have a nextcloud instance hosting calendars for a department.
2 new members got TB 60 fresh installed and i was wondering why the calenders didnt work anymore.
on my maschine however (archlinux) with TB 60 calendar and task are working fine.
the only other difference is that my TB is english and the windows ones are in german.

hope that helps.

Comment 65

[Stefan Sitter [:ssitter]]

(In reply to Martin Fink from comment #64)

Have you tried the "set pref network.cookie.same-site.enabled to false" workaround as noted on top of this bug report and on https://www.thunderbird.net/en-US/thunderbird/60.0/releasenotes/#known-issues ?

Comment 66

[Martin Fink]

(In reply to Stefan Sitter [:ssitter] from comment #65)
> (In reply to Martin Fink from comment #64)
> 
> Have you tried the "set pref network.cookie.same-site.enabled to false"
> workaround as noted on top of this bug report and on
> https://www.thunderbird.net/en-US/thunderbird/60.0/releasenotes/#known-
> issues ?

Hey, yes the workaround worked fine.

Comment 67

[Sisim Biva]

Hello Mark, happy to see you back. Did you make any progress? Are you in contact with the Nextcloud team here https://github.com/nextcloud/server/issues/10134 ?

Comment 69

[lars]

I can confirm this with Thunderbird 60.2.1 and Lightning 6.2.2.1 connecting to Nextcloud 14.0.3 (over apache 2.4.18 and https).

I had the error "[calCachedCalendar] replay action failed: null, uri=https://foo.bar/, result=2147500037, operation=[xpconnect wrapped calIOperation]" when starting thunderbird from terminal.
Disabling offline support made this error go away, but did not help otherwise.

The workaround to set network.cookie.same-site.enabled to false helped.

Comment 72

[christian.kohler]

Hi!
I am not 100% sure if my problem is related to this reported bug. But I cannot synchronize my nextcloud (14.0.4) calendar with Thunderbird 60.3.2 (32-Bit) / Lightning 6.2.3.3. Neither with enabled or disabled network.cookie.same-site. With a web browser I can connect to the nc WebDAV page, getting the message that this is for WebDAV clients only. So I suppose in principle it should be possible to connect to the calendar. 
As the nc installation is on a hosted / shared webspace (in some subfolder) I do not have too many debug options. But as far as I could see there are no errors reported in the logs provided. In TB it only states that the calendar is temporarily unavailable. Is there any option to investigate this further? 

Christian

Comment 73

[gol3m]

@christian.kohler(In reply to christian.kohler from comment #72)
> cannot synchronize my nextcloud (14.0.4) calendar with Thunderbird 60.3.2
> (32-Bit) / Lightning 6.2.3.3. Neither with enabled or disabled
> network.cookie.same-site.

CardDAV and CalDAV work fine for me with network.cookie.same-site.enabled = false in NC 14.0.4 with TB 60.3.2 (32-bit) and integrated Lightning 6.2.3.2 (not 6.2.3.3).


I used the following URL: https://example.org/remote.php/dav/calendars/MYUSERNAME/personal/

You could try connecting with another client to see whether CalDAV works at all. DAVdroid for Android has been a reliable client (free on f-droid, donations are encouraged).

Comment 74

[christian.kohler]

(In reply to gol3m from comment #73)
> @christian.kohler(In reply to christian.kohler from comment #72)
> > cannot synchronize my nextcloud (14.0.4) calendar with Thunderbird 60.3.2
> > (32-Bit) / Lightning 6.2.3.3. Neither with enabled or disabled
> > network.cookie.same-site.
> 
> CardDAV and CalDAV work fine for me with network.cookie.same-site.enabled =
> false in NC 14.0.4 with TB 60.3.2 (32-bit) and integrated Lightning 6.2.3.2
> (not 6.2.3.3).
> 
> 
> I used the following URL:
> https://example.org/remote.php/dav/calendars/MYUSERNAME/personal/
> 
> You could try connecting with another client to see whether CalDAV works at
> all. DAVdroid for Android has been a reliable client (free on f-droid,
> donations are encouraged).

Hi!
Thanks for the reply, I made a donation ;-) DAVDroid works like charm and I found some information in DAVdroid, that helped me getting things working with TB/L as well. I share this here in case someone else has the same problem.
Nextcloud pointed me to a url of the type https://mydomainname/firstsubfolder/secondsubfolder/remote.php/dav/ which worked fine in DAVdroid but caused the problem described in TB. But in the DAVdroid properties of the calendar I found a little extended version of this url: https://mydomainname/firstsubfolder/secondsubfolder/remote.php/dav/calendars/myname/personal. Using this url in TB and everything works, even with network.cookie.same-site.enabled = true
Cheers
Chris

Comment 76

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

I had this issue using Baikal (Sabre's dav implementation).

Setting network.cookie.same-site.enabled = false worked around the problem.

Owncloud, Nextcloud and Baikal all use the Sabre/dav server, which is the most popular WebDAV framework for PHP. Many apps use it to create WebDAV, CalDAV and CardDAV servers.

Just pointing out that this affects more than just Nextcloud but the issue could be with the Sabre/dav server.

Comment 77

[Mark Goodwin [:mgoodwin]]

OK, so the issue in this case is here:

https://searchfox.org/comm-central/rev/962ac074635de88addaa6ac8d19dc701d93aa4a2/calendar/base/modules/utils/calProviderUtils.jsm#42

The issue is that because we're using a system principal, there's no parentURI and so when we do the third party checks here:

https://searchfox.org/mozilla-central/rev/dac799c9f4e9f5f05c1071cba94f2522aa31f7eb/dom/base/ThirdPartyUtil.cpp#220

the call fails which, in turn, means that IsThirdPartyChannel exits with parentIsThird set to true.

The correct thing to do in this case is, from my understanding, to not use a system principal for fetching the resource. Christoph may be able to elaborate on this.

Comment 78

[Christoph Kerschbaumer [:ckerschb, back on Sept 9th]]

(In reply to Mark Goodwin [:mgoodwin] from comment #77)

OK, so the issue in this case is here:

https://searchfox.org/comm-central/rev/962ac074635de88addaa6ac8d19dc701d93aa4a2/calendar/base/modules/utils/calProviderUtils.jsm#42

Ideally we could pass a context to newChannelFromURI2. If that is not possible for that case, or too hard to accomplish we could fix the problem by actually passing a codebasePrincipal instead of the systemPrincipal. So something like this:

let triggeringPrincipal = Services.scriptSecurityManager.createCodebasePrincipal(aUri, {});

and then us triggeringPrincipal instead of the systemprincipal. Probably worth adding a comment explaining that we are generating a principal from the uri we are about to load to fix that same-site cookie issue.

Comment 79

[[:MakeMyDay]]

Marc and Christoph, thank you for the pointer and elaborating on a potential fix. Since this is a sync operation and not a request to display something directly, using the separate principle seems to be the right thing here.

Comment 80

[[:MakeMyDay]]

Attachment: DontUseSystemPrincipleForCalendarServerConncetion-V1.diff

With the attached patch, I was able to sync successfully with a NC demo calendar created at [1] and having network.cookie.same-site.enabled set to true (while the same fails without the patch). Calendars on my Davical test server works with and without it. The same is true for an ics holiday calendar.

[1] https://demo.nextcloud.com

Comment 81

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 9037870 [details] [diff] [review]
DontUseSystemPrincipleForCalendarServerConncetion-V1.diff

Review of attachment 9037870 [details] [diff] [review]:
-----------------------------------------------------------------

::: calendar/base/modules/utils/calProviderUtils.jsm
@@ +38,5 @@
>       */
>      prepHttpChannel: function(aUri, aUploadData, aContentType, aNotificationCallbacks, aExisting=null) {
> +        // We cannot use a system principle here, since the conncetion setup would fails if
> +        // same-site cookie protection is enabled in TB and server-side.
> +        let principle = aExisting ? null

principal or principle? :)

Comment 82

[[:MakeMyDay]]

Attachment: DontUseSystemPrincipalForCalendarServerConncetion-V2.diff

(In reply to Philipp Kewisch [:Fallen] [:📆] from comment #81)

principal or principle? :)

In principle, principal.

Comment 83

[[:MakeMyDay]]

Comment on attachment 9038080 [details] [diff] [review]
DontUseSystemPrincipalForCalendarServerConncetion-V2.diff

In case we will have a b4 for TB65, this would be a candidate for an uplift.

Comment 84

[Jorg K (CEST = GMT+2)]

We will, see: https://mzl.la/2RHUrtC, 8 bugs, 9 with this one.

Comment 85

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/94a2a4100e98
Don't use system principal for calendar server connections; r=philipp

Comment 86

[Jorg K (CEST = GMT+2)]

Comment on attachment 9038080 [details] [diff] [review]
DontUseSystemPrincipalForCalendarServerConncetion-V2.diff

Review of attachment 9038080 [details] [diff] [review]:
-----------------------------------------------------------------

::: calendar/base/modules/utils/calProviderUtils.jsm
@@ +37,5 @@
>       * @return {nsIChannel}                                     The prepared channel
>       */
>      prepHttpChannel: function(aUri, aUploadData, aContentType, aNotificationCallbacks, aExisting=null) {
> +        // We cannot use a system principal here, since the conncetion setup would fail if
> +        // same-site cookie protection is enabled in TB and server-side.

You have three grammar/punctuation/spelling errors in one sentence :-(

1) There is no comma before "since".
2) conncetion is misspelled.
3) Setup would fail if such-and-such *were* enabled, or, setup *will* fail if such-and-such is enabled.

Comment 87

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/30006894ce43
Follow-up: Fix comment. rs=comment-only,typo-fix DONTBUILD

Comment 88

[Jorg K (CEST = GMT+2)]

TB 65 beta 4, Cal 6.7:
https://hg.mozilla.org/releases/comm-beta/rev/108b096028369815e4b847f4ec88ead5664d69b9 (incl. spelling fixes)

Comment 89

[j94305]

It looks like the issue reappeared with TB 60.5.0 (32-bit). It's unable to sync CalDAV with Google Calendars.
Setting the config flag to false did not help.
If this issue should be unrelated to the issue dealt with in this thread, I'll open a new bug report.

Comment 90

[[:MakeMyDay]]

No, it doesn't - the fix hasn't yet been backported to ESR60 but is available with the latest beta version.

If you want to help out to verify everything is working as expected, download a the current TB beta [1] and test it with a new profile (if you want to test it with your current ESR profile, run the test only on a copy of the same to avoid any harm to your production profile. If you want to go that way, you will need to reset Lightning according to [2] since profile downgrading is not supported).

[1] https://www.thunderbird.net/channel/
[2] https://support.mozilla.org/en-US/kb/calendar-updates-issues-thunderbird#w_lightning-disappears-after-a-thunderbird-update-release-and-beta-versions

Comment 91

[Jorg K (CEST = GMT+2)]

I've come here to uplift this, so I'm looking at the code again:

+        let principal = aExisting ? null
+                                  : Services.scriptSecurityManager.createCodebasePrincipal(aUri, {});

This doesn't look correct. aExisting is described as @param {?nsIChannel} aExisting An existing channel to modify (optional). So why is it OK to pass in a channel where a principal is required?

Comment 92

[[:MakeMyDay]]

Please read the entire patch (and the code for motre context if needed). In case aExisting is passed, we're not creating a new channel but use the existing one, thus we don't need a principal. The principal creation is just moved to a seperate variable to not exceed the line length.

Comment 93

[Jorg K (CEST = GMT+2)]

Sorry, I got up too early and had "tomatoes on my eyes" (German idiom). Of course, if the existing channel is passed, you use null for the principal parameter. My apologies! Will uplift and include in the release notes. Sorry again.

Comment 94

[Jorg K (CEST = GMT+2)]

TB 60.5.1 / Cal 6.2.5.1
https://hg.mozilla.org/releases/comm-esr60/rev/2fb08caa568b5905461e7d49c88c7b8e57d85f27

Comment 95

[[:MakeMyDay]]

Philipp, we still need 6.2.5.1 target milestone and version.

Comment 96

[achim]

Hello bugzilla group,

In our environment we use an network based profile for thunderbird whic is used by windows 7 pro 32 bit and 64 bit clients. No issues with this for a few years. The changes in 62.5.1 cause the following. The lightning addon onyl works on 32 or only on 64 bit clients. If it worked on 32 bit clients the addon must be deactivated and activated on an 64 bit system again to make it work on the 64 bit platform and vice versa. Downgrading to 6.2.5 fixes this issue.

Comment 97

[Timo Eissler]

I'm still unable to sync to Nextcloud with network.cookie.same-site.enabled set to true.

Thunderbird 60.6.0
Fedora 29

Nextcloud 14.0.3

Someone tried to sync after setting network.cookie.same-site.enabled back to true?

Comment 98

[Sebastian Köhlmeier]

(In reply to Timo Eissler from comment #97)

I'm still unable to sync to Nextcloud with network.cookie.same-site.enabled set to true.

Thunderbird 60.6.0
Fedora 29

Nextcloud 14.0.3

Someone tried to sync after setting network.cookie.same-site.enabled back to true?

Tested with TB 60.6.1 on Win10 it seems to work now.

Comment 99

[bzzz]

(In reply to Timo Eissler from comment #97)

I'm still unable to sync to Nextcloud with network.cookie.same-site.enabled set to true.

Someone tried to sync after setting network.cookie.same-site.enabled back to true?

I tried. I am still unable to sync with Nextcloud.

Tested with

• Thunderbird 60.6.1 (64-Bit)
• Lightning 6.2.5
• Ubuntu 16.04.6
• Nextcloud 16.0.0 and 13.0.10

Comment 100

[Jorg K (CEST = GMT+2)]

Note that network.cookie.same-site.enabled was removed in bug 1551821 in the mozilla68 cycle.

Comment 101

[sunbird]

I cannot sync. I have network.cookie.same-site.enabled set true. Same problem if set to false. This is on TB 60.8.0 and Lightning 6.2.8. This is on Debian Buster.