1458475 - Synchronize CardBook with Synology CARDDAV Server

Status: RESOLVED INVALID

(Thunderbird :: Untriaged, defect)

Description

[kumstrecker]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299

Steps to reproduce:

Hello,

I'am using thunderbird with CardBook since some months.

For some days ago I add a new addressbook on my synology DS418 (CardDav Server - latest Software version is installed). Then I synchronized the CardDav Server with Outlook 2010(round about 990 contacts) with addon CALDAV Synchronize on outlook. The contacts now on the DS418 and I can synchronize android with DAVDROID.

Later I create the CARDBOOK adressbook on Thunderbird and tryit with standard adressbook too on thunderbird. The first problem, I can not connect(proof) the connection to the server DS418. When I delete the contacts from the DS418 then it possible to create the adressbook in Thunderbird. I can add some contacts and synchronize and all is fine. When I add all contacts from outlook then the synchronize from cardbook will never work. 
I tryit with outlook and 2300 contacts and it works but with thunderbird a found no way to do this.

The developer CardBook let me know he is thinking this is a Thunderbird bug.

In thunderbird I have the following error information under network.

XML-Verarbeitungsfehler: Kein Wurzel-Element gefunden
Adresse: https://ms-ds214play.synology.me:8443/addressbooks/users/DASYM/addressbook/
Zeile Nr. 1928, Spalte 4:

Comment 1

[Jorg K (CEST = GMT+2)]

Well, if the CardBook developer thinks it's a TB bug, he should diagnose it further. We're not going to install CardBook and try to track it down.

Comment 2

[Philippe VIGNEAU]

yes the response sent by the Synology server is truncated in Thunderbird... see the snapshot...

Comment 3

[Philippe VIGNEAU]

Attachment: console log

Comment 4

[Philippe VIGNEAU]

@kumstrecker : may you check this : https://cardbook.6660.eu/forums/topic/synology-and-return-code-0/

Comment 5

[Philippe VIGNEAU]

I have reproduced the problem which happens even if there is no message like :
Strict-Transport-Security: The connection to the site is untrustworthy, so the specified header was ignored.

I get this error in the console (which is not related to CardBook) :

addressbook:3538:18

XML Parsing Error: no root element found
Location: https://ms-ds214play.synology.me:8443/addressbooks/users/CBTest/addressbook/
Line Number 3538, Column 18:

may someone told which is the file "addressbook" that is called ?

Comment 6

[lorenzo.valori]

Hi, I tried to do several tests, and I noticed that disabling the ssl and using port 8008 synchronization works again

if they could help with SSL enabled the error logs were these :

Cardbook :

2019.03.01 23:56:02:715 : Validation module : Sincronizzazione non riuscita (Fase: discoverPhase1, URL: https://example.com:8443/addressbooks/users/example/addressbook/.well-known/carddav, Stato: 404)

Synology :

2019-03-01 23:56:24+0100 [-] [caldav-0] [HTTPChannel,7,::ffff:192.168.60.3] [twext.web2.server#info] PROPFIND /addressbooks/users/Lorenzo/addressbook/ HTTP/1.1
2019-03-01 23:56:24+0100 [-] [caldav-0] [HTTPChannel,7,::ffff:192.168.60.3] [twistedcaldav.directory.nss.NssUserRecord#debug] 1551480984.57
2019-03-01 23:56:24+0100 [-] [caldav-0] [HTTPChannel,7,::ffff:192.168.60.3] [twistedcaldav.directory.nss.NssUserRecord#debug] syno - PAM username Lorenzo
2019-03-01 23:56:24+0100 [-] [caldav-0] [HTTPChannel,7,::ffff:192.168.60.3] [twistedcaldav.memcacher.Memcacher#debug] Getting Cache Token for 'E8C37213-479A-577A-8A8A-BFC6AE9983F4'
2019-03-01 23:56:24+0100 [-] [caldav-0] [-] [twistedcaldav.memcacher.Memcacher#debug] Setting Cache Token for 'E8C37213-479A-577A-8A8A-BFC6AE9983F4'
2019-03-01 23:56:41+0100 [-] [caldav-0] [-] [twext.web2.channel.http#error] Connection aborted - took too long to close: IPv6Address(TCP, '::ffff:192.168.60.3', 23495)
2019-03-01 23:56:42+0100 [-] [caldav-0] [HTTPChannel,8,::ffff:192.168.60.3] [twext.web2.server#info] PROPFIND /addressbooks/users/Lorenzo/addressbook/.well-known/carddav HTTP/1.1
2019-03-01 23:56:42+0100 [-] [caldav-0] [HTTPChannel,8,::ffff:192.168.60.3] [twistedcaldav.directory.nss.NssUserRecord#debug] 1551481002.64
2019-03-01 23:56:42+0100 [-] [caldav-0] [HTTPChannel,8,::ffff:192.168.60.3] [twistedcaldav.directory.nss.NssUserRecord#debug] syno - PAM username Lorenzo
2019-03-01 23:56:42+0100 [-] [caldav-0] [HTTPChannel,8,::ffff:192.168.60.3] [twistedcaldav.memcacher.Memcacher#debug] Getting Cache Token for 'E8C37213-479A-577A-8A8A-BFC6AE9983F4'
2019-03-01 23:56:42+0100 [-] [caldav-0] [-] [twistedcaldav.memcacher.Memcacher#debug] Setting Cache Token for 'E8C37213-479A-577A-8A8A-BFC6AE9983F4'

Comment 7

[Koen Breeman]

Dear Thunderbird

I've got the same issues with the truncated responses. When I connect without using SSL everything works fine, but as soon as I connect using SSL everything stops working. I also noticed that this has to do with the amount of contacts which are present in the addressbook. I started with a new addressbook and added 1 contact, that worked also with a SSL connection, than I added several contact and for me now it's around 200 contacts and then the plugin stopped working and as Phillipe already pointed out, it's very likely that the response is truncated.

If someone could help us with this issue. I'm more than willing to supply any information, logs, etc.

Awaiting a response.

Comment 8

[Philippe VIGNEAU]

try these workarounds :
1/ either use http
2/ either try this : https://github.com/jobisoft/DAV-4-TbSync/issues/104

Comment 9

[Koen Breeman]

Hi Philippe

Many thanks. I'm aware of this work around, you have send it to me privately as well. I've also contacted Synology and they have investigated it and they say that there's nothing going wrong on the server side (carddav server side). So they concluded that TB truncates the message.

Using a reverse proxy from within the Synology setup will work, but I still would like to assist in solving this matter in TB, so that the work-around is not necessary and it works right out of the box

regards

Koen

Comment 10

[Magnus Melin [:mkmelin]]

From comment 8 and elsewhere, sounds like a server bug. (Thunderbird won't accept invalid xml.)

Comment 11

[Koen Breeman]

Hi Magnus

I've contacted the developer, Philippe and Synology.
Philippe mentioned to me that he is unsure if the problem arises from Thunderbird truncating the xml, or the Synology Carddav server truncating the xml. Synology is still in the process of investigating the truncation and I was hoping that the Thunderbird community was doing the same.

Regards

Koen

Comment 12

[Magnus Melin [:mkmelin]]

Hope it gets sorted out. Bugzilla only tracks problem in core code though, and since CardDAV is not yes supported this ends up as either a server bug or a bug in an the CardBook add-on we won't track it here.

Comment 13

[Koen Breeman]

Dear Magnus and others

I've asked Synology to look into it. And based on what they say all the addressbooks are fetched correctly, but it seems that Thunderbird itself truncates the response, since both TBSync and Cardbook have the same issues, but it also might be that both developers of the add-ons made a mistake.

This is the reply I got from Synology to fetch all the carddav data:

[1] Open Terminal and use command to encode the account:password info and you will get a string.

echo -n 'account:password' | openssl base64

[2]

curl --request PROPFIND --header "Depth: 1" --header "Authorization: Basic String generate in the first step"
--data "<?xml version='1.0' encoding='utf-8'?>
<D:propfind xmlns:D='DAV:'>
<D:prop>
<D:getcontenttype/>
<D:getetag/>
</D:prop>
</D:propfind>"
-i https://nas_ip:port/url_of_your_addressbook -k

This will get the whole content of the addressbook (according to Synology) and since it is all fetched, I think the problem arises either in Thunderbird or in both add-ons (but I find they less likely, since that would mean that 2 seperate developers would have made the same mistake).

I don't know how to retrieve the data from within Thunderbird, or how to test it. Can someone assist in this matter?

Regards

Koen