Last Comment Bug 1197320 - Free/Busy information of CalDAV calendar is no longer displayed after latest Lightning Calendar's update (v4.0.2)
: Free/Busy information of CalDAV calendar is no longer displayed after latest ...
Status: RESOLVED FIXED
: regression
Product: Calendar
Classification: Client Software
Component: Provider: CalDAV (show other bugs)
: Lightning 4.0.2
: x86_64 Windows 7
-- normal (vote)
: 4.0.2.1
Assigned To: [:MakeMyDay]
:
:
Mentors:
: 1201477 1203998 (view as bug list)
Depends on: 1204255 1205050
Blocks: ltn4021 1195993 1199942
  Show dependency treegraph
 
Reported: 2015-08-21 11:56 PDT by rebug
Modified: 2015-09-15 13:58 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
FixFBLookup-V1.diff (1.14 KB, patch)
2015-08-27 14:25 PDT, [:MakeMyDay]
philipp: review+
Details | Diff | Splinter Review
FixFBLookup-esr38-V1.diff (1.14 KB, patch)
2015-08-27 14:31 PDT, [:MakeMyDay]
philipp: review+
Details | Diff | Splinter Review
FixFBLookup-esr38-V2.diff (2.43 KB, patch)
2015-08-30 11:58 PDT, [:MakeMyDay]
philipp: review+
philipp: approval‑calendar‑esr+
Details | Diff | Splinter Review
FixFBLookup-V2.diff (4.55 KB, patch)
2015-08-30 12:11 PDT, [:MakeMyDay]
no flags Details | Diff | Splinter Review
FixFBLookup-V3.diff (4.54 KB, patch)
2015-08-31 12:10 PDT, [:MakeMyDay]
philipp: review+
Details | Diff | Splinter Review
FixFBLookup-V4.diff (4.52 KB, patch)
2015-09-05 14:26 PDT, [:MakeMyDay]
makemyday: review+
philipp: approval‑calendar‑aurora+
philipp: approval‑calendar‑beta+
Details | Diff | Splinter Review

Description User image rebug 2015-08-21 11:56:40 PDT
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.8.0
Build ID: 20150701023259

Steps to reproduce:

Client-side OS platform: MS Windows 7 SP1
Client-side software involved: Lightning 4.0.2 running on Mozilla Thunderbird 38.* (tested with 38.0.1, 38.1.0 and 38.2.0)

Server-side OS platform: Debian GNU/Linux 8 (Jessie)
Server-side software involved: DaviCal 1.1.3.1

Steps to reproduce the bug:

1. Install/Activate Lightning 4.0.2 Add-on on any supported Thunderbird version (38.*)
2. Add a CalDAV calendar if not already present
3. Create a new event
4. Click on "Invite atendees"
5. Select an atendee


Actual results:

Actual result: The atendee's free/busy information no longer appear


Expected results:

Expected result: The atendee's free/busy information is properly displayed

Important Remark: This bug was not present on previous Lightning versions (<= 4.0.1.2)
Comment 1 User image [:MakeMyDay] 2015-08-22 01:21:47 PDT
Do you have any messages in the error console when reproducing (ctrl+shift+j)?

Do you have network access when trying this? Is offline support enabled or disabled? Or in short: are you affected by bug 1195974 when reproducing the issue?
Comment 2 User image rebug 2015-08-22 05:18:52 PDT
There is one message on the error console that appears after clicking on "Invite atendees". My Thunderbird installation is not in english, but can be translated as:

<<The use of "Mutation Events" is deprecated. It is better to use "MutationObserver".
chrome://calendar/content/calendar-event-dialog-attendees.js		Line:  32 >>

However, I want to point out that this message also appears on the previous Lightning version (4.0.1.2), where the atendee's free/busy information is properly displayed. Thus it seems unrelated to the issue.

I have network access when trying this. The issue is present either if offline support is enabled or disabled.

On the Lightning versions not affected by this bug, when I'm typing the attendee's email address, I can see how the timetable where the free/busy information is shown "blinks". I guess that while typing, Lightning is querying the CalDAV server, trying to find a match for the typed email. When the whole email address has been entered and it is found on the CalDAV server, then the free/busy info of the atendee is shown. However, in 4.0.2 I'm not able to see this "blinking" behaviour anymore, it looks like Lightning is no longer querying the server while typing.


About the bug 1195974, yes I'm affected too. However the described behaviour is only present after a Thunderbird restart. I mean that, if Thunderbird is already open and I disable the network, my CalDAV calendar continue to be displayed, and I can even create new events. The calendar is displayed even after clicking on "synchronize calendars". However, if Thunderbird is restarted (while the network is already disabled), then my CalDAV calendar presents the behaviour described by the bug 1195974. A yellow triangle appears next to the calendar's name showing that "the calendar is temporarily unavailable", and no events are displayed. The events created after network disconnect and before TB restart are lost (I mean, they are not saved locally and written to the CalDAV server once the network is again available).
Comment 3 User image rebug 2015-08-22 05:35:16 PDT
 -- correction about my last comment on bug 1195974: I have now tried to write an event while the network is down (but before Thunderbird's restart, thus when the calendar events are still displayed), and after reconnecting the network, the event appears. Thus the events created on offline mode aren't lost. --
Comment 4 User image [:MakeMyDay] 2015-08-22 07:47:43 PDT
Thanks. No need to discuss the issue of bug 1195974 within this one. This is most probably a side effect of that bug. But to be sure there's no other issue, can you please enable calendar.debug.log and calendar.debug.log.verbose in the config editor and post what you get in the log when issuing a FB query? I was not able to reproduce the issue with a similar setup with an older Davical version.
Comment 5 User image rebug 2015-08-23 04:16:57 PDT
After following your instructions the error console display far more information before clicking on "Invite atendees", but there is no single message logged after clicking on "Invite atendees" beside the one that I posted on my first answer ("The use of Mutation Events is deprecated"), even when the email address of the atendee is being typed or is already typed.

However, when I do the same process with 4.0.1.2, there are a lot of VFREEBUSY requests recorded as messages on the error console. The first one is sent to myself, because as a organizer of the event my email address appear by default on the top of the atendee's list (and my free/busy information immediately appear on screen). And then requests are sent to the email addresses being entered (and their free/busy info appears when they exist on the CalDAV server).

Thus I can only post the messages that appear on Lightning 4.0.1.2 after clicking on "Invite atendee" and do --not-- appear on Lightning 4.0.2 :



CalDAV: send (Originator=mailto:rebug@gmx.com,Recipient=mailto:rebug@gmx.com): BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REQUEST
BEGIN:VFREEBUSY
DTSTAMP:20150823T095400Z
ORGANIZER:mailto:rebug@gmx.com
DTSTART:20150821T220000Z
DTEND:20150906T220000Z
UID:e95df673-0ec2-4f86-87b5-706387eb8e9a
ATTENDEE;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL:mail
 to:rebug@gmx.com
END:VFREEBUSY
END:VCALENDAR

--------------------------------------------------------------------------------------------------
-------	NEXT MESSAGE RECORDED ON "error console" FOLLOWS	----------
--------------------------------------------------------------------------------------------------

CalDAV: recv: <?xml version="1.0" encoding="utf-8" ?>
<C:schedule-response xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
 <C:response>
  <C:recipient>
   <href>mailto:rebug@gmx.com</href>
  </C:recipient>
  <C:request-status>2.0;Success</C:request-status>
  <C:calendar-data>BEGIN:VCALENDAR
PRODID:-//davical.org//NONSGML AWL Calendar//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:REPLY
BEGIN:VFREEBUSY
DTSTAMP:20150823T095425Z
DTSTART:20150821T220000Z
DTEND:20150906T220000Z
FREEBUSY:20150822T120000Z/20150822T123000Z
FREEBUSY:20150822T143000Z/20150822T150000Z
FREEBUSY:20150824T091500Z/20150824T094500Z
FREEBUSY:20150827T061500Z/20150828T061500Z
FREEBUSY:20150828T103000Z/20150828T110000Z
FREEBUSY:20150902T064500Z/20150902T071500Z
UID:e95df673-0ec2-4f86-87b5-706387eb8e9a
ORGANIZER:mailto:rebug@gmx.com
ATTENDEE;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL:
 mailto:rebug@gmx.com
END:VFREEBUSY
END:VCALENDAR
</C:calendar-data>
 </C:response>
</C:schedule-response>

--------------------------------------------------------------------------------------------------
-------	NEXT MESSAGE RECORDED ON "error console" FOLLOWS	----------
--------------------------------------------------------------------------------------------------

CalDAV: send (Originator=mailto:rebug@gmx.com,Recipient=mailto:rebug2@gmx.com): BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REQUEST
BEGIN:VFREEBUSY
DTSTAMP:20150823T095607Z
ORGANIZER:mailto:rebug@gmx.com
DTSTART:20150821T220000Z
DTEND:20150906T220000Z
UID:938a0f96-e6e8-405a-a080-4af582ae3c48
ATTENDEE;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL:mail
 to:rebug2@gmx.com
END:VFREEBUSY
END:VCALENDAR

--------------------------------------------------------------------------------------------------
-------	NEXT MESSAGE RECORDED ON "error console" FOLLOWS	----------
--------------------------------------------------------------------------------------------------

CalDAV: recv: <?xml version="1.0" encoding="utf-8" ?>
<C:schedule-response xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
 <C:response>
  <C:recipient>
   <href>mailto:rebug2@gmx.com</href>
  </C:recipient>
  <C:request-status>2.0;Success</C:request-status>
  <C:calendar-data>BEGIN:VCALENDAR
PRODID:-//davical.org//NONSGML AWL Calendar//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:REPLY
BEGIN:VFREEBUSY
DTSTAMP:20150823T095632Z
DTSTART:20150821T220000Z
DTEND:20150906T220000Z
FREEBUSY:20150822T113000Z/20150822T120000Z
FREEBUSY:20150822T121500Z/20150822T124500Z
FREEBUSY:20150822T141500Z/20150822T144500Z
FREEBUSY:20150822T151500Z/20150822T154500Z
UID:938a0f96-e6e8-405a-a080-4af582ae3c48
ORGANIZER:mailto:rebug@gmx.com
ATTENDEE;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL:
 mailto:rebug2@gmx.com
END:VFREEBUSY
END:VCALENDAR
</C:calendar-data>
 </C:response>
</C:schedule-response>
Comment 6 User image Sébastien Le Ray 2015-08-26 04:56:52 PDT
Same issue here with Davical 1.1.3.1-1 (debian package), TB 38.2.0, lightning 4.0.2.
No trace in log except the Observer warning, no server access
Comment 7 User image Sébastien Le Ray 2015-08-26 06:40:29 PDT
After some dirty tracking by randomly adding logs, it seems that the issue is here http://hg.mozilla.org/comm-central/annotate/e6c5bce2b8fb/calendar/providers/caldav/calDavCalendar.js#l2402
getFreeBusy seems to assume that aCalId is a mailto: URI whereas it's not (no mailto prefix in my tests).
Comment 8 User image Stefan Sitter 2015-08-26 07:06:56 PDT
Might be regression from Bug 1048035. I already asked for comment in that bug.
Comment 9 User image Sébastien Le Ray 2015-08-26 07:46:28 PDT
You're right, it's a regression from Bug 1048035, prependMailTo (http://hg.mozilla.org/comm-central/file/e6c5bce2b8fb/calendar/base/modules/calUtils.jsm#l296) is buggy:

(!aId.search(/^MAILTO:/i) && aId.includes("@")) ? "MAILTO:" + aId : aId;

String.prototype.search returns -1 not false when regexp doesn't match. Which is lucky since, String.prototype.includes does not exist… This should read

(aId.search(/^MAILTO:/i) == -1 && aId.indexOf("@") >= 0) ? "MAILTO:" + aId : aId;
Comment 10 User image [:MakeMyDay] 2015-08-27 14:25:55 PDT
Created attachment 8653738 [details] [diff] [review]
FixFBLookup-V1.diff

Patch for cc, aurora and beta fixing the wrong check for search() result.
Comment 11 User image [:MakeMyDay] 2015-08-27 14:31:26 PDT
Created attachment 8653741 [details] [diff] [review]
FixFBLookup-esr38-V1.diff

Fix for esr38, additionally using contains() instead of includes() (separate patch neccessary due to the switch from conatins to includes in version 40).
Comment 12 User image Philipp Kewisch [:Fallen] 2015-08-29 01:06:24 PDT
Comment on attachment 8653738 [details] [diff] [review]
FixFBLookup-V1.diff

Review of attachment 8653738 [details] [diff] [review]:
-----------------------------------------------------------------

::: calendar/base/modules/calUtils.jsm
@@ +293,5 @@
>       * @param  {string}        the string to prepend the prefix if not already there
>       * @return {string}        the string with prefix
>       */
>      prependMailTo: function(aId) {
> +        return (aId.search(/^MAILTO:/i) == -1 && aId.includes("@")) ? "MAILTO:" + aId : aId;

I have a bold suggestion, to be sure we should probably not use it on ESR:

return aId.replace(/^(?!MAILTO:)(.*)@/i, "MAILTO:$1@");

This is faster than the previous solutions on jsperf:
http://jsperf.com/regex-vs-lowercase-and-startswith

Hope I didn't miss anything :)
Comment 13 User image Philipp Kewisch [:Fallen] 2015-08-29 01:10:31 PDT
Comment on attachment 8653741 [details] [diff] [review]
FixFBLookup-esr38-V1.diff

Review of attachment 8653741 [details] [diff] [review]:
-----------------------------------------------------------------

r+ for the esr38 variant.
Comment 14 User image [:MakeMyDay] 2015-08-30 11:58:47 PDT
Created attachment 8654633 [details] [diff] [review]
FixFBLookup-esr38-V2.diff

Updated patch for esr38: changed prepended mailto: to lowercase. Additionally, a fix for calItipUtils was added, where the email property of the msgIAddressObject must be checked to determine the invitation sender.
Comment 15 User image [:MakeMyDay] 2015-08-30 12:11:07 PDT
Created attachment 8654636 [details] [diff] [review]
FixFBLookup-V2.diff

The same as for ers38, but with the above comment for using a regex considered, although the regex is slightly modified to catch all cases. Additionally, a test for removeMailTo and prependMailTo is added.

A try build for the test is currently running:

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=8fa1981cd14d
Comment 16 User image [:MakeMyDay] 2015-08-30 14:37:56 PDT
Comment on attachment 8654636 [details] [diff] [review]
FixFBLookup-V2.diff

The test failed. I will provide an updated patch tomorrow once it passed. Removing the review request until then.
Comment 17 User image Sébastien Le Ray 2015-08-30 22:48:15 PDT
Your remove test uses prepend instead of remove.
Moreover you could add a { input: "mailto:there.is.no.at", expected: "mailto:there.is.no.at" } (to cover stupid but handled case)
Comment 18 User image [:MakeMyDay] 2015-08-31 12:10:55 PDT
Created attachment 8655001 [details] [diff] [review]
FixFBLookup-V3.diff

Updated patch with fixed test. I haven't picked up Sebastiens suggestion to add a bogus test, because such case would for the functions standalone but not in a prepend after remove scenario and would thereby break the overall use case anyway (beside that using a mailto prefix for a non email address would not be standard complient iirc).
Comment 19 User image [:MakeMyDay] 2015-09-05 06:54:54 PDT
*** Bug 1201477 has been marked as a duplicate of this bug. ***
Comment 20 User image Philipp Kewisch [:Fallen] 2015-09-05 14:13:53 PDT
Comment on attachment 8655001 [details] [diff] [review]
FixFBLookup-V3.diff

Review of attachment 8655001 [details] [diff] [review]:
-----------------------------------------------------------------

::: calendar/test/unit/test_calutils.js
@@ +6,5 @@
> +
> +function run_test() {
> +    removeMailTo_test();
> +    prependMailTo_test();
> +    run_next_test();

run_next_test not needed since the test is synchronous

@@ +9,5 @@
> +    prependMailTo_test();
> +    run_next_test();
> +}
> +
> +// tests for calutil.jsm

calUtils.jsm, but you might as well remove the comment.

@@ +19,5 @@
> +                {input: "first.last.example.net", expected: "first.last.example.net"}];
> +    for (let test of data) {
> +        equal(cal.removeMailTo(test.input), test.expected);
> +    }
> +};

No need to add semicolons here

@@ +29,5 @@
> +                {input: "first.last.example.net", expected: "first.last.example.net"}];
> +    for (let test of data) {
> +        equal(cal.prependMailTo(test.input), test.expected);
> +    }
> +};

No need for semicolons
Comment 21 User image [:MakeMyDay] 2015-09-05 14:26:03 PDT
Created attachment 8657497 [details] [diff] [review]
FixFBLookup-V4.diff

Updated patch with comments considered.
Comment 22 User image aleth [:aleth] 2015-09-07 02:41:23 PDT
https://hg.mozilla.org/comm-central/rev/f48fd117644f24c1f2d91a157ce31d3ee64c00d5
Bug 1197320 - Free/Busy information of CalDAV calendar is no longer displayed after latest Lightning Calendar's update (v4.0.2). r=philipp
Comment 23 User image aleth [:aleth] 2015-09-07 02:50:03 PDT
Please set approval? flags on patches that need uplift.
Comment 24 User image Philipp Kewisch [:Fallen] 2015-09-07 04:59:56 PDT
adding checkin-needed for the backports
Comment 25 User image Philipp Kewisch [:Fallen] 2015-09-08 06:41:18 PDT
Backported to releases/comm-aurora changeset 32968d0d7746
Comment 26 User image Philipp Kewisch [:Fallen] 2015-09-08 06:45:39 PDT
Backported to releases/comm-beta changeset e7b0fc9b04af
Comment 27 User image Philipp Kewisch [:Fallen] 2015-09-08 12:12:16 PDT
Backported to releases/comm-esr38 changeset 2ac04a2aa9de
Comment 28 User image Stefan Sitter 2015-09-11 11:12:40 PDT
*** Bug 1203998 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.