1493008 - [meta] Integrate Calendar/Lightning into Thunderbird

Status: RESOLVED FIXED

(Thunderbird :: General, task)

Description

[Worcester12345]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Build ID: 20180917143811

Steps to reproduce:

Open Thunderbird


Actual results:

Calendar functionality is not where it should be. Addons status questionable. Keeping updated/synched with the Thunderbird application is a chore.

Google Provider is just another missing link, and hopefully that would be easier to deal with once separate Lightning is out of the picture.


Expected results:

Open the program, and have email and calendar both readily available with no aggravation.

Comment 1

[Wayne Mery (:wsmwk)]

Seems like this should be a duplicate. But in a quick spin I didn't find one

Comment 2

[Magnus Melin [:mkmelin]]

(In reply to Worcester12345 from comment #0)
> Calendar functionality is not where it should be. Addons status
> questionable. Keeping updated/synched with the Thunderbird application is a
> chore.

It's already included with all shipped builds and kept updated when thunderbird updates. There is no chore.

> Expected results:
> 
> Open the program, and have email and calendar both readily available with no
> aggravation.

That is already the case.

Re: "remove all Addon/Extension code", that's really not a lot of code. Mostly the meta to get it set up.

Comment 3

[Jorg K (CEST = GMT+2)]

(In reply to Magnus Melin [:mkmelin] from comment #2)
> It's already included with all shipped builds and kept updated when
> thunderbird updates. There is no chore.
Sorry, that's just not true, see: https://support.mozilla.org/en-US/kb/calendar-updates-issues-thunderbird

(In reply to Wayne Mery (:wsmwk) from comment #1)
> Seems like this should be a duplicate. But in a quick spin I didn't find one
I had bug 1449487 for this.

I think the bug makes good sense, but there are two problems:
1) Calendar/Lightning has traditionally been a separate project
2) Calendar/Lightning still uses XUL overlays which we've removed from TB.

Comment 4

[Magnus Melin [:mkmelin]]

(In reply to Jorg K (GMT+2) from comment #3)
> Sorry, that's just not true, see:
> https://support.mozilla.org/en-US/kb/calendar-updates-issues-thunderbird

Well, under normal circumstances it's not a problem. There can be bugs.

Comment 5

[Jorg K (CEST = GMT+2)]

Define "normal". I can wear a green shirt today and a red one tomorrow and when I return to the green one, I don't have to reattach the buttons and the pockets. In TB, I can't go back a version, TB xx beta yy doesn't update to TB xx beta (yy+1). That's normal?

Comment 6

[Magnus Melin [:mkmelin]]

Normal as in "don't fiddle around" ;)

While it's unfortunate add-ons have poor downgrade support - if it'd be up to me I'd pin them down version-to-version - downgrading to previous versions isn't a huge priority (they usually are EOL very soon anyway).

Comment 7

[Worcester12345]

(In reply to Jorg K (GMT+1) from comment #3)
> (In reply to Magnus Melin [:mkmelin] from comment #2)
> > It's already included with all shipped builds and kept updated when
> > thunderbird updates. There is no chore.
> Sorry, that's just not true, see:
> https://support.mozilla.org/en-US/kb/calendar-updates-issues-thunderbird
> 
> (In reply to Wayne Mery (:wsmwk) from comment #1)
> > Seems like this should be a duplicate. But in a quick spin I didn't find one
> I had bug 1449487 for this.
> 
> I think the bug makes good sense, but there are two problems:
> 1) Calendar/Lightning has traditionally been a separate project
> 2) Calendar/Lightning still uses XUL overlays which we've removed from TB.

OK, I just added:

Bug 1508119

Comment 8

[Worcester12345]

(In reply to Worcester12345 from comment #7)
> OK, I just added:
> 
> Bug 1508119

This now puts the wheels in motion to solving this bug. 

Thanks for helping clarify.

Comment 9

[Philipp Kewisch [:Fallen][📱📆]]

Worcester12345, I know you've been advocating for this since even before Lightning shipped with Thunderbird. I believe we've closed a few bugs like this one as WONTFIX because we've not positively decided to do this. However, I'm willing to entertain the possibility.

Going this road is a major step and before we start with very specific bugs like removing XUL overlays (which we can't really do unless we've already integrated Lightning) I think we need to start earlier. There are a number of process issues to resolve, and a few design decisions to be made.

One of the process issues for example, there is a high additional translation burden for Thunderbird localizers if Calendar is integrated. Lightning is not translated into all languages that Thunderbird is, so this needs someone to go through the locales and see who is missing, and get in touch with them to see if they'd be willing to pick up the extra work.

In addition, there are situations where Lightning contributes to performance issues by just being enabled in a way we have not yet been able to resolve. On the UI side we'd need to work on a complete proposal how Lightning would be integrated by default. What I've been dreaming of was to not show any calendaring features by default, but provide a few integration points that would allow you to set up calendars along with a guided tour that introduces you (e.g. when receiving an email with an invitation, or when creating new accounts.

These last two points together require that Lightning startup is changed so that background services like the alarm service are only spun up when calendaring has been added by the user. The first step would likely be resolving the UI issue, then optimizing performance by not running calendar code unless absolutely necessary.

This just off the top of my head, there are likely some other issues we'll find on the way. Given the work required on this it is also a roadmap question. I'm reluctant to add additional dependent bugs that would just be sitting there without action until this is on the roadmap for a specific Thunderbird release. We obviously need to focus on those changes with the most impact first.

Comment 10

[Worcester12345]

Would it be possible to take certain chunks of code and move those over sooner? In other words, if there is something which works well, and could be common to both calendar and email, take that and get it off Lightning and into Thunderbird itself. Then just reduce the Lightning addon by removing that and having them interface. I'm sure there is a LOT of overlap, and if that could be reviewed and begun first, it might make finding some of the outstanding bugs easier, and it will also lighten the load of additional moving. Have to think as modular as possible, even if it means breaking outside the box for a little while.

Comment 11

[Worcester12345]

Need to come up with different sections to do. Would make sense to break into smaller pieces and tackle one at a time. A "proof of concept", with an easy one would be a great first step. Is there a tool you can run to find similar code in each, then do a comparison? If not, take something like printing, or something which is broken anyhow and see if it is easier to fix once moved. There is much to be gained by this integration.

Comment 12

[Magnus Melin [:mkmelin]]

Worcester12345: I doubt there is any such duplicate code. Lightning can easily access any Thunderbird functionality so creating direct duplications would be pointless.

Comment 13

[Magnus Melin [:mkmelin]]

(In reply to Philipp Kewisch [:Fallen] [:📆] from comment #9)

Going this road is a major step and before we start with very specific bugs like removing XUL overlays (which we can't really do unless we've already integrated Lightning) I think we need to start earlier.

I would like to get rid of (even our hacky) xul overlays yes, so I think we should get started with this.

One of the process issues for example, there is a high additional translation burden for Thunderbird localizers if Calendar is integrated. Lightning is not translated into all languages that Thunderbird is, so this needs someone to go through the locales and see who is missing, and get in touch with them to see if they'd be willing to pick up the extra work.

At the moment that would be
-af (Afrikaans)
-ast (Asturian)
-be (Belarusian)
-cak (Cakchiquel)
-el (Greek)
-fa (Persian)
-hy-AM (Armenian)
-kk (Kazakh)
-pt-BR (there is a pt-PT though)
-si (Sinhala; Sinhalese)
-sr (Serbian)
-uz (Urdu)
-vi (Vietnamese)

I don't know if they are going to be able to translate lightning too, but all in all, it looks to be a pretty good situation. Worst case scenario, those locales will have some english string in them when accessing Thunderbird functionality. I don't think we can block this work on possibly missing translators though.

In addition, there are situations where Lightning contributes to performance issues by just being enabled in a way we have not yet been able to resolve. On the UI side we'd need to work on a complete proposal how Lightning would be integrated by default. What I've been dreaming of was to not show any calendaring features by default, but provide a few integration points that would allow you to set up calendars along with a guided tour that introduces you (e.g. when receiving an email with an invitation, or when creating new accounts.

Getting performance 100% to the same as now without lightning is probably not feasible, since it's still some more code that would have to run, but of course we can do work to improve the situation where no calendars are configured. For the UI, well, in an out-of-the-box experience, there is not that much I would hide. The menus need to be enabled so people can find it (though the menus are pretty hidden). Not sure if the standard, local "home" calendar is all that useful for people in general. And then the today pane probably shouldn't be shown before you have a calendar set up. Agreed we should offer to set up a calendar when needed.

These last two points together require that Lightning startup is changed so that background services like the alarm service are only spun up when calendaring has been added by the user.

We could key that on setting up a "real" calendar, instead of the local default one?

The first step would likely be resolving the UI issue, then optimizing performance by not running calendar code unless absolutely necessary.

This just off the top of my head, there are likely some other issues we'll find on the way. Given the work required on this it is also a roadmap question. I'm reluctant to add additional dependent bugs that would just be sitting there without action until this is on the roadmap for a specific Thunderbird release.

The sooner the better, but clearly in the cycle up to 76. That will also give localizers more time to pick up.

Putting this on Paul's plate.

Comment 14

[Rolf Gloor]

I think, that this localization issue is not an issue at all.
Those lightning translations are already missing today. So nobody is missing out on anything by the integration.
But chances are, things are getting better for them over time, because it is integrated in Thunderbird and those who are already doing those translations might step by step fill-in the missing parts.

Comment 15

[Rolf Gloor]

Thought about Lightning integration:
@Philipp Kewisch:
Would it make sense, while touching the base-code of Thunderbird for the Lightning integration, also to think about the new address book integration? (CardBook or alike. Honestly, I don't know how far that work is and what the actual status about that topic is).
But one could eventually do some preparation for this at the same time. (Thinking of things like CardDAV / CalDAV sync.)
I know it could mean some additional burden, but it might be worthwhile for the longer run.

Comment 16

[Worcester12345]

(In reply to Rolf Gloor from comment #14)

I think, that this localization issue is not an issue at all.
Those lightning translations are already missing today. So nobody is missing out on anything by the integration.
But chances are, things are getting better for them over time, because it is integrated in Thunderbird and those who are already doing those translations might step by step fill-in the missing parts.

Good point.

I am guessing of these 13, these might be the ones with the most users:
-be (Belarusian)
-el (Greek)
-vi (Vietnamese)

What I am wondering or looking for, is how to expedite this, to get rid of the entire addon part of Lightning, and get this integrated from the get-go, without compromising things. I am wondering if the developers would consider a separate branch, even temporarily, to do this. I would be more than happy to test it out (might want good instructions on backing up profiles, as MozBackup is not so up to date any more).

If there can be a calendar menu (NOT "Events and Tasks"), with some checkoffs to expose different parts of the interface, I think that would be a great start.
I like the ideas of long term thinking as well, such as CardDAV/CalDAV sync, etc.

Comment 17

[Magnus Melin [:mkmelin]]

Looking at https://l10n.mozilla.org/shipping/dashboard?filter=filter-result, here are the completion percentages for each of the locales not available for lightning.

-af (Afrikaans): 51%
-ast (Asturian): 75%
-be (Belarusian): 85%
-cak (Cakchiquel): 86%
-el (Greek): 95%
-fa (Persian): 22%
-hy-AM (Armenian): 81%
-kk (Kazakh): 92%
-pt-BR (there is a pt-PT though): 93%
-si (Sinhala; Sinhalese): 59%
-sr (Serbian): 89%
-uz (Urdu): 79%
-vi (Vietnamese): 86%

Comment 18

[Magnus Melin [:mkmelin]]

Posted to m.d.l10n. There are around 2458 strings in lightning.

Comment 19

[Worcester12345]

Are there any small steps which can be taken forward, to get this under way? Even if to ID which locales to do, and/or what order to do them in, or even if to decide to wait on that, and do something else. Baby steps towards the end goal.

Comment 20

[Paul Morris [:pmorris]]

(In reply to Magnus Melin [:mkmelin] from comment #13)

Going this road is a major step and before we start with very specific bugs like removing XUL overlays (which we can't really do unless we've already integrated Lightning) I think we need to start earlier.

I would like to get rid of (even our hacky) xul overlays yes, so I think we should get started with this.

One simple thing we could start with, while other pieces are being decided and moved into place, would be to begin removing XUL overlays that are internal to calendar. Philipp, what do you think? (There are 29 overlay files in calendar. I'm not sure how many are internal ones.)

Comment 21

[Magnus Melin [:mkmelin]]

After further investigation, it appears the list of missing locales was in-correct. Apparently the calendar all-locales/shipped-locales files are not really used for anything, and we should get rid of them. Checking released files, both from ATN and localized Thunderbird, I found they all have the same locales as Thunderbird. Some will have more en-US strings than others.

Comment 22

[Magnus Melin [:mkmelin]]

Numbers from Pontoon. Only be, ast, cak, fa and hy-AM are not in very good state.

-af (Afrikaans): https://pontoon.mozilla.org/af/thunderbird/ 65%
-ast (Asturian): https://pontoon.mozilla.org/ast/lightning/ missing
-be (Belarusian): https://pontoon.mozilla.org/be/lightning/ 1%
-cak (Cakchiquel): https://pontoon.mozilla.org/cak/lightning/ 10%
-el (Greek): https://pontoon.mozilla.org/el/lightning/ 100%
-fa (Persian): https://pontoon.mozilla.org/fa/lightning/ 5%
-hy-AM (Armenian): https://pontoon.mozilla.org/hy-AM/lightning/ 1%
-kk (Kazakh): https://pontoon.mozilla.org/kk/lightning/ 100%
-pt-BR (there is a pt-PT though): https://pontoon.mozilla.org/pt-BR/lightning/ 100%
-si (Sinhala; Sinhalese): https://pontoon.mozilla.org/si/lightning/ 63%
-sr (Serbian): https://pontoon.mozilla.org/sr/lightning/ 99%
-uz (Urdu): https://pontoon.mozilla.org/uz/lightning/ 70%
-vi (Vietnamese): https://pontoon.mozilla.org/vi/lightning/ 100%

Comment 23

[Jorg K (CEST = GMT+2)]

Funny that you mention hy-AM. Check bug 1582936 ;-)

Comment 24

[Philipp Kewisch [:Fallen][📱📆]]

Hey Folks, it seems we are conflating a lot of issues, I've just skimmed and read stuff about overlays and l10n. Can we maybe move discussions into respective dependent bugs?

Comment 25

[Worcester12345]

(In reply to Philipp Kewisch [:Fallen] [:📆] from comment #24)

Hey Folks, it seems we are conflating a lot of issues, I've just skimmed and read stuff about overlays and l10n. Can we maybe move discussions into respective dependent bugs?

Agreed. See:
(In reply to Worcester12345 from comment #19)

Are there any small steps which can be taken forward, to get this under way? Even if to ID which locales to do, and/or what order to do them in, or even if to decide to wait on that, and do something else. Baby steps towards the end goal.

Comment 26

[Worcester12345]

Each of these small steps might be a new bug, so please start that bug, if it is your area of expertise, and refer it back to here.

Are there any actual small steps to take on moving THIS bug forward at this time?

Comment 27

[Magnus Melin [:mkmelin]]

Making it a system add-on (https://firefox-source-docs.mozilla.org/toolkit/mozapps/extensions/addon-manager/SystemAddons.html) is one option Philipp and me have discussed.
Reading up on it a bit, I'm not sure how feasible that is: must be restartless and multi-process compatible. When Overlays.jsm goes away there isn't really a good way to inject UI as much as Lightning currently does. The current Firefox system add-ons https://searchfox.org/mozilla-central/source/browser/extensions have UI that's very concentrated in scope.

Fluent-transition would also be a bit of a concern. System add-ons just recently got the ability to use Fluent, so I guess that could work out. It does sound like it would hover create extra work in catering for this abnormal case wrt. file locations etc. (There's a risk we'd be the guinea pig for that too.)

Comment 28

[Worcester12345]

Nope. Would rather just see full integration, maybe with its own dedicated calendar/schedule menu.

Remove everything that does not pertain to that. K.I.S.S.

The goal is to strip it back to the most basic functions of the combined Thunderbird application and Lightning extension, only without the modularity.

Comment 29

[Worcester12345]

What are the actual steps to make it so when I go to Tools:Add-Ons, I will no longer see Lightning installed as an extension/addon, but still have the calendar function working? I don't care about the XUL, XML, CSS stuff, but the actual move to change it over? The other removal can happen after.

Again, what are those steps, and how can they be accomplished in the shortest time possible?

Comment 30

[Jorg K (CEST = GMT+2)]

We're tired of comments like comment #29.