713843 - support for IMAP notes - 'Notes' in IMAP Folder

Status: NEW

(Thunderbird :: Folder and Message Lists, enhancement)

Description

[Matthew Morley]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Build ID: 20111220165912

Steps to reproduce:

NB:  Not sure if this is a bug or would be classed as a feature request... Please do let me know if it's the latter!

I set up a new account in Thunderbird for Apples iCloud service with a view to using TB to edit/view notes from my iPhone, iPad etc. I hate Apple's Mail app so wanted to use Thunderbird.


Actual results:

The account created with no issues and the 'Notes' folder shows up. Clicking on the folder also shows all the notes  but they are treated as emails - i.e. I cannot edit them - only send them etc.

Is this the expected, normal behaviour? Have Apple simply 'fudged' their idea of IMAP by creating a specific folder that the contents can be edited or is the 'notes' a feature of IMAP specification?


Expected results:

I had expected to be able to edit the 'notes' in the notes folder instead of just being read-only. Possibly also expected the 'Notes' folder to havea  special icon to show it's special status.

Comment 1

[Matthew Morley]

Sorry, I forgot to add that this is 9.0.1

Comment 2

[Matthew Morley]

Hmm, it looks like it may need a specific add-on creating? Unless there are a lot of iCloud users to warrant it being added as a main feature!

I dragged/dropped an email from another folder into the 'Notes' folder but it did not show up in my iPhone notes.

I then viewed the source of an existing note and noticed the headers:


[quote]Subject: First line of note here
From: Matthew Morley <*****@me.com>
X-Universally-Unique-Identifier: 46C48CB1-A2D7-4AAE-86E8-04BF581AE438
Content-Type: text/html;
	charset=utf-8
X-Uniform-Type-Identifier: com.apple.mail-note
Message-Id: <[e-mail]>
Date: Thu, 22 Dec 2011 22:11:18 +0000
X-Mail-Created-Date: Thu, 22 Dec 2011 21:30:58 +0000
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (1.0)

First line of note here<div>Second line of note here</div><div>Third line of note here</div>
[/quote]

I would assume that the key part is 'X-Uniform-Type-Identifier: com.apple.mail-note' so any add-on or new feature would need to create messages in that format for it to be used by iOS devices - presumably other 'notes' systems will use their own custom headers.

Would I be better off pursuing this elsewhere? I have changed it to an enhancement request froma bug as I don't think it's a bug now.

Comment 3

[David :Bienvenu]

None of this is standard imap behavior - we could treat the folder as a drafts folder so that you could edit and save messages inline, but the uniform-type-identifier is an Apple thing.

Comment 4

[Matthew Morley]

OK, as soon as I saw the 'custom' header I had a feeling it would be so!

Is that a header that could be created in a new message via an add-on for Thunderbird or doesn't the API allow that level of access?

I'd be interested in learning and having a go myself with a plugin if it is technically possible. If there's no way to add a custom header like that in the first place then there's no point me even looking into learning!

Also, is there an easy way to make a folder acting like a 'Drafts' folder with the editing and saving back to the folder or is that something that is done in the actual TB programming itself?

Comment 5

[Matthew Morley]

Right, done a bit more testing.

It looks like the key factors are that the headers MUST contain 'X-Uniform-Type-Identifier: com.apple.mail-note' AND MUST NOT have a 'To' header.

I shall look into creating some kind of plugin, unless you guys think it is worth adding iCloud Notes support to the feature list? This would work on both Windows and Mac as, unlike me.com, iCloud is cross-OS.

Comment 6

[WADA:World Anti-bad-Duping Agency]

Same request as bug 700120?

Comment 7

[Matthew Morley]

(In reply to WADA from comment #6)
> Same request as bug 700120?

Yes, it does seem similar although 700120 refers to gmail notes - without knowing gmail notes ins and outs it would be difficult to guess just how they do it. Apple's iCloud notes requires no 'to' address and the Apple-specific header.

The workaround of defining the 'Notes' folder as a Draft would work in terms of getting the message itself into that folder on Apples server, BUT it would not be able to add 'X-Uniform-Type-Identifier: com.apple.mail-note' to the message.

As such I think it is a different request. 700120 is requesting a 'Notes' folder with special properties akin to the Draft folder (which is achievable with a little workaround [although not very user friendly]) however this request, I suppose, has become more of a 'how to get the apple-specific header into the message. The workaround in 700120 would be useful for editing existing notes on the server but not for creating new ones.

Comment 8

[Matthew Morley]

Setting the notes folder as a Draft folder and editing a message removes the custom headers so the 'note' no longer appears in iOS devices...

However - I found this: http://email.about.com/od/mozillathunderbirdtips/qt/Add_an_Arbitrary_Custom_Header_to_Mozilla_Thunderbird.htm

So I have now added 'X-Uniform-Type-Identifier' to the list of custom headers that appear in the 'to' drop-down area and I just manually enter 'com.apple.mail-note'

The message also needs to be composed in plain text.

It's a little fiddly but it's useable at least.

Thanks for all your input!

Is it worth marking this as 'Resolved' or would it be worth leaving open for if it could be added as a feature?

Comment 9

[Matthew Morley]

It's semi-working at the moment but there's an issue with editing existing messages. Apple in their wisdom have the content-type as text/html which makes the TB editor use HTML mode. Even switching to plaintext using Options > Format > Plain Text Only helps but still includes html elements. So when saved (after adding the apple-specific header) and viewed/edited on an ios device the 'title' becomes indented from the <html><head>... etc elements in the message body.

Could this be a bug with the message compose pane? Should changing the format to plain text still keep html elements?

I've been unable to find a message source editor that allows me to edit the headers and rmeove the <html> elements. I found a html editor BUT it still keeps the main <html> and </html> which is no use.

At the moment the 'hack' is useable as I am aware of the short comings and tricks to get it working, so for me it is fine. But there seem to be a few things if it is decided to add it as a general feature or if I find time to learn to create a plugin!

Comment 10

[WADA:World Anti-bad-Duping Agency]

FYI.

Another way for manual creation of mail data for Apple's iCloud Notes.
(1) Create .eml file which includes next line and required message headers by text editor.
X-Uniform-Type-Identifier: com.apple.mail-note
Note:
Line ending should be [CRLF], because of .eml file which is mail data stream.
(2) Import the .eml file as mail to Tb's Notes folder by Drag&Drop.
Drag&Drop the .eml file to thread pane of the Notes folder of Tb, or Drag&Drop the .eml file to Notes folder at folder pane of Tb.

".eml file creation" can easily be achieved by simple Script or Command Shell, if message format of Apple's iCloud Notes is clear.
If import of many .eml files is required, extesion like ImportExportTools can be used.

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to Matthew Morley from comment #9)
> the 'title' becomes indented from the <html><head>... etc elements in the message body.

"Automatic formatting of html source in mail, such as line wrap at hard coded 72 chars, indention by spaces merely for convenience in HTML source viewing, is already known behavior of Tb, and problems caused by it are known; bug 653342 and bug 650206.

> I've been unable to find a message source editor that allows me to edit the headers and remove the <html> elements.

"Simple text editor" is always one of best solutions in such case :-)
As short term circumvention, I believe "simple script which generates required .eml file" is a best solution for Apple's iCloud Notes folder and mail data in it. Once existent mail data is saved as .eml file, simple script can very easily alter its content, because .eml file is equivalent to .txt file except new line char on Linux/Mac OS X. So, it's applicable to modification of existent data too.

Comment 12

[WADA:World Anti-bad-Duping Agency]

FYI.
"Simple Script" can be a JavaScript code which runs under browser such as Firefox.
(1) Create simple HTML by any HTML editor or text editor.
(2) Copy&Paste HTML source to <textarea> of HTML under which JavaScript runs.
(3) Run JavaScript code at the HTML, and convert pasted text to required mail source for Apple's iCloud Notes, and Copy&Paste to text editor, then save as .eml.
(4) Import the .eml to Tb's mail folder by Drag&Drop.

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to Matthew Morley from comment #9)
> Should changing the format to plain text still keep html elements?

String of <html_tag> is kept in text mode editing of Tb, but, if text/html mail is edited in text mode of Tb, html part is converted to text then html tag is lost. In order to keep <html_tag> as-is, Content-Type: text/html needs to be faked as if text/plain, if use of Tb's composer is mandatory.

Comment 14

[WADA:World Anti-bad-Duping Agency]

(In reply to Matthew Morley from comment #0)
> I set up a new account in Thunderbird for Apples iCloud service with a view
> to using TB to edit/view notes from my iPhone, iPad etc.

Gmail IMAP? Yahoo! IMAP? Or Apple's server for Apple's iCloud service? Or other IMAP server?
> http://lifehacker.com/5569758/seamlessly-sync-iphone-notes-to-a-gmail-account

If Gmail IMAP, many additional concerns arise due to particularity of Gmail/Gmail IMAP, for example,
 If update of existent note in Gmail IMAP "Notes" folder using IMAP mail client,
 old version should be moved or copied to [Gmail]/Trash folder by the client.
 Unless moved/copied to [Gmail]/Trash folder, old version of the note remains in
 "[Gmail]/All Mail" via Gmail IMAP(==Gmail folder of "All Mail") forever.

Comment 15

[WADA:World Anti-bad-Duping Agency]

FYI.
Tricky, but very simple and easy way for manual creation of new note.
(1) Create a local mail folder(call NotesX) under Local Folders as work folder.
(2) Copy a note in "Notes" folder to NotesX as template, Compact NotesX.
(3) Edit .../Mail/Local Folders/NotesX by text editor.
(4) Modify required message headers.
    Message-ID:
    X-Universally-Unique-Identifier:
      Adding timestamp+something to Message-ID:/X-Universally-Unique-Identifier:
      is perhaps sufficient to keep uniequness of these Identifier.
    Subject:
    some other headers like From:
(5) Modify message body text as required
(6) Save
(7) At NotesX, Repair Folder.
(8) Move the mail to "Notes" folder of IMAP account for iPhone Notes.
(9) Compact NotesX.

Comment 16

[WADA:World Anti-bad-Duping Agency]

I think generic enhancement/add-on like next is conveient.
- New mode for editing of existent mail data :
    comibination of "Edit As New" + "Edit Draft" at any mail folder.
  Calling "Edit As Note" is better?
(1) Direct message header manipulation.
(1-1) Modify existent message headers.
(1-2) Keep existent headers as-is, unless modification is requested.
(1-3) Add/Remove of any header, except Content-Type: multipart like one.
      (mail structure change shouldn't be permitted in this feature)
(2) Modify message body text. 
(3) Generate Unieque Identifier for header like
      X-Universally-Unique-Identifier:.
(4) Save
(4-1) Save to any mail folder
(4-2) Save As New(add/append), or Replace(Save As New + Delete of old version)
(5) Because purpose is "Editing", "Send", "Send Later" should be disabled.

Above is diferent from existent requests but very similar functionality to request of bug 203913 and bug 254739(see dependency tree for bug 91106 which is for change of Subject: only).
These bugs are for "alteration of existent mail data", so these are considered "not mandatory", and, as written in bug 254739 comment #68, there already is solution by external application based on Thunderbird, although it's not charge free.

If generic feature for "modification of existent mail data" will be implemented, additional ehancements for iPhone Notes and/or Gmail Notes can be achieved by not-so-heavy add-on.

Comment 17

[WADA:World Anti-bad-Duping Agency]

(In reply to Matthew Morley from comment #5)
> AND MUST NOT have a 'To' header.

Sure?

> http://forums.macrumors.com/archive/index.php//t-948942.html
Mandatory condition looks for me next only.
(1) Mail is placed in "Notes" folder of IMAP.
    (Gmail Label of "Notes" is aded to mail if Gmail)
(2) Existence of X-Uniform-Type-Identifier: com.apple.mail-note
(3) Existence of X-Universally-Unique-Identifier: <hex-string>
        hex bytes grouped like this: 43433028-0184-4885-8FDC-15CF8CEE0629
    with same <hex-string> as one in old version of note,
    if replacement of existent old version of the note by iPhone is needed.

> http://www.shivkaushal.co.uk/blog/2011/01/create-an-iphone-note-from-your-mac-and-push-it-to-gmail/
And, X-Mail-Created-Date: is better added.
     X-Mail-Created-Date: Mon, 10 Jan 2011 23:40:15 +0000

> http://www.perlmonks.org/?node_id=851322
An example of Perl Script to upload mail data for iPhone Notes to IMAP folder of "Notes".

Comment 19

[Matthew Morley]

(In reply to WADA from comment #14)
> (In reply to Matthew Morley from comment #0)
> > I set up a new account in Thunderbird for Apples iCloud service with a view
> > to using TB to edit/view notes from my iPhone, iPad etc.
> 
> Gmail IMAP? Yahoo! IMAP? Or Apple's server for Apple's iCloud service? Or
> other IMAP server?

Using Apple's me.com/iCloud IMAP servers.

Server: p01-imap.mail.me.com  (can also use p02 or p03)
Username: [username without @me.com]
Port 993
SSL/TLS

And to edit the 'notes' set the Drafts settings to use the Notes folder.

Comment 20

[Matthew Morley]

(In reply to WADA from comment #17)
> (In reply to Matthew Morley from comment #5)
> > AND MUST NOT have a 'To' header.
> 
> Sure?
> 
> > http://forums.macrumors.com/archive/index.php//t-948942.html
> Mandatory condition looks for me next only.
> (1) Mail is placed in "Notes" folder of IMAP.
>     (Gmail Label of "Notes" is aded to mail if Gmail)
> (2) Existence of X-Uniform-Type-Identifier: com.apple.mail-note
> (3) Existence of X-Universally-Unique-Identifier: <hex-string>
>         hex bytes grouped like this: 43433028-0184-4885-8FDC-15CF8CEE0629
>     with same <hex-string> as one in old version of note,
>     if replacement of existent old version of the note by iPhone is needed.

Yes, you are correct. When I added a 'to' address to test I had not realised it lost the com.apple.mail-note header. When that header was added along with a to address it worked fine.

Comment 21

[Matthew Morley]

(In reply to WADA from comment #16)
> I think generic enhancement/add-on like next is conveient.
> - New mode for editing of existent mail data :
>     comibination of "Edit As New" + "Edit Draft" at any mail folder.
>   Calling "Edit As Note" is better?
> 
>   [SNIP]
>
> If generic feature for "modification of existent mail data" will be
> implemented, additional ehancements for iPhone Notes and/or Gmail Notes can
> be achieved by not-so-heavy add-on.

I do like the sound of a feature/add-on of "Edit as Note" as suggested above but obviously the work involved needs to be weighed up alongside how many people would use it!

I use Notes a lot on my iPhone and iPad but I cannot stand Apple's Mail program (hence I've used Thunderbird on my Mac for the 3 years I've had it). At the moment I occasionally use Mac Mail to edit notes as it 'just works' - however, I'd rather use Thunderbird for obvious reasons. At the end of the day, I can always pull out my phone and edit the notes :)

Comment 22

[Lee_Dailey]

howdy y'all,

the following extension may help you with making sure you have exactly the headers you want. 
https://nic-nac-project.org/~kaosmos/headertoolslite-en.html

the addition of a notes entity in tbird would be right handy. far more so [for me] than rss has ever been. [*grin*] 

take care,
lee

Comment 23

[Matthew Morley]

(In reply to Lee_Dailey from comment #22)
> https://nic-nac-project.org/~kaosmos/headertoolslite-en.html

Thanks Lee - the extension works great. Thing is I tried it the other day and it kept giving errors but now it's working fine... Maybe it was clashing with another plugin I was trying.

So, I'm almost workable now even i it never becomes an official feature!

MaFt

Comment 24

[volker]

Hi there,

I had some time to create this extension over the holidays:
https://addons.mozilla.org/en-US/thunderbird/addon/ios-imap-notes/?src=search

Please let me know if it works for you. I can only test Windows but think it might also run under Linux and Mac OS.

Greetings,v.

Comment 25

[Matthew Morley]

(In reply to volker from comment #24)
> Please let me know if it works for you. I can only test Windows but think it
> might also run under Linux and Mac OS.

Says it's not available for my platform (Mac). So I assume you would either have to send me the file directly or adjust the settings somewhere for it to be downloadable on Mac?

MaFt

Comment 26

[volker]

Hi MaFt,

I changed the Platforms so you should be able to download it from the link above...

v.

Comment 27

[Matthew Morley]

(In reply to volker from comment #26)
> Hi MaFt,
> 
> I changed the Platforms so you should be able to download it from the link
> above...
> 
> v.

Yes - that seems to work. It could do with a bit of work with regards to user-friendliness in how to set up the account etc in the first place. Perhaps this is not the right place to discuss though - perhaps we could email directly so as not to clog up BugZilla?

Comment 28

[Wayne Mery (:wsmwk)]

(In reply to volker from comment #24)
> Hi there,
> 
> I had some time to create this extension over the holidays:
> https://addons.mozilla.org/en-US/thunderbird/addon/ios-imap-notes/?src=search
> 
> Please let me know if it works for you. I can only test Windows but think it
> might also run under Linux and Mac OS.
> 
> Greetings,v.

works great for me.  Thanks for the addon

Comment 29

[Nicholas Fish]

Any plans for an update to make this add-on work with Thunderbird 45.x?

Comment 30

[volker]

Hi, I use it with Thunderbird 45.x (win) on a daily basis. 
Are you experiencing problems?
v-

Comment 31

[Nicholas Fish]

OK, I can see it works fine if installed from the XPI on 45.4.0 on Linux.

Is it possible to update https://addons.mozilla.org/en-US/thunderbird/addon/ios-imap-notes/ to allow installation directly from Thunderbird's Addons browser for newer versions of Thunderbird? At present, this is only possible with Thunderbird vrsions 3.0 - 31.x.

Comment 32

[Mikhail T.]

Even if an add-on may provide complete functionality, it should still be possible to edit notes originally created on an Apple device with Thunderbird.

Currently, I can view such notes -- which is great -- but an attempt to edit (right-click, "Edit as New Message") such notes in Thunderbird moves the note to "Drafts" and strips the header(s) necessary for the Notes-app on the phone to recognize it as such.

So, even if I move the edited note from Drafts to Notes after I'm done with the editing, the new variant will not be recognized by the device...

This silent removal of the existing headers of an edited message is gratuitous and does not need to happen...

(In reply to Matthew Morley from comment #2)
> Unless there are a lot of iCloud users [...]

I use my own Cyrus IMAP-server. Works great. Wish TB was more helpful, though...

Comment 33

[Richard Leger]

What's the current plan/progress about support of IMAP notes sync for read and write and inter-operability so far?

Comment 34

[Richard Leger]

Any update?