217431 - inbox shows only post privilege with fastmail.fm account

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Mike Auty]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5b) Gecko/20030813 Mozilla Firebird/0.6.1+
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5b) Gecko/20030826

Fastmail.fm has recently required a change of mail server, and the new mail
server seems to break when used with mozilla mail.   When subscribing to the
account, the inbox says its privileges are Post only, and intermittently it also
states that the folder is shared.

Reproducible: Always

Steps to Reproduce:
1. Setup account with fastmail.fm
2. Add account to mozilla (no special settings)
3. Open inbox and check privileges.

Actual Results:  
Inbox was said to be shared, and had Post privileges.

Expected Results:  
Inbox should have been "not shared" and had Full Control privileges.

Additional side effects seem to be that messages cannot be deleted from the
inbox (probably related to Bug 216439) and also when starting mozilla the inbox
is not checked for new mail.

Other users of the service have also reported problems and it appears to be a
fastmail/mozilla combination breakage only.  See
http://www.emailaddresses.com/forum/showthread.php?s=&threadid=15194 for further
information.

Comment 1

[Mike Auty]

Attachment: Trace of imap login and check

The trace contains the first connection to the server and checking of inbox and
folders, then checking of drafts as an example of a standard folder that does
work properly.	Finally thunderbird was restarted and another folder checked. 
Note that after restart the inbox is *not* checked.

Comment 2

[Mike Auty]

Found another fastmail.fm report from long ago (Bug 170597).  Bug says fixed,
but the symptoms are remarkably similar.  The only difference is that its now
the inbox rather than some other folder, and worse, this seems to stop the inbox
being checked on startup or even using get messages.  

Comment 3

[David :Bienvenu]

what's your username for the account that the log is posted with? I'm assuming
it's not "ikelosorg". It looks to me like fastmail is not returning an acl for
your username, and returning "p" as the only privilege that everyone has
(everyone in this case, including you). Unfortunately, Mozilla pays attention to
the ACL that the server returns :-(

Comment 4

[Mike Auty]

Recently all users were requested to move over to <username>@<domain> since
fastmail serves many different domains.  The account I am now using to log in is
[e-mail].

It probably is the case that Mozilla is using the correct ACL, and I've
submitted a bug report to fastmail itself, but that still leaves the problem of
previous builds (since I was using thunderbird I can only say 2003-08-13 and
before) checked the inbox folder, whereas newer builds do not.

I'm also confused as to why my subfolders say they are not shared and tell me I
have full control privileges?

Comment 5

[David :Bienvenu]

ah, so your username is "[e-mail]", and the host name is
mail.messagingengine.com?  The server is not returning acl for [e-mail]
but just for ikelosorg, so we don't match the usernames in the acl response and
you get the "anyone" privileges.

I don't think anything changed in this area in Thunderbird/Mozilla for quite
some time but I'll check. Are you saying that going back to an 08/13 thunderbird
build works with the same profile, and a newer build does not?

Comment 6

[Mike Auty]

No, only in recent versions (that last two versions of thunderbird, well as of
today 3) the inbox was no longer checked at start up (in fact no imap connection
was made until one of the other folders was selected).  All versions of
thunderbird that I've tried have said that I have post privileges only (with the
new username) and that the folder is shared.  The subfolder always show up as
not shared and as having full privileges (which is odd, because from what I
could see they also had ACLs for ikelosorg rather than [e-mail]).

Comment 7

[David :Bienvenu]

Do you have your imap server directory set to "INBOX."?  That seems to have the
(in this case) fortunate side effect of breaking Mozilla's acl code, so the acl
is ignored. I've recreated these acl problems by changing my user name to
[e-mail]. I can't delete messages in any of my imap folders (INBOX or
sub-folders) once I do that, and I only have post privileges.  Setting my imap
server directory to "INBOX." makes everything work.

So I'm not seeing exactly what you're seeing, but what I'm seeing makes more
sense :-) I don't know about the inbox getting checked at startup (or at all)
but I'll look into it with both settings.

Comment 8

[Mike Auty]

That was how I'd always had it set, then the last two builds it stopped working
there.  There's been a report that it works ok with 0.2 RC which I tested
earlier but admittedly not with the "INBOX." (which I thought was just confusing
matters).  

I've just tested it again and here's what happened:
Delete profile, put in thunderbird 0.2 RC (as released earlier today)
Added profile, it checked and found the inbox with subfolders, it said the inbox
had post only and was shared.  I changed the settings to include INBOX. and
unchecked show subscribed folders only, collapsed the account, and re-expanded
it on the tree view, to get the folders in the right place.  Checked the inbox,
still the same.  Exited, restarted, inbox properties says "folder does not
support sharing", opened a different folder (made connection), went back to
inbox properties, now back to shared with post.  Clicking check mail does *not*
get new mail in the inbox (third party checker shows I have mail).  Very strange.

Then I go back to the subscribe box, which now lists INBOX and says it isn't
subscribe.  Subscribe to it, then mail checking works again.  Still post only
(so still no deleting).  Restart and again no mail checking and inbox is no
logner subscribed (unchecked in subscribe box).  Subscribe again, then go *back*
to the subscribe box to check, and inbox isn't even listed.  Something is *very*
screwy here.  

As as side note, I found out the two builds that didn't work were the 13th and
the 20th, and it was the 7th that checked the inbox ok.

Something *somewhere* is definitely going wrong, but I haven't a clue what.  I'm
willing to test out any suggestions anyone can make and provide as much feedback
as I can...

Oh, and since I forgot to say earlier, thanks for all your help so far David,
it's really appreciated!  5:)

Comment 9

[Mike Auty]

One of the developers at fastmail suggests that the problem arises from how
mozilla handles ACLs.  He the following from RFC 2086:

   It is possible for multiple identifiers in an access control list to
   apply to a given user (or other authentication identity).  For
   example, an ACL may include rights to be granted to the identifier
   matching the user, one or more implementation-defined identifiers
   matching groups which include the user, and/or the identifier
   "anyone".  How these rights are combined to determine the user's
   access is implementation-defined.  An implementation may choose, for
   example, to use the union of the rights granted to the applicable
   identifiers.  An implementation may instead choose, for example, to
   only use those rights granted to the most specific identifier present
   in the ACL. A client may determine the set of rights granted to the
   logged-in user for a given mailbox by using the MYRIGHTS command.

This leaves me slightly wondering what the point of listing the ACLs is, if you
can have rights not listed there.  Anyway, I hope this helps in someway...

Comment 10

[David :Bienvenu]

Mike, thx. I'm pretty sure that's not what's going on, but if you have access to
the developer, can you tell him this:

the user name returned in the acl list by fastmail is not the same as the user
login, so we end up using the anyone rights. For example, if the user name used
to logon is "user@foo", the acl user is returned as simply "user". That's the
crux of the problem.

Comment 11

[Mike Auty]

I think he knows this, but I'll pass it on.  Just to clarify this seems almost
certainly to be nobodies fault, but I think a lot of users would like to get the
incompatibilities sorted out, so collaboration sounds like a great idea.  My
contact with him is through the forum, so if you'd like to cut me out as a
middle man that would be the place to go.  I can also suggest to him that he
possibly starts posting to this bug report to try and help solve the problem
too.  I'll post this there and post back a reply until I know you two are
talking more directly.  Thanks for your patience!

Comment 12

[Konrad Paumann]

This is the answer from Jeremy Howard (Developer at fastmail.fm):
"    quote:
    "the user name returned in the acl list by fastmail is not the same as the user
    login, so we end up using the anyone rights. For example, if the user name used
    to logon is 'user@foo', the acl user is returned as simply "user". That's the
    crux of the problem."

Yes, that is the source of the problem. IMAP provides the MYRIGHTS command to
avoid any ambiguity in determining the logged in user's rights, and is the only
way to do this correctly - note that the RFC says that combinations of rights
are application defined. So Moz really needs to fix this.

There's not much we can do at our end, unfortunately - renaming mailboxes to
include the domain part is going to take a long time, and I can't see any way to
avoid this problem otherwise.

I'll have a think about this... in the meantime hopefully Moz will start using
MYRIGHTS as the RFC requires, which will fix the problem."

Comment 13

[Mike Auty]

I've suggested to Jeremy a possible solution which might help Mozilla users
quickly, we'll have to wait to see what he says, but in case this discrepancy
arises again how likely/difficult would it be to implement the MYRIGHTS check in
Mozilla, and how would the logic work if Mozilla carried out both checks?  It
looks as though there will need to be some work on the ACLs since adding
"INBOX." seemingly breaks Mozilla's use of them enough to get fastmail fairly
operational.  Could the MYRIGHTS check be added if/when work is being done to
correct this?  That way both sides could be working towards a solution, and we
might even set a standard for how to deal with a poorly worded RFC.

Comment 14

[David :Bienvenu]

I think it's fairly easy to add a call to get myrights for personal folders (we
only get myrights for shared,public folders now). I'm not sure why we get acl
for folders anyway since I think we only ever display the rights for the logged
in user...

Comment 15

[Mike Auty]

That's great news!  Is there anything I can do to help, or do we just leave you
to it?  I'm more than willing to bug test any binaries you may make or help in
any other way I can...

Comment 16

[David :Bienvenu]

Attachment: proposed fix

this patch makes us issue a myrights for personal folders, like we do for
public+shared folders. Myrights will override any rights returned by getacl. I
also had to make sure we use the real user name for the acl (the real user name
is different if the user edits their user name for the server) because the
fastmail users that will have this problem will have edited their user name.
This patch also contains some unrelated work to add the ability to remember the
supportedUserFlags for an imap folder, and isn't currently called.  

I am a little reluctant to check this in for 1.5 final because I've only got
access to one imap server that supports ACL, and that's the fastmail server. I
can be convinced, however.

Comment 17

[(not reading, please use [e-mail] instead)]

Comment on attachment 130546 [details] [diff] [review]
proposed fix

r/a=sspitzer for 1.5

Comment 18

[David :Bienvenu]

fix checked in. Should be in tomorrow's build.

Comment 19

[Mike Auty]

Wow,
  That's amazing turn around time, two days from start to finish!  Thanks david,
I know all the fastmail/mozilla users really appreciate the work you've put into
this thing!  This is a prime example of how the people behind mozilla make it
such a good product...

Comment 20

[Scott MacGregor]

FYI: this is in the thunderbird 0.2 branch

Comment 21

[Scott MacGregor]

*** Bug 217788 has been marked as a duplicate of this bug. ***

Comment 22

[Peter Lairo]

The product for this bug is MailNews (not Thunderbird), however, it seems the
fix only landed in Thunderbird, NOT in Mozilla (win, 20030830). Could someone
either submit a fix for MailNews, or reopen this bug? Thanks!

Comment 23

[David :Bienvenu]

No, it landed on the mozilla trunk, so it should be in both Thunderbird and 1.5b
nightly builds.