90064 - sending mail without opening mailnews and having sent mail folder on remote server causes SUBSCRIBE error

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Christopher Blizzard (:blizzard)]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2) Gecko/20010709
BuildID:    20010709

If you try to send mail without actually opening mailnews first the browser will
try to SUBSCRIBE to the Sent folder on the remote IMAP server.

Reproducible: Always
Steps to Reproduce:
1. Open the browser
2. ctrl-M to compose a new mail message
3. compose message
4. send mail

Actual Results:  You will get an error message that says that you are already
subscribed to the Sent mail folder.  The "sending mail..." dialog and the
compose window hang forever, spinning.

Expected Results:  Mail will be saved on the remote server.

Remote end is an IMAP server.

Comment 1

[Karen Huang]

I cannot reproduce this problem on Linux 07-09-04-0.9.2 & Wondows 07-09-5-0.9.2 
builds...... 

Comment 2

[Karen Huang]

I cannot reproduce this problem on today's Linux 07-10-08-0.9.2 build either.
Christopher, what kind of the mail are you trying to send?

Comment 3

[Christopher Blizzard (:blizzard)]

Just standard plain text mail.

Comment 4

[Karen Huang]

What's the size of this plain text mail?

Comment 5

[Christopher Blizzard (:blizzard)]

The length is just one line.  Doesn't seem to matter.  My instructions probably
aren't that clear here, either.  When I say "Open the browser" I mean, start the
browser from scratch and don't open mail.  Try to send mail before you log into
your mail account in mail.

Here's the actual error message:

"The current command did not succeed.  The mail server responded: SUBSCRIBE
failed: Already subscribed to mailbox Sent."

Comment 6

[Christopher Blizzard (:blizzard)]

This bug is still here and it's kicking my butt.  FYI, My Sent mail folder is on
a  remote IMAP/SSL server.  Also, I do not have biff turned on.

1. mozilla -compose
2. write a mail message
3. send the message

At this point I get asked for my security password ( I save the imap password
using the security manager ) and it tries to re-subscribe me to the Sent mail
folder.  That fails and then the message isn't saved.

Can someone else try to reproduce it?

Comment 7

[Christopher Blizzard (:blizzard)]

OK, scratch the previous test.  It works fine.  Here's a way to reproduce this:

1) Fire up mozilla with just a browser window open
2) Compose a new mail message from the browser window
3) Send the message.

I get the error, assuming that the Sent mail is on the IMAP server.

Can anyone reproduce this?  It's pretty obvious that it has something to do with
the mail window never being opened.

Seth?  Anyone?  This bug has been around for months and I can't get anyone else
to reproduce it.

Comment 8

[(not reading, please use [e-mail] instead)]

this worked for me against nsmail-1

* OK dredd.mcom.com IMAP4 service (Netscape Messaging Server 4.15 Patch 4 (built
 Dec  7 2000))

blizzard, what type of imap server are you on?

can you get an IMAP protocol log?  
http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Comment 9

[Christopher Blizzard (:blizzard)]

Attachment: imap log

IMAP log.  Over SSL.

Comment 10

[Christopher Blizzard (:blizzard)]

Attachment: imap log that shows success

This IMAP log shows me sending a piece of email and it being stored
successfully.  This happens if I fire up the browser and bring up mail/news
first.	I don't have any servers that I are opened automatically, don't use
biff and don't have any servers that are opened on startup so the Sent mail
delivery is the first contact with a mail server.

Comment 11

[Christopher Blizzard (:blizzard)]

Here's the stack when the subscribe happens:

#0  nsImapProtocol::Subscribe (this=0x86720c0, mailboxName=0x43024bf8 "Sent")
    at nsImapProtocol.cpp:6130
#1  0x436207b1 in nsImapProtocol::OnEnsureExistsFolder (this=0x86720c0, 
    aSourceMailbox=0x43024bf8 "Sent") at nsImapProtocol.cpp:4938
#2  0x43624b45 in nsImapProtocol::ProcessAuthenticatedStateURL (this=0x86720c0)
    at nsImapProtocol.cpp:6305
#3  0x43614f7a in nsImapProtocol::ProcessCurrentURL (this=0x86720c0)
    at nsImapProtocol.cpp:1136
#4  0x43614935 in nsImapProtocol::ImapThreadMainLoop (this=0x86720c0)
    at nsImapProtocol.cpp:993
#5  0x43613bfa in nsImapProtocol::Run (this=0x86720c0)
    at nsImapProtocol.cpp:759
#6  0x4021961d in nsThread::Main (arg=0x86268e8) at nsThread.cpp:120
#7  0x402ed352 in _pt_root (arg=0x8672a68) at ptthread.c:214
#8  0x40312c6f in pthread_start_thread (arg=0x4439dbe0) at manager.c:284
#9  0x40312d5f in pthread_start_thread_event (arg=0x4439dbe0) at manager.c:308

Comment 12

[Christopher Blizzard (:blizzard)]

I've been looking at nsImapProtocol::ProcessAuthenticatedStateURL() to figure
out why things have been going bad.  In the good case, where this doesn't happen
the imapAction looks like this:

nsImapAppendMsgFromFile
nsImapListFolder
nsImapListFolder

and in the bad case:

nsImapEnsureExistsFolder
nsImapListFolder
nsImapListFolder

Comment 13

[Cavin Song]

In nsImapProtocol::OnEnsureExistsFolder, I'm not sure if the following code is 
necessary:

  if (exists)
  {
      Subscribe(aSourceMailbox);
  }

because I don't know if some scenarios will need it.

Comment 14

[Christopher Blizzard (:blizzard)]

I've finally tracked why I get this error only when I haven't started up mail. 
It's because of this code in nsMsgCopy.cpp:

nsresult 
nsMsgCopy::CreateIfMissing(nsIMsgFolder **folder, PRBool *waitForUrl)
{
  nsresult ret = NS_OK;
  if (folder && *folder)
  {
    nsCOMPtr <nsIFolder> parent;
    (*folder)->GetParent(getter_AddRefs(parent));
    if (!parent)
    {
      nsCOMPtr <nsIFileSpec> folderPath;
      // for local folders, path is to the berkeley mailbox. 
      // for imap folders, path needs to have .msf appended to the name
      (*folder)->GetPath(getter_AddRefs(folderPath));
        PRBool isImapFolder = !nsCRT::strncasecmp(mSavePref, "imap:", 5);
      // if we can't get the path from the folder, then try to create the storage.
      // for imap, it doesn't matter if the .msf file exists - it still might not
      // exist on the server, so we should try to create it
      PRBool exists = PR_FALSE;
      if (!isImapFolder && folderPath)
        folderPath->Exists(&exists);
        if (!exists)
        {
          (*folder)->CreateStorageIfMissing(this);
          if (isImapFolder)
            *waitForUrl = PR_TRUE;
 
          ret = NS_OK;
        }
      }
    }
  return ret;
}

In the case where the mail window has been loaded,
(*folder)->GetParent(getter_AddRefs(parent)) will get the parent and the
function will return immediately.  If the mail window hasn't been opened, that
code doesn't return a parent and the code below it is executed which eventually
causes the error.

This is because the folder information is populated from RDF when the mail
window is opened.  If you've never opened it mParent on that folder is empty so
we try to make sure it exists.

I'll keep looking at this and try to figure out why we are subscribing to a
folder when we're already subscribed to it.

Oh, can someone please fix the indentation under folderPath->Exists(&exists) ? 
It's making me nuts since for a long time I thought the block for the if() above
it went a lot farther than it does.

Comment 15

[Greg K.]

Confirming using Mozilla/2002020405 on Mac OS. Setting All/All.

Comment 16

[[e-mail]]

*** Bug 91858 has been marked as a duplicate of this bug. ***

Comment 17

[[e-mail]]

Making my bug a duplicate. As [e-mail] commented in that bug, the
below bug is probably also a duplicate:

http://bugzilla.mozilla.org/show_bug.cgi?id=96932

Comment 18

[[e-mail]]

I'm seeing this also now on OS X when I have opened the mail window, for some
odd reason. I might be doing something wrong, but there seem to be problems
here, indeed.

Comment 19

[Nahor]

*** Bug 134471 has been marked as a duplicate of this bug. ***

Comment 20

[Karen Huang]

*** Bug 107102 has been marked as a duplicate of this bug. ***

Comment 21

[Ben Bucksch (:BenB)]

*** Bug 90147 has been marked as a duplicate of this bug. ***

Comment 22

[Ben Bucksch (:BenB)]

I can reliably reproduce this, too. (Usually unintentionally :-/ .) (FWIW, I use
UW-IMAP with a root path (advanced server prefs) of "foobar/".)

Comment 23

[Bryce Mozilla Nesbitt]

I get this also.  To reproduce, set up Mozilla with an imap server.  No special
folders are required.  Send a message BEFORE or AFTER, but not during, reading
mail.  Mozilla responds with:

    The current command did not succeed.
    The mail server responded: SUBSCRIBE failed:
    Already subscribed to mailbox Sent.

Mozilla will keep trying forever with the message:
     Copying message to Sent folder

The old Netscape 4.7 trick of using 'send later' or 'save as draft' does not
work.  You're stuck, the only choice is to cut & paste the message.

-----------------------------------------------------------------------------

My version numbers are:
     * OK idiom.com IMAP4rev1 v11.241 server ready
      Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020424

See also bug #89285 and bug #96932

Comment 24

[Bryce Mozilla Nesbitt]

Duplicate bug #134471 has the best summary of steps to reproduce the bug.  It's
100% repeatable, 100% cross-platform.  Using File->New->Message from the browser
is the easiest way to see the problem.

Why is this bug still NEW, not OPEN?

Comment 25

[Greg K.]

NEW is the correct status for a confirmed bug. The next step is ASSIGNED.

Comment 26

[Karen Huang]

This might be a UW IMAP Server interop bug....
Nominating nsbeta1 since I saw many similar bugs complaining about this problem.

Comment 27

[Ben Bucksch (:BenB)]

I don't think that this is limited to UW IMAP, blizzard's log doesn't small like
UW IMAP to me. huang, can you confirm?

Comment 28

[Greg K.]

Happens with my Solstice server, too.

Comment 29

[Christopher Blizzard (:blizzard)]

For what it's worth I think I've only run against UW IMAP servers.

Comment 30

[David :Bienvenu]

no, nothing to do with UW IMAP, I believe.

Comment 31

[R.K.Aa.]

*** Bug 145642 has been marked as a duplicate of this bug. ***

Comment 32

[Michael Buckland]

Discussed in mail news bug meeting. Decided to plus this bug.

Comment 33

[Christopher Blizzard (:blizzard)]

Yay! *dance*

Comment 34

[Gregg Meehan]

*** Bug 96932 has been marked as a duplicate of this bug. ***

Comment 35

[scottputterman]

reassigning to naving.

Comment 36

[Navin Gupta]

Attachment: proposed fix

The fix is to not issue a subscribe if LSUB command succeeds for that
mailboxName.
Can I get reviews, thx.

Comment 37

[David :Bienvenu]

I think this should be fixed at the problem area, not way downstream like this.
Did you investigate trying to build up the folder list in this situation if it's
not built up?

Comment 38

[Navin Gupta]

We cannot rely on rdf folder getting created, we have to check if the folder
exists on the server. So we are correct when we do "EnsureFolderExists". The
problem is we are subscribing "Sent" folder again and some servers don't seem to
mind like nsmail-2 but others do(throw that alert). 

Comment 39

[David :Bienvenu]

I'm not saying we should rely on it - I'm saying we should cause it. I'm not
100% sure that saying LSUB on an unsubscribed or non-existent folder would
actually give an error - the RFC seems to say that it would, but it's not explicit.

Comment 40

[Navin Gupta]

Attachment: proposed fix, v2

This fix will build the folder hierarchy when we try to send mail without
opening mailnews. This will avoid calling CreateStorageIfMissing that will
avoid calling 
EnusreFolderExists, so there will be no alert for subscribe. 

David, thanks for making me look into it some more. please review, thx.

On a different note LSUB command should also have fixed this problem.

Comment 41

[David :Bienvenu]

I think I have a cleaner approach - try calling GetChildWithUri on the root
folder (with the deep param set to true)
this will cause the hiearchy to get created as a side effect, w/o changing the
normal imap GetSubFolders call.

Comment 42

[Navin Gupta]

Attachment: fix, v3

This fix uses GetChildWithURI.

Comment 43

[David :Bienvenu]

Comment on attachment 86270 [details] [diff] [review]
fix, v3

sr=bienvenu, if you get rid of this line since it's not needed:
+	 nsCOMPtr<nsIEnumerator> subFolders;

Comment 44

[Navin Gupta]

Cavin, can I get r=? thx. 

Comment 45

[Cavin Song]

Comment on attachment 86270 [details] [diff] [review]
fix, v3

r=cavin.

Comment 46

[Navin Gupta]

fixed

Comment 47

[Navin Gupta]

marking fixed

Comment 48

[scottputterman]

Greg, can you verify this on the trunk?  Could someone who was seeing this
verify this as well?

Comment 49

[Greg K.]

FWIW, bug still occurred using FizzillaCFM/2002052918 (rv1.0.0).

Verified fixed using FizzillaCFM/2002060508.

Comment 50

[scottputterman]

adding adt1.0.1+.  Please get drivers approval before checking into the branch.

Comment 51

[Randell Jesup [:jesup] (needinfo me)]

A couple of problems with the patch:

1) Spurious partial style change (I can live with this, but it shouldn't have
gotten past review).

2) if rootMsgFolder is null (or if GetChildWithURI() doesn't set *msgFolder),
then we return NS_OK without setting *msgFolder.  Depending on what's there,
this could be bad, and the old code would never do that - it only returned NS_OK
if *msgFolder was a non-null addref'd object pointer.  If the caller doesn't
initialize it (or if it does but doesn't handle the NS_OK w/ nsnull *msgFolder
case) we could crash.

Resolve these problems (w/ r&sr as needed) or convince me these concerns are
invalid, and I'll approve for branch.

Comment 52

[Scott Collins]

Comment on attachment 86270 [details] [diff] [review]
fix, v3

this code looks pretty good, but I think that you should |*msgFolder = 0;| in
the case where |if (server)| fails.  Else you are returning |NS_OK|, but
haven't initialized |msgFolder|.  The most robust way to accomplish this is
probably to clear the output parameter on entry, just after your NULL test for
it.

After that fix, I think this will be a no-brainer for approval.

Comment 53

[Randell Jesup [:jesup] (needinfo me)]

Don't forget the implicit API change in that previously it would never return
NS_OK with a null *msgFolder.  Now, perhaps callers should have always handled
that - and perhaps they have; perhaps it was documented as possible.  If it
isn't documented as a possible result, however, it should be, and all the
callers should be checked (regardless).  Or, if *msgFolder is null don't return
NS_OK (but that might defeat the purpose of this patch - I'm not sure).

Comment 54

[Navin Gupta]

As far as I can see we are using nsCOMPtr<nsIMsgFolder> everywhere so we are
initializing to null when we pass to LocateMessageFolder and before we use this
nsIMsgFolder we do check for null (CreateIfMissing). 
So this patch would work fine. Also the chances of not getting server is very less
unless you pass in a bogus url from the prefs file which no sane user will do. 

Comment 55

[Randell Jesup [:jesup] (needinfo me)]

Even if all the current callers happen to initialize it, the routine must set it
to null, or not return NS_OK in that case, or document it must be null when
passed in and add an assert for that.  The API doesn't currently specify (except
perhaps implicitly) that *msgFolder should be null on entry.

This is an interface; you can't assume all future callers will be the same as
current callers.  Also, you can't assume users won't put screwy values in prefs
files.  Some users aren't sane.  Others can't type.  And others are just plain
confused.

Comment 56

[Navin Gupta]

Attachment: patch

I have initialized *msgFolder = nsnull and return NS_MSG_ERROR_FOLDER_MISSING
if we don't have a rootMsgFolder or server.

Comment 57

[Judson Valeski]

this is such a nasty bug. we want it in 1.0.1 once the final issues get resolved.

Comment 58

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 88196 [details] [diff] [review]
patch

approval noted for 1.0 branch.	Please remove mozilla1.0.1+ when fixed and add
fixed1.0.1

Comment 59

[Navin Gupta]

fixed on branch

Comment 60

[Gregg Meehan]

Tested on branch build 20020620, this is working correctly now. Using CTRL-M
with browser open and mail window closed, mail sends correctly without error.
Marking as  VERIFIED

Comment 61

[Greg K.]

Gregg, did you test by launching Mozilla and then sending a message without
invoking MailNews at all first? That was the problem, not sending a message
merely with the MailNews window closed.

Comment 62

[Gregg Meehan]

Yes, launched Mozilla after a fresh reboot, mailnews had not been launched at all.

Comment 63

[Judson Valeski]

I believe I'm seeing this, or something similair, w/ mozilla trunk winXP builds
again. The error I get back isn't SUBSCRIBE, but, something else (sorry, don't
have the dialog text right now).

Comment 64

[David :Bienvenu]

Sounds like a new bug, Jud. I tried this with a trunk build from yesterday and
it worked fine. If you can reproduce it, and attach the error, and perhaps an
imap protocol log (see c#8), then we can see what's going on.

Comment 65

[Judson Valeski]

Attachment: screenshot of error msg

using a two day old winXP trunk build, I'm seeing this again. I've attached the
error dialog screen shot.

Comment 66

[Ben Bucksch (:BenB)]

Jud, I think what you see is very different, in fact a user error. I think Yahoo
wants SMTP-after-POP authentication for sending mail, because the "relaying
denied" suggests a policy violation of the server.

Comment 67

[Christopher Blizzard (:blizzard)]

That's an entirely unrelated message to the error that this bug is about.  This
is about IMAP subscribe errors when sending mail, that's an error for sending
over SMTP.

Comment 68

[timeless]

i actually got the error listed for this bug. my cvs build is ~2weeks old.
i had something like this:
timeless@myrealbox: timeless@localhost (imap)

I needed to reply using a different return address, so i created a new account:
real@name: timeless@raistlin (imap) - raistlin is localhost

I never selected that account or its children.

I replied to a message in timeless@myrealbox's inbox, selected my real@name
account and sent the message.

I think the message was sent, but I got

"The current command did not succeed.  The mail server responded: SUBSCRIBE
failed: Already subscribed to mailbox Sent."

I still have the "Sending Messages - ..." dialog
  Status: Copying message to Sent folder...
Progress: [gray=============================][indeterminate\\\\\\\\\\\\\]

[  Cancel  ]

(offtopic, i'm sending one message, is there a bug about the title being silly,
 it should be Sending Message - %messagesummary% no 's'.)

Comment 69

[Judson Valeski]

I just opened a new bug for what I'm seeing
http://bugzilla.mozilla.org/show_bug.cgi?id=184439

thanks

Comment 70

[Sven Krohlas]

*** Bug 141437 has been marked as a duplicate of this bug. ***