Closed Bug 492921 Opened 11 years ago Closed 11 years ago

archiving against dovecot server sends tb in a tailspin


(MailNews Core :: Networking: IMAP, defect, major)

Not set


(Not tracked)

Thunderbird 3.0b3


(Reporter: davida, Assigned: Bienvenu)



(3 files)

Attached file imap log

 - get an imap account against dovecot

sleet:MacOS davida$ dovecot --build-options
Build options: ioloop=kqueue notify=kqueue ipv6 openssl
Mail storages: maildir mbox dbox cydir raw
SQL drivers:
Passdb: checkpassword pam passwd passwd-file
Userdb: passwd passwd-file prefetch static
sleet:MacOS davida$ dovecot --version

  - pick a message in the inbox, hit 'a'

--> thunderbird takes 100% CPU, forever.

Attaching tail end of imap log.

Key line seem to be the refusal to create a sub-folder:

-1332359168[1bfb5480]: fa6000:localhost:S-Archives:CreateNewLineFromSocket: + idling
-1331826688[1f5ac710]: ReadNextLine [stream=1e18eb88 nb=25 needmore=0]
-1331826688[1f5ac710]: af1e00:localhost:S-Trash:CreateNewLineFromSocket: 5841 OK List completed.
-1331826688[1f5ac710]: af1e00:localhost:S-Trash:SendData: 5842 create "Archives/2009"
-1331826688[1f5ac710]: ReadNextLine [stream=1e18eb88 nb=50 needmore=0]
-1331826688[1f5ac710]: af1e00:localhost:S-Trash:CreateNewLineFromSocket: 5842 NO Mailbox doesn't allow inferior mailboxes
-1332359168[1bfb5480]: fa6000:localhost:S-Archives:SendData: DONE
-1332359168[1bfb5480]: ReadNextLine [stream=1847ad98 nb=25 needmore=0]
-1332359168[1bfb5480]: fa6000:localhost:S-Archives:CreateNewLineFromSocket: 6032 OK Idle completed.
-1332359168[1bfb5480]: fa6000:localhost:S-Archives:ProcessCurrentURL: entering
-1332359168[1bfb5480]: fa6000:localhost:S-Archives:ProcessCurrentURL:imap://davida@localhost:143/ensureExists%3E/Archives/2009:  = currentUrl
-1331826688[1f5ac710]: af1e00:localhost:S-Trash:SendData: 5843 IDLE
-1331826688[1f5ac710]: ReadNextLine [stream=1e18eb88 nb=10 needmore=0]
-1331826688[1f5ac710]: af1e00:localhost:S-Trash:CreateNewLineFromSocket: + idling
Flags: blocking-thunderbird3+
the dovecot server gozer setup is perfectly happy with this - you set up your own dovecot server on your mac, I guess...I suppose I could do that.
So this fixes the spin, which is the most important thing. How to handle the failure downstream is slightly more complicated. We could treat failure to create any of the sub-folders as a reason to use the top-level archive folder. Beyond that, it would be nice to not always try to create the sub-folders when it's going to fail. Bug 495318 might be helpful here, and/or we could explicitly LIST the parent folder in this failure case to see if it has /NoInferiors set, and if it does, that should prevent us from attempting to create the sub-folders anymore...
Attachment #380276 - Flags: review?(david.ascher)
Comment on attachment 380276 [details] [diff] [review]
get rid of spin cycle - checked in.

That does indeed fix the spin, and the fix looks good to me. I don't have real review privs, but I think you know that and were just delegating to me.

About the bigger strategy, I don't have an architectural opinion.  I have lots of UX thoughts:

a) if a server won't support nested folders, the archiving system should not try -- so maybe the code that determines the archive folder name should make folders like "Archive 2009" instead of Archive/2009 for those servers.  I don't know if we collect capabilities about servers on setup, but it'd be nice if we did.

b) failure handling on archive should probably log something in activity manager.
Attachment #380276 - Flags: review?(david.ascher) → review+
(In reply to comment #3)
> (From update of attachment 380276 [details] [diff] [review])
> That does indeed fix the spin, and the fix looks good to me. I don't have real
> review privs, but I think you know that and were just delegating to me.

Yes, you know the code better than anyone else, and you're encountering the bug, so I had to delegate to you :-)

> a) if a server won't support nested folders, the archiving system should not
> try 

We don't, actually, Buried in my previous comment was an explanation that in this case, we don't know until we try, and some ideas for teasing out this information from the server. I suspect, for example, if you turned off imap subscription in the advanced imap server settings for your dovecot account, we would simply use the Archives folder. The IMAP List command, which we use if subscription is turned off, tells us that the Archives folder can't have inferiors, and we persist and obey that setting already.
fix for tailspin checked in - I'll leave this open for dealing with the problem, but removed blocker severity.
Severity: blocker → major
I'll try to get this for b3, but putting in b4 for now.
Target Milestone: --- → Thunderbird 3.0b4
Attachment #380276 - Attachment description: get rid of spin cycle → get rid of spin cycle - checked in.
What I didn't say explicitly is that generally the imap server tells us when a folder can't have sub-folders by returning the /NoInferiors flag. We do respect that flag in the archive code. But LSUB (as opposed to LIST) sometimes does not return that flag because the IMAP RFC gives the server flexibility w/ the LSUB responses. Which is very convenient for the server but unfortunate for the client that wants to use LSUB.

The LIST-EXTENDED extension deals with this by allowing the client to ask the server for subscription *and* full flag info at the same time.  But it's a very new extension and the dovecot servers we're testing against don't support it. So we need to do something like LIST on failure to try to get the /NoInferiors flag from the server. We'll also need to not clear the /NoInferiors flag on a folder when LSUB doesn't tell us about the /NoInferiors flag. And we'll have to live with the case where the /NoInferiors flag becomes stale - which should be rare.
This makes it so we'll try to create a year (e.g., 2009) sub-folder; that will fail, and the archive code will use the top level folder. When we fail to create the sub-folder, we explicitly LIST it - that should cause the server to tell us that the top level folder doesn't allow sub-folders, and we won't try to create sub-folders for the current session. 

The next time you run, we'll clear the /NoInferiors flag on startup, because LSUB doesn't tell us about it, and the first archive command will attempt to create the sub-folder(s). Rinse, repeat...

Gozer has a dovecot server setup at momo that doesn't allow sub-folders, for testing.
Attachment #382553 - Flags: superreview?(bugzilla)
Attachment #382553 - Flags: review?(bugzilla)
Attachment #382553 - Flags: superreview?(bugzilla)
Attachment #382553 - Flags: superreview+
Attachment #382553 - Flags: review?(bugzilla)
Attachment #382553 - Flags: review+
fix checked in.
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: Thunderbird 3.0b4 → Thunderbird 3.0b3
You need to log in before you can comment on or make changes to this bug.