787605 - Movemail cannot find mail spool in Thunderbird 15

Status: RESOLVED WONTFIX

(MailNews Core :: Movemail, defect)

Description

[Martijn Dekker]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0
Build ID: 20120824154833

Steps to reproduce:

Upgraded from TB 14 to TB 15.
Checked mail on a UNIX Movemail account.
(System: Slackware Linux)


Actual results:

An error message occurred: "cannot localize mail spool file" (or similar; this is translated back from Dutch). Downgrading to TB 14 fixes the problem.


Expected results:

Messages should have been transferred from the mail spool into TB as usual.

Comment 1

[Martijn Dekker]

I am not a C++ programmer, but I suspect the patch at http://hg.mozilla.org/comm-central/rev/d3a9bb8b85f9 has something to do with the introduction of this bug. This patch removed a "printf" which was for debugging purposes, but embedded in that printf statement is a call to spoolPath.get() which could be essential to get the path to the spool file in the first place. Its removal might explain the error message. I could be way off base but I thought I'd throw this out there.

Comment 2

[Martijn Dekker]

I neglected to add that the symptom in question occurs with Mozilla's official Dutch-language Linux i686 build of Thunderbird 15.0.

New information: 

I built TB from source (using the SlackBuild script from slackware-current), and with that build, segfault occurs immediately when trying to check mail for a movemail account.

When I rebuilt TB with the patch at http://hg.mozilla.org/comm-central/rev/d3a9bb8b85f9 reverse-applied, movemail functionality appears to work as normal, corroborating my suspicion in Comment 1.

Comment 3

[Ludovic Hirlimann [:Usul]]

Martijn could you come up with a patch that would remove the debug statment and fix your issue ?

Comment 4

[Joshua Root]

Movemail seems to be working fine for me in the en-GB version of TB 15, FWIW.

Comment 5

[fimbles]

I have the problem on Ubuntu 12.04 64bits after upgrading to TB 15.0. I tried with Thunderbird 15.0 downloaded from Mozilla and have the same problem (EN and FR version).
After removing .thunderbird/ folder, TB started fine but it crashed after I configured a Movemail account.

Comment 6

[Martijn Dekker]

Attachment: Possible movemail fix

In response to Comment 3, here is a patch which appears to fix the problem for me - it preserves the call to spoolPath.get() but not the surrounding printf() that was there before. A build compiled with that patch transferred mail from my movemail account successfully.

Disclaimer: I don't actually know what I'm doing (see Comment 1).

Comment 7

[Kent James (:rkent)]

It's hard to believe that spoolPath.get() is the real issue here, as that would just return a pointer to the character string. Maybe some side effect of that, such as timing, could have affected this?

Are there any crash ids associated with the reported crashes?

Comment 8

[[e-mail]]

(In reply to Kent James from comment #7)
> It's hard to believe that spoolPath.get() is the real issue here, as that
> would just return a pointer to the character string.

As shown by the source code:

const char_type* get() const
  {
    return mData;
  }

So I would expect the compiler to completely optimise it away.

Comment 9

[[e-mail]]

So, assuming the original Dutch error message was "Kon e-mailspoolbestand niet lokaliseren." then that error message can only happen if LocateSpoolFile fails or returns an empty string.

If you have the MAIL environment variable set, then LocateSpoolFile only uses that variable.

If instead you have the USER or USERNAME environment variable set, then LocateSpoolFile looks for a file of that name in any of the following directories: /var/spool/mail/ /usr/spool/mail/ /var/mail/ /usr/mail/ and returns the first one it finds.

You can try working around the problem by manually setting the spool file location. You do this by creating a string preference "mail.server.server"<N>".spoolDir" containing the full path.

Comment 10

[[e-mail]]

Comment on attachment 659364 [details] [diff] [review]
Possible movemail fix

Sorry, I can't see how this is relevant at all.

Comment 11

[Martijn Dekker]

Re Comment 9: yes, that was the error message.

Meanwhile I cannot seem to reproduce the problem anymore. It is not occurring for me in TB 15.0.1, but there is no listed change in that version that should affect this. Even more strangely, when I downgrade back to TB 15.0, the problem stays vanished. I don't get it.

At least fimbles in Comment 5 had the same problem, so there is some indication that I didn't simply make it up.

Apparently the appearance that my clueless patch made any difference was a simple coincidence. Could the real problem be a race condition somewhere?

Comment 12

[fimbles]

I have a Crash ID: bp-684d764f-9895-4572-9722-84d3b2120905 (submitted on sept, 5th)

I just tried again and like Martijn Dekker, I can not reproduce the crash. I got the error message that the spool file can not be located but Thunderbird did not crash.

Comment 13

[ardbug]

Problem still exists in 1.7. Movemail appeared to run (it left a file in the .thunderbird/*.default/Mail directory; also, strace showed the (/var/spool) mailfile being accessed) then segfaulted. Based on the previous comment, it sounds like it only dies if it finds the file and there is something in it.

Haven't looked at the sourced, yet.

Comment 14

[Tony Mechelynck [:tonymec]]

(In reply to ardbug from comment #13)
> Problem still exists in 1.7. Movemail appeared to run (it left a file in the
> .thunderbird/*.default/Mail directory; also, strace showed the (/var/spool)
> mailfile being accessed) then segfaulted. Based on the previous comment, it
> sounds like it only dies if it finds the file and there is something in it.
> 
> Haven't looked at the sourced, yet.

Have you got one of the environment variables mentioned in comment #9? (and which one, to what value?)

FWIW, I get eight Movemail messages per week (from cron-daily and cron-weekly) with no problem, but I'm on Linux, not on Mac OS X. The latest one is dated two minutes ago.
Mozilla/5.0 (X11; Linux x86_64; rv:20.0) Gecko/20121129 Firefox/20.0 SeaMonkey/2.17a1 ID:20121129092632 c-c:efd79c122d2f m-c:85471409cbfb

Comment 15

[ardbug]

Running Fedora 17. I set the "mail.server.server"<N>".spoolDir" preference variable. It found the mail file and I saw a read in the strace. Then it died.

Comment 16

[Tony Mechelynck [:tonymec]]

P.S. I use mbox storage (all messages in one file) with no problems. Bug 793524 comment #0 describes a crashing problem with maildir storage (each message in a separate file) for a Movemail account.

Fimbles, ardbug: what is mail.server.server<n>.storeContractID set to (replacing <n> by the integer corresponding to your Movemail account)? The default is @mozilla.org/msgstore/berkeleystore;1 but if you have @mozilla.org/msgstore/maildirstore;1 instead then this bug and bug 793524 might be dupes of each other.

Comment 17

[ardbug]

It's set to berkelystore:
user_pref("mail.server.server3.storeContractID", "@mozilla.org/msgstore/berkeleystore;1");
user_pref("mail.server.server3.type", "movemail");

Comment 18

[fimbles]

My settings:
user_pref("mail.server.server2.storeContractID", "@mozilla.org/msgstore/berkeleystore;1");
user_pref("mail.server.server2.type", "movemail");

Comment 19

[Tony Mechelynck [:tonymec]]

Ah. Then IIUC this bug is not a dupe of bug 793524.

Comment 20

[Wayne Mery (:wsmwk)]

(In reply to fimbles from comment #12)
> I have a Crash ID: bp-684d764f-9895-4572-9722-84d3b2120905 (submitted on
> sept, 5th)

filed bug 856577

Comment 21

[Mikhail T.]

Attachment: Updated patch to apply to 17.0.5esr

Martijn's patch would not apply directly because of an unrelated change.

With this updated version I am finally able to read mail coming into /var/mail/FOO -- unpatched 17.0.5esr was not able to do it, without any reasonable explanation.

Comment 22

[Joshua Cranmer [:jcranmer]]

Comment on attachment 745675 [details] [diff] [review]
Updated patch to apply to 17.0.5esr

There is likely a deeper symptom going on here, which this patch does not fix.

Comment 23

[Mark Banner (:standard8)]

Comment on attachment 745675 [details] [diff] [review]
Updated patch to apply to 17.0.5esr

Sorry, but this definitely isn't right.

Looking at hg history, it is more likely this changeset the caused a possible change:

http://hg.mozilla.org/comm-central/rev/14888d952b42

The best way forward on this, is for someone who can reproduce the issue, figure out the exact message in english, and to look for that in mxr (http://mxr.mozilla.org/comm-central/). That should then give the name/id of the message, and hopefully you should be able to trace that back to the specific failure point.

Once we've got that I think debugging this should be a bit easier as we'll have an idea of where this is going wrong.

Comment 24

[:aceman]

Could the spool file have moved? Into another directory? Or your linux username changed? It looks like if you have the path stored in the preference and the file is not found, TB shows the error. Only if the pref is empty, TB tries to locate the file in standard locations.

Can we get more info here?

Comment 25

[krichter]

I can confirm the problems described above for TB 24.2.0 running on Ubuntu 13.10 (and TB installed from repositories). Mail is delivered to /var/mail/$USER and /var/spool/mail/$USER by procmail (listing the directory shows that there's content) (retrieved from AOL IMAP by fetchmail).

What preference can I investigate in order to move this forward? I set up a movemail account with $USER as username and $USER@none as address.

Comment 26

[krichter]

The problem is that the preference mail.server.serverX.spoolDir is not created. I set it to /var/spool/mail and invoked chmod 01777 /var/spool/mail and now I when invoking "get mail" in TB a warning dialog pops up and tells me that /var/spool/mail/.lock could not be created and that I have to set /var/spool/mail to 01777 (probably a vague translation). I assume that before .lock there an empty string which is the invalid value of a preferences which is also missing.

Comment 27

[:aceman]

Try to also add the file (user) name to that path.

Comment 28

[:aceman]

But also, do you have write access to that directory? I am not sure it is needed (we just need to truncate the spool file) but so that we know what the situation is.

Comment 29

[krichter]

In the previous comment I confused /var/mail/ and /var/spool/mail. No I found out that my fetchmail delivers to /var/mail/username.

Appending the username (richter) to spoolDir causes error message that the spool file can't be truncated (I set mail.server.serverX.spoolDir to /var/mail/username). All files in /var/mail/username have at least rw permission for owner and are owned by richter:mail. 

Does that bring you further, :aceman?

Comment 30

[:aceman]

OK, I looked at the code and the pref mail.server.serverX.spoolDir is never created automatically. But it is obeyed if it is found. If it does not point to a valid path/file we still fall back to the automatic lookup for the file: check the value of MAIL environmental variable. If set but the path is wrong, we fail. If it is empty, we try /var/spool/mail/$USER, /var/mail/$USER, /usr/spool/mail/$USER, /usr/mail/$USER .

So krichter722, I think you haven't yet described what your MAIL and USER env. variables are set to. I think the username set inside TB (in account manager) is ignored (that is already filed elsewhere).

Comment 31

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #30)
>... 
> So krichter722, I think you haven't yet described what your MAIL and USER
> env. variables are set to. I think the username set inside TB (in account
> manager) is ignored (that is already filed elsewhere).

krichter722, feedback?

Comment 32

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 33

[Wayne Mery (:wsmwk)]

Support for Movemail has been removed as of version 87 per Bug 1625741