Closed Bug 81387 Opened 24 years ago Closed 7 years ago

Check POP/IMAP before SMTP send (SMTP-after-POP)

Categories

(MailNews Core :: Networking: SMTP, defect)

Product:
MailNews Core
Component:
Networking: SMTP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: matthew, Unassigned)

References

Details

(Whiteboard: [parity-becky])

The ability to have Mozilla automatically poll POP/IMAP prior to any SMTP traffic would be great for those of us using this mechanism for anti-relaying on our servers. Simply connect to POP, issue USER and PASS and then QUIT once it has validated prior to sending any SMTP traffic. For reference, "Becky" has this functionality and it works wonderfully.
Marking NEW.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Check POP/IMAP before SMTP send → [RFE] Check POP/IMAP before SMTP send
*** Bug 96312 has been marked as a duplicate of this bug. ***
This is a 4xp bug. NS4.7x has this feature. Therefore, this is not RFE, is it?
Severity: enhancement → major
Keywords: 4xp
Wait. This has been supposed to work by setting "Use user name" in the SMTP settings, right? Now, I do not see this working any more. Mozilla does not ask for password for the pop account when sending. So, this is actually a regression of bug 48799. Changing key work
Keywords: 4xp → regression
Summary: [RFE] Check POP/IMAP before SMTP send → Check POP/IMAP before SMTP send
Depends on: 90507
Bug 90507 might be the blocker of this bug. Adding dependency. FYI, 060708 trunk build can send mails with "use username" feature. However, 090508 trunk build for Mac is failing.
SMTP can also take a username/password pair with the AUTH command so there's really two different methods that need to be addressed: authenticating to the SMTP server by sending AUTH [PLAIN|LOGIN|DIGEST-MD5] authenticating by connecting to IMAP4/POP3 server before send I'm not sure that both cases are being addressed so just pointing this out. Sorry if I've missed it.
*** Bug 128235 has been marked as a duplicate of this bug. ***
Has there been any progress on this? I would like to nominate this functionality for "Mozilla 1.0". SPAM is an always increasing problem and therefore Mozilkla should support the commin methods against ist, such as "smtp after pop/imap". Note that this should be impmeneted for either imap or pop3. Our mail relay post.gaia.de checks both pop and imap logins.
Why has this been marked regression? It never worked in mozilla so far. Removing "regression" keyword. BTW, how about some triaging of this bug (it has no assigned milestone)
Keywords: regression
*** Bug 142968 has been marked as a duplicate of this bug. ***
I would like to stress again that this is a feature of growing importance - especially when fighting spam. Providers need closed-relays and there needs to be some kind of authentification. Therefore a POP3/IMAP login before sending SMTP stuff out makes sense. And it would save a lot of support cases with people using synchronization for both mail and news. None of the "big" Browser supports this feature on Win right now ... I would be happy if this could be implemented in post 1.2 builds ....
Just another note: I do not mean the mechanism of username authentification inside SMTP. I mean that when a browser requests POP3/IMAP mail, the current IP address of this user is listed in a special table on the server. SMTP calls are opened for this adress for some limited time. See http://www.davideous.com/smtp-poplock/ for details - here it's described for qmail mail servers. The only thing that needs to be changed inside Mozilla is the ORDER of mail processing: first login as POP or IMAP user, then sending SMTP stuff. NOT vice versa! I can't believe that this is a hard thing to do ;-)
I would like to nominate this as an "essential" for post-1.2b builds. Anti-relaying through "smtp after pop/imap" is a simple, but vital element of fighting spammers and therefore very common, at least here in Germany. Mozilla should support efforts to support this. Can we hope that will be assigned soon ;-)?
Why don't you just add the code from Linux? AFAIK, Everything under Linux seems to be GNU GPL FREE ! So just get the code out of Kmail or other applications which give POP before SMTP functionality. I am not a programmer, so don't have much idea about it, but I think using other people's code seems like sense to me since it saves time and makes software better. Over and Out. atul.
*** Bug 182484 has been marked as a duplicate of this bug. ***
I have 2 accounts in gmx.net: if I check mail before send, I can send mail OK. I don't mind if it's no made automatic or no ( but I do prefer Mozilla made this for me). My problem is I can't find a way to send from my 2 accounts. I can only send from one account, the other said: error bla bla bla . the mail server respond: {mp013-rz3} need to authenticate via SMTP-AUTH-Login Am I doing somethig wrong or there is no way to configure the accounts? Is this a new bug?
Hello. I hope you are not too annoyed of all the stupid questions you might read here but sorry, I simply have no clue where I can get help but here. This bug describes exactly my problem. I use Mozilla 1.4 and still have the same problem as reported in May 2001. In October 2002 everybody seemed to agree that this is an "essential" for Mozilla 1.2 builds. I don't want to be rude for I can imagine that it is a lot of work and takes time to do it but I would like to know if this problem is about to be solved or not? yours fabian fabi_gigi@hamburg.de
*** Bug 213427 has been marked as a duplicate of this bug. ***
I also vote for this to be fixed - as over 1 Million potential domains holder on Strato - one of the leading German cheapo web providers - are affected. Currently I use the "send later" work around, but the failure to perform a POP3 before any SMTP is the single reason why I cannot install thunderbird on some of my non-techy friend computers.
Characteristics of this bug =========================== - First noticed on 17.05.2001 - Keyword mail3 mail4 mail6 could be added - 27.000 ("resolved") dups of this bug have been filed. - Really Resolved: 0. - Workarounds: Millions. - Affected Users: - My Ma... - Dial-Up users not able to poll mail regularly (which would do the job) - All clients of providers running DRAC/popbsmtp-alike anti-spam solutions - In germany: at least all strato customers. (>1.000.000!) - Widely used in japan: Google for "pop before smtp eudora" - Look at http://www.activatormail.com/smtpauth.htm Currently, only The Bat! and Pegasus seem to have useable implementations. - Comment: Spam (and its abuse) is a every-day topic. The techniques used (by strato for example) may not be perfect, but may at least somewhat help reducing spam. Users are unable to force providers to disable this feature. "Reasons" for that "bug" =========================== Note that it's not about RFCs 2554 or 2195. (Comment #6)! Pop after Smtp is used for spam abuse. At least two implementations seem to be used by "various" (?) providers: - POP before SMTP : http://spam.abuse.net/adminhelp/smPbS.shtml Home (?) http://popbsmtp.sourceforge.net/ - Dynamic Relay Authorization Control : http://mail.cc.umanitoba.ca/drac/ A different approach to the problem: http://whoson.sourceforge.net/ popbsmtp is a perl script, scans logs and incorperates with postfix. drac consists of sendmail patches & uses rpc with pop/imap servers. Both products store a hash table of "active" (30 mins in case of drac), pop/imap sessions. They are used (instead of/in addition to regular smtp authentication [RFC above]) by sendmail/postfix to verify the sender's authencity. DRAC seems to be more widely used. Workarounds =========================== Every half a year, I have to tell my ma, that whenever she faces a cryptic message like "Please use POP before SMTP", that she needs to fetch mail before sending. The message she faces is the SMTP error reply created by the mailserver - which isn't that helpful to my ma. But why are such workarounds needed in a cutting-edge mail suite (that supports spam filters on the client side...)? ... life could be nicer... Proposed Solution =========================== For every outgoing SMTP session, we need to make sure to check/login to the mailbox of the corresponding POP/IMAP account. Starting point would be to add a additional UserPref, specifying a Pop/Imap-Account to be used to check before sending. Like userpref.smtpserver-1.popbsmtp="popaccount-2"; ...or for imap... userpref.smtpserver-1.imapbsmtp="imapaccount-1"; (Volunteers may implement it into the xul interface.) (Some pseudo code scrambled with real function names): Affected files: /mozilla/mailnews/local/src/nsPop3Service.cpp --> CheckForNewMail() Using this call might be easiest way to go, although the request for new mails is not needed. As mentioned in a related (duped) bug report, a simple Pop3Server::login() + auth() would be sufficent... /mozilla/mailnews/compose/nsSmtpService.cpp fix funciton SendMailMessage (around line 145), add call to new smtpServer->GetTryPopBeforeSmtp(&identity); // fetch userprefs smtpServer->GetTryImapBeforeSmtp(&identity); if (pop/imap server defined for this smtp server) exec...: func GetTryPopBeforeSmtp(&identity) { if (pref.this.mailserver.popbsmtp) return pop3Service::CheckForNewMail( &identity ) else return false; } func GetTryPopBeforeSmtp(&identity) { if (pref.this.mailserver.imapbsmtp) { //printf("Imap-Before-Smtp: Fix Milestone #2\n"); //ImapService?::CheckForNewMail( userpref.popbsmtp.value ) } return false; } Questions: Can these messages be sent at any time to pop3service instance? Had only little time to investigate, but could be the right places to hook in. Will this cause a big slowdown or are these methods available anyway? Any hint of a core developer wheter that's the approach to go would encourage me to investigate further. Alternatives =========================== - Ask Strato et al to be more friendly to spammers. - Tell Ma to use a diffrent mailclient. (NO!! Nightmare!!!) - Let people pay billy g. some money to keep our mailboxes clean. - Tell us all that we're stupid and close this bug as "wontfix" ? - Tell us that it's already done for thunderbird? cya, Jan
Matthew Peddlesden, the original reporter, pointed out to me by mail: - "Becky" he refered to, was not a patch by a Person called "Becky" to mozilla, but just another mail reader ( http://www.rimarts.co.jp/becky.htm ) that supports exactly what we need. - He also made a proposal, which would be the deluxe-way to solve this issue: ----quote---- Essentially all it needs it the ability to make sure it issues a POP3/IMAP login request prior to sending any email out on a matched SMTP server (i.e. for SMTP server A check POP3 server A first, for SMTP B use POP3 B first and so forth). If the SMTP message fails to send due to spam, perhaps also then try to check all of the POP3/IMAP registered servers one at a time until the message sends or you run out of servers, recording whatever was successful, as this will then auto-configure it in the case of something slightly less intuitive for the end user. It might even do that without the user requesting it, try sending the smtp message, if it bounces immediately with a "you cannot relay" then try checking POP3/IMAP and resending, if that works, flag it as such in the config and do that in future, that way users don't have to find an option and tick a box - which many probably wouldn't be able to understand :-) ----/quote---- I really like his proposal! Anyway, I'll try the first step / easier approach - as soon I get some feedback.
*** Bug 237127 has been marked as a duplicate of this bug. ***
> But why are such workarounds needed in a cutting-edge mail suite POP before SMTP as "technique" itself is a workaround. I couldn't even find a RFC for it. If a server wants a SMTP client to authenticate, it should use SMTP-Auth, that's what it is for.
This shouldn't be too hard to do - basically, we'd add a new smtp server pref that's the account to check new mail before connecting to the smtp server. The tricky part is that checking for new mail via pop3 or imap is asynchronous, and we'd need to wait for the check to finish before doing the smtp send. I wonder if we could take advantage of the nsIMsgLogonRedirectionRequester feature to do this. We'd add a custom redirector that did a check for new mail on the account, and when it's finished, tell the smtp protocol to go ahead with the original user name, password, etc.
Assignee: mscott → bienvenu
David, I am not shure if we really need a serialized sequence where smtp only starts after all pop/imap connections have ended. The common implementations of "smtp after pop/imap" simply put the IP adress of the requesting pop/imap client into a table (simply a berkeley dbfile), immediately after the POP/IMAP authentication. No matter how long the mail download takes, the user's IP is know to the smtp server from this point and thes smtp channel is kept open for at least 15-20 mins. It should be sufficient if smtp sending is just delayed for a few seconds after the pop/imap auth has started. 20-30 seconds should be enough.
Hmm, the timer approach would be a pain - you'd have to wait 20 or 30 seconds for each message send whereas it probably only takes a second or two to authenticate.
(In reply to comment #25) > David, I am not shure if we really need a serialized sequence where smtp only > starts after all pop/imap connections have ended. [snip] > authentication. No matter how long the mail download takes, the user's IP is > know to the smtp server from this point and thes smtp channel is kept open for > at least 15-20 mins. Yep, that works. When I get the "no relay" error, I hit retrieve and then immediately resend the email. So, they work asynchronously as long as POP has logged in. Of course, it is possible that the POP server could be slower to respond than the SMTP server (if they're different machines) which would result in a "no relay" situation again. I don't know if you can get an event when POP has successfully logged in; that would be best.
If we'd issue POP login just for authentication purposes we could add a two-liner in AuthFallback() right after m_nsIPop3Sink->SetUserAuthenticated(PR_TRUE); Something like if(justAuthenticate) m_pop3ConData->next_state = POP3_SEND_QUIT; But as I wrote earlier, the whole SMPT after POP is a piece of crap.
David; I am not an expert in pop3/imap4 protocols but as far as I understand, there is only one authentication at the beginning of a mail transmission (e.g. POP3 RFC1081: USER/PASS command from client answered by OK+ from the server meaning "maildrop locked and ready" and similar in RFC2060 for imap). So, the delay until the startup of SMTP send would be all the same, no matter how many messages are waiting for being fetched by client. Shure, the smtp after pop is "is a piece of crap" but popup blockers also occured only after people/companies have misused the "popping feature" of windows. Mozilla should give a hand in fighting that kind of misuse, I think ;-)
SMTP after POP is better than nothing. But people here talk if it were the only remedy for spam and one isn't a good user if he don't use it. SaP is more complex, needs more resources and is less secure than SMTP-Auth. I wonder why (normally) economically thinking companies use it.
Thomas, I was referring to this part of your comment: >It should be sufficient if smtp sending is just delayed for a few seconds after >the pop/imap auth has started. 20-30 seconds should be enough. I think it would be better to do the smtp send right after the pop/imap auth, which requires knowing when the pop/imap auth has completed. I don't know if it's better to do a full get new mail, or just do a user + pass - it seems like rather a waste to logon to the pop3 server and not do anything...I'm starting to think it might be better to only do this if the smtp send failed - is there a well-known error code (not string, but error code) returned in this case?
The common message is: "550 Relaying denied, authenticate with POP3 or IMAP4 first!" RFC1893 states in chapter "3.8 Security or Policy Status" ---------------------- [...] X.7.1 Delivery not authorized, message refused The sender is not authorized to send to the destination. This can be the result of per-host or per-recipient filtering. This memo does not discuss the merits of any such filtering, but provides a mechanism to report such. This is useful only as a permanent error. ---------------------- So, the "550" is a general message for any kind of unauthorized access to smtp. It is not really specific for the "smtp after pop/imap" case. And there is no way how one could check the "smtp after pop/imap" capability beforehand (like the CAPABILITY command in IMAP). I will check with some other mail clients how they implemented this in detail ...
OK, the results of my little research were not very successful: none of the common free clients I checked (kmail, mutt, sylpheed) implements the "smtp after pop" now. The only client I could find was EUDORA: http://www.active-venture.com/support/cp/email-client-eudora.htm#pop describes it from the user side. But this is "closed-source", no chance to grab the code ... Maybe its useful to contact the sylpheed guys: they mention "smtp after pop" as a ToDo feature (see http://sylpheed.good-day.net/#todo). They should have made some suggestions on this ...
Chris, in Comment #28 you write... > But as I wrote earlier, the whole SMPT after POP is a piece of ****. Full ack on this, but what should users do who are facing that **** and have to live with it? ... and in Comment #30 > SaP is more complex, needs more resources and is less secure than SMTP-Auth. I > wonder why (normally) economically thinking companies use it. As far as I could see, large scale providers decided to use it to share auth information across multiple smtp relays. Oppinion was, that it takes less time to fetch auth information from a DB instead of having to deal with smtp auth for each request. (?) In comment #33 Thommie writes, that only Eudora supports SaP. The Bat! and Pegasus also support it, as visible on http://www.activatormail.com/smtpauth.htm Some providers offer SMTP-Auth and/or SaP. Some (large ones) stay running SaP only,as by doing so, they also support (bad) SMTP-Auth-unaware mail readers... In a few years, this feature may have become useless --- but... the original bug was filed 2001, so... Thanks for re-engaging on this topic.
OK, I talked to few mail-server admins and user-support people: The best solution would be a simple user interface where people can: a) decide for a smtp server that "pop/imap before smtp" is needed (simple yes/no decision with "no" as default) b) a menu where one can select the pop/imap server which needs to be contacted for authentication (in most cases, but not in all, this will be the same DNS name for smtp and imap/pop) c) a two digit field where people can select the "delay time" between the successful pop/imap auth and the start point of the smtp send command. A default value of some seconds (5-10) should be sufficient in most cases. http://www.faqs.org/rfcs/rfc1939.html shows the typical POP3 case. It is NOT necessary to grab all waiting messages first! The POP3 login can be reduced to an absolute minimum, just for authentication: 1) INIT S: +OK POP3 server ready 2) AUTHORIZATION state. C: USER mrose S: +OK mrose is a real hoopy frood C: PASS secret S: +OK mrose's maildrop has 2 messages (320 octets) -> At this point, "SMTP send" could be triggered, with an optional/default "delay time" as stated in c) 3) QUIT Command C: QUIT S: +OK dewey POP3 server signing off A similar approach could be done for IMAP. The idea of using the error code from the smtp server (550) and triggering "smtp after pop" automagically, is not really good: The error message is not standardized enough and, moreover, this could be a security threat: people could automatically grab POP/IMAP login passwords with a "fake pop/imap server". Therefore, users should always have full control whether they want to send their password for "smtp after pop" authentication or not.
I am in this same boat with Mozilla 1.6. My web hosting service proclaims it to be a known issue with work arounds for Outlook/Outlook Express. Here's my hosting service page: http://kb.discountasp.net/article.aspx?id=10006 Here's the Microsoft page: http://support.microsoft.com/default.aspx?scid=kb;en-us;289945 Any and all help would be greatly appreciated.
Still no change in 1.7 and in Thunderbird 0.7. It's a real trouble! It's no problem for me to check mail before sending, but I can't make ordinary users do it every time...
Still no change in 1.7 and in Thunderbird 0.7. It's a real trouble! It's no problem for me to check mail before sending, but I can't make ordinary users do it every time...
Product: MailNews → Core
*** Bug 271708 has been marked as a duplicate of this bug. ***
Summary: Check POP/IMAP before SMTP send → Check POP/IMAP before SMTP send (SMTP-after-POP)
If the server remembers the IP number that checked POP3/IMAP for 15-20 minutes as we see in comment #25 than all the backend code we need is simply making sure that email is checked every 10 minutes. If the user switches on th "Check POP before SMTP" UI, an ugly pop-up window (remember it is an ugly authentication method, after all) saying "The browser changes your preferences to automatically check your mail every 10 minutes" would be helpful addition.
Aloha David, This bug has become a major issue since the release of TB1.0 When will it be solved? Instead of doing proper SMTP authentication (no idea why ISP's don't implement this in a more widespread fashion, all they have to do is enter a few more lines in their "how to set up your mail client" sections), it seems this kludge is being used. Basically, once you do an IMAP login, somehow the IP address is kept in some sort of stack so that you can send SMTP through their SMTP servers. But if you have not logged in to IMAP (no idea if you need an active connection, or if it's just login, that times out after a specified period), then the SMTP server will reject your IP as un-authenticated. I agree with #28 From Christian Eyrich of course the workaround "is ****" but not having it; results in: http://forums.mozillazine.org/viewtopic.php?p=1109867 About 2000 views within 3 weeks.
*** Bug 280361 has been marked as a duplicate of this bug. ***
It seems to be an ignorant world around user friendly SMTP solutions. This bug exists for more than 3 years now. And there is another major Bug around SMTP in Mozilla / Thunderbird: The SMTP GUI (thanks to ch.ey for his work on this). There is much time spent in functions like "spell as you type", so users have mails with correct spelling, but they are not able to send the mails... In Germany we would say "Ohne Worte".
The idea to check Mails automatically every xx minutes (see comment #40) would be a not good idea for users, which are not online all the time. A Login into the related POP account just before sending should be the smartest solution. Maybe we are able to check if there is / was a successfull login the last minutes, to avoid to much (unnecessary) logins?
We run our own Exim server. I was just looking at some packets with Etherpeek, and got the following: Line 1 : -ERR Unknown AUTHORIZATION state command <CR> <LF> .. 100 this is followed by: TCP Checksum: 0x4877 Checksum invalid. Should be: 0x0724 Any idea if this is the right bug? Mail seems slow, but does come through. Thanks.
FYI. Many ISP's started "Outbound Port 25 Blocking" recently, then lazy administrators of SMTP server should enable "Mail Submission Port", thus they have to enable SMTP AUTH in order to support "Mail Submission Port". This will certainly reduce number of lazy SMTP servers who force "POP before SMTP" without enabbling SMTP AUTH support.
To David Bienvenu(person who is set in Assigned To: currently): Why still "not WONTFIX"? (See Bug 228198 Comment #18) Is there any reason to keep this bug open? Is there any plan to implement "POP before SMTP" for lazy administrators of SMTP server?
Product: Core → MailNews Core
(In reply to comment #47) > To David Bienvenu(person who is set in Assigned To: currently): > > Why still "not WONTFIX"? (See Bug 228198 Comment #18) > Is there any reason to keep this bug open? Is there any plan to implement "POP > before SMTP" for lazy administrators of SMTP server? bienvenu ping. and resetting QA
QA Contact: esther → networking.smtp
I'm conflicted about this - it's a silly way for authentication to work, but it exists in the real world, and it' snot hard to implement on the client, and other clients to it, as I understand it.
"SMTP after POP" was dying even when this bug was filed 8 years ago. Almost all (serious) ISPs don't do it anymore (AFAICT, etc.), not even as a fallback. So while this may be nice-to-have if someone comes up with a patch, we shouldn't waste precious development time on it, IMO. ;-)
Assignee: bienvenu → nobody
I agree with Karsten, It is no longer an issue in the times of "normal" smtp authentication methods. This should be closed with status "wontfix"
taking a cue from comment 51 => wontfix
Severity: major → normal
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Whiteboard: [parity-becky]
You need to log in before you can comment on or make changes to this bug.