Closed Bug 544644 Opened 11 years ago Closed 11 years ago

Add Telenor Danmark to Thunderbird autoconfiguration

Categories

(Webtools :: ISPDB Server, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bugzilla, Unassigned)

References

Details

Attachments

(1 file, 10 obsolete files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; da; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)
Build Identifier: 

Please add Telenor Danmark to the Thunderbird autoconfiguration list.

I will attach configurations shortly.

The source of the information is (page is in Danish):
http://www.telenor.dk/privat/kundeservice/kundeservice_bredbaand/teknisk_hjaelp/email/opsaetning_af_email/outlook_express_6_0/index.aspx

The source of the domain list is http://mailselvbetjening.telenor.dk (requires login)

I made some changes in the XML files compared to http://ispdb.mozillamessaging.com, because: It was not possible to specify a user name for the incoming server, so I mistakenly put that in the field for the outgoing server. It was not possible to specify port number for the outgoing server, and the post chosen by the site was wrong.

I have two configurations. One for domains where the username can be extracted from the email address, and one for domains where it cannot. I am not sure how to represent that in the last case, and I have written it like this: <username>Example: dsl12345</username>.

Reproducible: Always
Attached file Telenor Danmark (main addresses) (obsolete) —
Attached file Telenor Danmark (alias addresses) (obsolete) —
Jesper can you provide a diff  against the live momo file (I'll find the url, if you don't find it.) ?
Which file?
(In reply to comment #4)
> Which file?

No need to. My bad.
Attachment #425632 - Flags: review?(bwinton)
Comment on attachment 425632 [details]
Telenor Danmark (alias addresses)

> <?xml version='1.0' encoding='UTF-8'?>
> <clientConfig version="3.0">
>   <emailProvider id="">
>    <domain>email.dk</domain>

We need an id here.  I suggest "email.dk".

And we need one file per domain.

>     <incomingServer type="pop3">
>       <hostname>mail.telenor.dk</hostname>
>       <port>110</port>
>       <socketType>plain</socketType>
>       <authentication>plain</authentication>

No security?  There's no way for people to check their mail without sending their username and password in plain text over the internet?

>       <username>Example: dsl12345</username>

I don't think this is going to do the right thing.
What if you made that "%EMAILLOCALPART%", and told people to put in "ds112345@email.dk" as their email address?

>     <outgoingServer type="smtp">
>       <addThisServer>false</addThisServer>

We don't want to add this server?  That seems a little strange.

Thanks,
Blake.
Attachment #425632 - Flags: review?(bwinton) → review-
(In reply to comment #6)
> (From update of attachment 425632 [details])
> > <?xml version='1.0' encoding='UTF-8'?>
> > <clientConfig version="3.0">
> >   <emailProvider id="">
> >    <domain>email.dk</domain>
> 
> We need an id here.  I suggest "email.dk".

I would prefer something containing mail.telenor.dk, as that is the domain these aliasses forward messages to, but I guess the ID cannot be the same as for the other file? Does the ID have to be a domain name?

> 
> And we need one file per domain.

Sure, but does they have to have different content?

> 
> >     <incomingServer type="pop3">
> >       <hostname>mail.telenor.dk</hostname>
> >       <port>110</port>
> >       <socketType>plain</socketType>
> >       <authentication>plain</authentication>
> 
> No security?  There's no way for people to check their mail without sending
> their username and password in plain text over the internet?

Yes! Telenor is no different than most other internet providers in this regard, sadly.

> 
> >       <username>Example: dsl12345</username>
> 
> I don't think this is going to do the right thing.

But what would do the right thing, then? I cannot find a complete documentation of the XML format anywhere.

> What if you made that "%EMAILLOCALPART%", and told people to put in
> "ds112345@email.dk" as their email address?

No. How would you "tell" people to enter a specific address in the autoconfig dialog? Besides, wouldn't entering such a non-existent address make it impossibly to properly reply to emails you send?

It works like this: The user can create an email inbox on the form dsl12345@mail.telenor.dk. They can then create a number of aliasses for that mailbox as somename@somedomain.dk, and that alias is then the mail address the user remembers and is most likely to enter in the account config dialog. The username must then be the localpart of the mailbox to which the alias maps, but there is no way to retrieve that automatically.

> 
> >     <outgoingServer type="smtp">
> >       <addThisServer>false</addThisServer>
> 
> We don't want to add this server?  That seems a little strange.

That is what the ispdb website generated. If that is strange, then you should fix it.

> 
> Thanks,
> Blake.
Attachment #425631 - Attachment is obsolete: true
Attachment #428513 - Flags: review?(bwinton)
Attachment #425632 - Attachment is obsolete: true
Attachment #428514 - Flags: review?(bwinton)
(In reply to comment #7)
> > > <?xml version='1.0' encoding='UTF-8'?>
> > > <clientConfig version="3.0">
> > >   <emailProvider id="">
> > >    <domain>email.dk</domain>
> > We need an id here.  I suggest "email.dk".
> I would prefer something containing mail.telenor.dk, as that is the domain
> these aliases forward messages to, but I guess the ID cannot be the same as
> for the other file? Does the ID have to be a domain name?

I don't believe the id can be the same as the other file, no.  It doesn't have to be a domain name, but it usually is, and it's usually one of the domain names in the list of domains.

> > And we need one file per domain.
> Sure, but does they have to have different content?

No, but if we only have one file, then when someone goes to check it in, they will forget to check in one of the copies of the file.  (It's happened before, to an experienced autoconfig developer, and I expect it'll happen again.)

> > >       <socketType>plain</socketType>
> > >       <authentication>plain</authentication>
> > No security?  There's no way for people to check their mail without sending
> > their username and password in plain text over the internet?
> Yes! Telenor is no different than most other internet providers in this
> regard, sadly.

:(

> > >       <username>Example: dsl12345</username>
> > I don't think this is going to do the right thing.
> But what would do the right thing, then? I cannot find a complete
> documentation of the XML format anywhere.

The most complete documentation we have is:
https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

I don't know if anything you put there would work, because if you have a config, then we assume you've entered all the settings, and we don't allow the user to edit them.  We do have the idea of "User Input Fields", but haven't implemented them yet.

Ben, do you have any ideas of what we can do while we wait for User Input Fields?

> > >     <outgoingServer type="smtp">
> > >       <addThisServer>false</addThisServer>
> > We don't want to add this server?  That seems a little strange.
> That is what the ispdb website generated. If that is strange, then you should
> fix it.

That sounds reasonable.  Please file a bug, and feel free to assign it to me.

Thank you,
Blake.
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

(I'm going to clear out the review flag until we can get some of the questions answered.)
Attachment #428513 - Flags: review?(bwinton)
Attachment #428514 - Flags: review?(bwinton)
(In reply to comment #10)
> > > And we need one file per domain.
> > Sure, but does they have to have different content?
> 
> No, but if we only have one file, then when someone goes to check it in, they
> will forget to check in one of the copies of the file.  (It's happened before,
> to an experienced autoconfig developer, and I expect it'll happen again.)

I will make a patch once the content of the file is reviewed.

> 
> > > >       <socketType>plain</socketType>
> > > >       <authentication>plain</authentication>
> > > No security?  There's no way for people to check their mail without sending
> > > their username and password in plain text over the internet?
> > Yes! Telenor is no different than most other internet providers in this
> > regard, sadly.
> 
> :(

Well, i sort of agree. But I don't think the risk is so great. The network is fully controlled by the company controlling the mail server, so if you don't trust the network, you wouldn't trust the mail server. There is of course the possibility of someone digging up the road and wiretapping the copper cable...

> 
> > > >       <username>Example: dsl12345</username>
> > > I don't think this is going to do the right thing.
> > But what would do the right thing, then? I cannot find a complete
> > documentation of the XML format anywhere.
> 
> The most complete documentation we have is:
> https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

But that is only a problem for the alias addresses. Is there still something wrong with the main addresses configuration?

> > > >     <outgoingServer type="smtp">
> > > >       <addThisServer>false</addThisServer>
> > > We don't want to add this server?  That seems a little strange.
> > That is what the ispdb website generated. If that is strange, then you should
> > fix it.
> 
> That sounds reasonable.  Please file a bug, and feel free to assign it to me.

Will do.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

> But that is only a problem for the alias addresses. Is there still something
> wrong with the main addresses configuration?

Yeah, that's fair enough.

It passes the sanity checks, so I'll give it the first r+.

And since I've already asked Ben for input, I suggest you ask him for the second review.

Thanks,
Blake.
Attachment #428513 - Flags: review+
The ID should be a domain that belongs to the ISP, and is the "primary" domain.
In this case, I think "telenor.dk" would be appropriate as ID, because it's more specific than "email.dk".

The ID must be unique per configuration. If any of the config values differs, the ID must differ.
(We can and do have many files with the exact same content, though, see e.g. hotmail.com and live.com.)

> The network is fully controlled by the company controlling the mail
> server, so if you don't trust the network, you wouldn't trust the mail server.

That's easy to think, but not true. The configuration must also work when the user is connected to a different network, e.g. hotel, internet cafe, a friend etc..

> >       <username>Example: dsl12345</username>

> I don't know if anything you put there would work, because if you have a
> config, then we assume you've entered all the settings, and we don't allow the
> user to edit them.  We do have the idea of "User Input Fields", but haven't
> implemented them yet.

That is what Custom Fields were designed for, including a ISP-specific synonym for the username and an example. Unfortunately, Bryan shut them down for now.

No, there is no workaround. We need the Custom Fields implementation.
(Note that a generic field "username" would not be sufficient, because the user would wonder what the "username" is, given that there are likely several candidates from the user's perspective including the EMAILLOCALPART.)
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

> that alias is then the mail address the
user remembers and
> is most likely to enter in the account config dialog.
> The username must then be the localpart of the mailbox to which the
> alias maps, but there is no way to retrieve that automatically.

OK, given that, the config is wrong. The user enters someone@email.dk, we'd use "someone" as incomging username, and it won't work. Even less so for the outgoing, where we either use an empty string or outright reject the config.

I fear that we currently can't configure these addresses with the ISP DB at all.

We need either
1) the Custom Fields or
2) the ISP installs an autoconfig server which returns the correct username. (unlikely and potentially problematic due to information exposure)
Attachment #428513 - Flags: review-
(In reply to comment #15)
> (From update of attachment 428513 [details])
> > that alias is then the mail address the
> user remembers and
> > is most likely to enter in the account config dialog.
> > The username must then be the localpart of the mailbox to which the
> > alias maps, but there is no way to retrieve that automatically.
> 
> OK, given that, the config is wrong. The user enters someone@email.dk, we'd use
> "someone" as incomging username, and it won't work. Even less so for the
> outgoing, where we either use an empty string or outright reject the config.

You quoted text related to the alias configuration, but you put review- on the main configuration. Didn't you put the review flag on the wrong attachment?

So if I use mail.telenor.dk as ID for the main config and telenor.dk for the alias config, would that be ok?
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

Oh, so main addresses are always from one domain, and aliases are *always* from the other domains, never from the domain where the main addresses are? That's practical! :)
Attachment #428513 - Flags: review-
Attachment #428514 - Attachment description: Telenor Danmark (alias addresses), v2 → Telenor Danmark (alias addresses), v2 - can't support it yet, needs Custom Fields
Attachment #428514 - Flags: review-
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

Are you entirely sure that he ISP offers no config which allows the user to get his mail (per POP) when he's not at home, but e.g. at work? Do not only rely on official documentation. Please contact the ISP and try hard to find such a config.

I don't want to configure setups that suddenly break (or are insecure) when the user takes his notebook elsewhere.
Comment on attachment 428513 [details]
Telenor Danmark (main addresses), v2 - bad config

OK, so I probed mail.telenor.dk manually, on port 110, 143, 25 and 587, and the SSL ports.

This server supports *all* of POP3, IMAP (!) and SMTP, with auth (!). In fact, it openly advertizes STARTTLS (and DIGEST-MD5 auth) on all of them! And they work.

Nevertheless, you offered the worst possible config here.

Jesper, bwinton specifically asked you for that, and you denied that in comment 7.
Please see https://wiki.mozilla.org/MailServerList .

bwinton, we need to verify such claims when reviewing, and I have a bug to automate that.
Attachment #428513 - Attachment description: Telenor Danmark (main addresses), v2 → Telenor Danmark (main addresses), v2 - bad config
Attachment #428513 - Flags: review-
I explicitly tried SSL and TLS for incoming mails when I filed the bug (It is a couple of years since I tested IMAP), and I know from our support forum that some people using Telenor have had problems sending mails through this provider after the upgrade from TB2 to TB3 turned on SMTP-auth, which is a known bug in TB3. One of them is here: http://forum.mozilladanmark.dk/viewtopic.php?p=36304#p36304

Either they have changed practice since I filed the bug, or your check is not complete. Unfortunately I cannot test, because I have changed internet provider from Telenor to TDC.
CRAM-MD5 (secure auth) indeed is not offered. (DIGEST-MD5 is, but we don't support that.)
SSL on SSL ports doesn't seem to work either.
STARTTLS, however, does work.
Oh, I am stupid. It turns out that if I disable Mail Scanner in my Avast Antivirus, IMAP+STARTTLS suddenly works.
Attached file Telenor Danmark (main addresses), v3 (obsolete) —
Attachment #428513 - Attachment is obsolete: true
Comment on attachment 436921 [details]
Telenor Danmark (main addresses), v3

I now got confirmation that this secure version also works from within Telenor's network.
Attachment #436921 - Flags: review?(ben.bucksch)
Comment on attachment 436921 [details]
Telenor Danmark (main addresses), v3

> It turns out that if I disable Mail Scanner in my Avast
> Antivirus, IMAP+STARTTLS suddenly works.

Urgs. Dump it :) At least that explains it.

Config looks fine now.
Please add 2 things:
1. POP server in an XML comment.
2. Webpages from the ISP describing the config (even if it's different from what we chose).
Attached file Telenor Danmark (main addresses), v4 (obsolete) —
Do you mean like this?
Attachment #436921 - Attachment is obsolete: true
Attachment #436953 - Flags: review?(ben.bucksch)
Attachment #436921 - Flags: review?(ben.bucksch)
Further testing shows that you may use the full e-mail address as username on both the main and the alias domains. This means that the config files can be combined into a single file.

I have also added a commented POP3 incomingServer as requested by Ben.
Now with links to the ISP in comments (from Jesper's version)
Attachment #436954 - Attachment is obsolete: true
> Further testing shows that you may use the full e-mail address as username on
> both the main and the alias domains

Great! Thanks!

Jesper, can you double-check that this works for you, too?
Comment on attachment 436956 [details]
Telenor Danmark (combined) with full mail address as username, v2

That's exactly what I meant, yes.

Can you please remove the nonsense domains like trashcan, trustme and testmail (gah!) and add only those which have the most users? Comment out the others.
Oh, and port 587 for SMTP
Attachment #436953 - Attachment is obsolete: true
Attachment #436953 - Flags: review?(ben.bucksch)
Attachment #436922 - Attachment is obsolete: true
Attached file combined, nonsense domains removed (obsolete) —
How about this?
If you know 3 or so domains which are widely used, we can add them, too.
Attachment #436956 - Attachment is obsolete: true
Attachment #436966 - Attachment mime type: application/octet-stream → text/xml
Yes, full email address works. I know TDC started allowing it a few months ago. Maybe Telenor followed them.

I don't think a configuration with most domains removed will be much useful.

I only know from our support forum that the biggest Danish provider (of those causing trouble for users) is TDC, followed by Telenor and Telia and then followed by the energy companies. I think usually people choose a personal address, but I don't know, as people usually don't tell us their email address, only their provider's name when they ask for help.

Maybe Mozilla Messaging has some stats from their servers? I don't know if you have enough data to say anything as Denmark is a small country, but if you could tell the top domains used in Denmark, that would be great. That might also allow us to update our manual reference of email providers in Denmark, if the list contains providers which we don't know about: http://wiki.mozilladanmark.dk/wiki/Thunderbird/Kontoindstillinger_for_diverse_udbydere
I agree with Jesper. We have no way of knowing which e-mail aliases the users have decided to use. The web interface for creating the aliases allows you to select freely between all the domains. I would find it a bit hard to decide which ones to leave out unless we have some stats to back it up.
I currently have to manually add and maintain every single domain as separate file. There's no way I'm adding 100 domains for one ISP.

Esp. with stuff like "trashcan", "testmail" and "popme" in there.

Ask the ISP which ones are most popular. We can add the most popular 5. Rule of thumb: Everything with less than 10000 users is omitted.

Alternatively, advocate the ISP to run an autoconfig server, see <https://developer.mozilla.org/en/Thunderbird/Autoconfiguration>.

We have stats, but I don't have access to the full stats, only our admin I think.

Thanks, Jesper, for testing.
Out of curiosity, have you considered adding a single file and a bunch of symlinks for all of the alias domains? That way you only have to change a single file in case of updates.

Regarding the 100 domains, looking at https://live.mozillamessaging.com/autoconfig/ it seem that you have many more than that for the biglobe.ne.jp provider. Has the police changed since that was added?
> have you considered ... symlinks

It's managed by Subversion, and that doesn't know symlinks, to my knowledge.

> you have many more than that for the biglobe.ne.jp provider.

Yes, and it's a big problem for me, and I want them removed. See bug 556655.
And I'm not maintaining them at all.
Hmm…  In bug 556140, we talked about having a script that would create the v1.0 files from the v1.1 files.  Perhaps it would be worth it to extend that script to create the v1.1 sub-domains from the v1.1 master domain.  (Uh, please comment over in that other bug if you think that's a good/bad idea.)

Thanks,
Blake.
> Perhaps it would be worth it to extend that script
> to create the v1.1 sub-domains from the v1.1 master domain.

Great idea! Filed bug 557401.
Attachment #436966 - Flags: review+
Attachment #436966 - Flags: review?(bwinton)
(In reply to comment #31)
> Can you please remove the nonsense domains like trashcan, trustme and testmail
> (gah!) and add only those which have the most users? Comment out the others.

I guess that with bug 557401 it would then be possible to add Telenor. Is that correct?
Depends on: 557401
> I guess that with bug 557401 it would then be possible to add Telenor.

We can add it now, as is, with just a few domains (see file). I already approved it, we just need one more review.

Whether we'll add all domains after bug 557401, we'll have to decide, I don't know the policy yet.
I don't think that "trashcan.dk" and "popme.dk" and "I-want-to-wake-up-with-you.de" should be added even then, but that's a matter of policy we'll decide when we get there.
Comment on attachment 436966 [details]
combined, nonsense domains removed

><?xml version='1.0' encoding='UTF-8'?>
><clientConfig version="3.0">

I think this should be removed/made into "1.0", but see below.

>  <emailProvider id="mail.telenor.dk">
>    <!-- Main domains -->
>    <domain>mail.telenor.dk</domain>
>    <domain>vip.cybercity.dk</domain>
>    <domain>post.cybercity.dk</domain>
>
>    <!-- Alias domains -->
>    <domain>email.dk</domain>
>    <!--
>    <domain>mobil.dk</domain>
>    <domain>privatmail.dk</domain>

I think we can comment these all in, so that the script will create them for us.

>
>    <displayName>Telenor Danmark</displayName>
>    <displayShortName>Telenor</displayShortName>
>    <incomingServer type="imap">
>      <hostname>mail.telenor.dk</hostname>
>      <port>143</port>
>      <socketType>STARTTLS</socketType>
>      <username>%EMAILADDRESS%</username>
>      <authentication>plain</authentication>
>    </incomingServer>
>    <!--
>    <incomingServer type="pop3">
>      <hostname>mail.telenor.dk</hostname>
>      <port>110</port>
>      <socketType>STARTTLS</socketType>
>      <username>%EMAILADDRESS%</username>
>      <authentication>plain</authentication>
>    </incomingServer>
>    -->

So, we could make this a 1.1 config, change the authentication, and uncomment the pop3 server.  What do you think of that?

>    <outgoingServer type="smtp">
>      <hostname>mail.telenor.dk</hostname>
>      <port>587</port>
>      <socketType>STARTTLS</socketType>
>      <username>%EMAILADDRESS%</username>
>      <authentication>plain</authentication>
>      <addThisServer>true</addThisServer>
>      <useGlobalPreferredServer>false</useGlobalPreferredServer>
>    </outgoingServer>
>  </emailProvider>
>  <!--
>    Configuration source: http://www.telenor.dk/privat/kundeservice/kundeservice/bredbaand/opsaetning/email-og-hjemmeside/tabs/opsaetning/outlook-express-6.0.aspx
>     http://www.telenor.dk/privat/kundeservice/kundeservice/bredbaand/opsaetning/email-og-hjemmeside/tabs/opsaetning/windows-live-mail.aspx
>    Domain list source (requires login): http://mailselvbetjening.telenor.dk/
>  -->
></clientConfig>

(In reply to comment #42)
> > I guess that with bug 557401 it would then be possible to add Telenor.
> We can add it now, as is, with just a few domains (see file). I already
> approved it, we just need one more review.
> 
> Whether we'll add all domains after bug 557401, we'll have to decide, I don't
> know the policy yet.
> I don't think that "trashcan.dk" and "popme.dk" and
> "I-want-to-wake-up-with-you.de" should be added even then, but that's a matter
> of policy we'll decide when we get there.

Just to say it again, if we're using the script, I don't see any huge problems with adding the sub-domains.

Anyways, if you don't want to change it to a v1.1 config, then r=me if you change the version to 1.0.  If you do want to change it, then it should probably get a quick re-review.

Thanks,
Blake.
Attachment #436966 - Flags: review?(bwinton) → review+
Attached patch v1.1 config fileSplinter Review
Fixed review comments.

> change it to a v1.1 config, then ... get a quick re-review.

Done.
Attachment #436966 - Attachment is obsolete: true
Attachment #442752 - Flags: review?(bwinton)
Comment on attachment 442752 [details] [diff] [review]
v1.1 config file

We shouldn't have "Offensive domains".  Either they exist, and we should list them, or we are categorically opposed to them, and we should just drop them silently (and wait to see if they show up on the top-missing-domain list ;).

But the rest of the info seems fine, so, with those either added or removed, r=me.

Later,
Blake.
Attachment #442752 - Flags: review?(bwinton) → review+
OK, removed domains "silently" :)

Commited as SVN revision 66658.
FIXED

Thanks Mogens Isager for the useful config and resolving the problem we had!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
note: commited to v1.1 (TB 3.1) only.
Component: ispdb → ISPDB Server
Product: Mozilla Messaging → Webtools
You need to log in before you can comment on or make changes to this bug.