Open Bug 556655 Opened 10 years ago Updated 1 year ago

Move the 1500 Japanese .jp domains (for just 2 ISPs biglobe.ne.jp and plala.or.jp) to autoconfig server

Categories

(Webtools :: ISPDB Database Entries, defect)

defect
Not set

Tracking

(Not tracked)

People

(Reporter: BenB, Assigned: u217243)

References

Details

Attachments

(2 files, 1 obsolete file)

See bug 524045 comment 82:

Ben Bucksch wrote:

Biglobe now has 1400 (!) domains in our database.
I think that is unreasonable.

Many of these are even vanity domains like "cancer.", "milk.". Some German
hosters also have vanity domains like "want-to-wake-up-with-you.de" (in
German), "gotaquestion.de" and stuff like this. Total nonsense. Almost nobody
is using them. I intentionally did not add them.
... cost for us, like the slowness that all svn operations now take. Nevermind the inability to browse the list of domains anymore.

Kohei Yoshino wrote:

Certainly 1,400 domains are too many, but the solutions -- dynamic ISPDB nor
fetchConfigFromISP -- have not been implemented at this time, unfortunately. I
believe hosting static XML files on the live.momo server is temporary.

----

> the solutions -- ... fetchConfigFromISP -- have not been implemented at
> this time

It is now implemented.

Preliminary documentation at https://developer.mozilla.org/en/Thunderbird/Autoconfiguration_in_Thunderbird#Configuring_a_server_at_an_ISP

> static XML files [for the many domains] on the live.momo server is temporary.

Given that we now moved to a new directory v1.1, and I need to manually update the XML files, I think this is a good time to remove the many duplicated .jp domains, at least from the new v1.1 directory.

I am speaking of the following:
*gunmanet*.jp
*wind*.jp
*plala*.jp
*janis*.jp
*tiki*.jp
*eeyo.jp *memail.jp *biglobe.jp

That's 6 ISPs and accounts for almost 1600 (!) domains.
eeyo.jp=memail.jp=biglobe.jp alone is 1400 domains and should be first priority.

(Without the above mentioned, we have ~150 domains remaining, of which ~50 are *.jp. That'd be fair.)
Summary: Move the many .jp domains to autoconfig → Move the 1600 Japanese .jp domains (for just 6 ISPs) to autoconfig
Blocks: 556729
With bug 557401 and generated configs, the situation changed a bit. The domains are now expanded on the server.

However, 2 ISPs are still a problem:

* biglobe.ne.jp, because it has 1400 domains. That's insane even for
  generating them.
  I checked the logs of 5 days and memail.jp and eeyo.jo are not used at all,
  they didn't have a single request.
  *.biglobe.ne.jp (627 domains) made up 0,15% of the requests.
* plala.or.jp, because it uses IMAP server sky.mail.plala.or.jp for
  email domain @sky.plala.or.jp , so we cannot generate them.
  Plala made up 0,18% of our the requests.

Kohei, could you please contact these two ISPs to set up an autoconfig server? It's documented at <https://developer.mozilla.org/en/Thunderbird/Autoconfiguration>. Thanks.
Summary: Move the 1600 Japanese .jp domains (for just 6 ISPs) to autoconfig → Move the 1400 Japanese .jp domains (for just 2 ISPs biglobene.jp and plala.or.jp) to autoconfig server
Summary: Move the 1400 Japanese .jp domains (for just 2 ISPs biglobene.jp and plala.or.jp) to autoconfig server → Move the 1500 Japanese .jp domains (for just 2 ISPs biglobene.jp and plala.or.jp) to autoconfig server
Actually, I think we can use DNS MX for biglobe (in TB 3.1), so no need for an autoconfig server.

plala remains a problem. Kohei, can you check with them whether they really use different mail stores? If there's only one mail store, we could unite them all by using only one IMAP server hostname.
Attached patch Biglobe, v1 (obsolete) — Splinter Review
Biglobe has bgmgate1.biglobe.ne.jp for all 1400 domains, so we can rely on MX to resolve this, luckily.

I also noticed that the file is still in v1.0 format.

What's worse, I noticed that the config may not even work! It's using auth = secure for POP and SMTP. While the SMTP server says:
$ netcat mail.biglobe.ne.jp 587
220 mail.biglobe.ne.jp SMTP-Gateway Version 2.12 at Wed, 18 Aug 2010 05:39:18 +0900 (JST)
EHLO abc
250-mail.biglobe.ne.jp Hello abc [IP], pleased to meet you
250-AUTH CRAM-MD5 LOGIN PLAIN
the POP server says:
$ netcat mail.biglobe.ne.jp 110
+OK mail.biglobe.ne.jp POP Gateway starting. <6699.1170.1282077702@mail.biglobe.ne.jp>
CAPA
-ERR Unknown command: "capa".
AUTH
-ERR Unknown command: "auth".
AUTH CRAM-MD5 fdghfgj
-ERR Unknown command: "auth".

So, it seems the POP server doesn't support CRAM-MD5, meaning the config is broken!

Kohei, can you please test? (see Thunderbird Account Settings | Server | Authentication, or better yet test with the config file.)
Attachment #466789 - Flags: review?(bwinton)
Attachment #466789 - Flags: review?(gozer)
Attachment #466789 - Flags: feedback?(kohei.yoshino.bugs)
Attachment #466792 - Flags: review?(bwinton)
Attachment #466792 - Flags: feedback?(kohei.yoshino.bugs)
Attachment #466789 - Flags: review?(gozer) → review+
Comment on attachment 466789 [details] [diff] [review]
Biglobe, v1

>+++ biglobe.ne.jp	(Arbeitskopie)
>@@ -1,1423 +1,23 @@
> <?xml version="1.0" encoding="UTF-8"?>
> <clientConfig>

You'll need to add 'version="1.1"' to this element, since no version is defined as being version 1.0.

>   <emailProvider id="biglobe.ne.jp">
[snip…]
>-    <domain>bcs.biglobe.ne.jp</domain>

This one lists mailx10.mesh.ad.jp as its MX host, not bgmgate1.biglobe.ne.jp, so I think we should leave it in this file.

>-    <domain>test.test.biglobe.ne.jp</domain>

And this one doesn't list any MX hosts, but I think I'm okay deleting it.  ;)

>+    <!-- more than 1400 domains, all with bgmgate1.biglobe.ne.jp as MX -->

Also, there were exactly 1400 domains, by my count.  :)

>     <incomingServer type="pop3">
>       <hostname>mail.biglobe.ne.jp</hostname>
>       <port>110</port>
>       <socketType>plain</socketType>
>       <username>%EMAILADDRESS%</username>
>-      <authentication>secure</authentication>
>+      <authentication>password-cleartext</authentication>

So, passwords _really_ go over cleartext on an unsecured socket?  Really?

r=me, with the changes listed above, and Kohei's feedback.

I'm a little hesitant to make this change, since it will prevent Thunderbird 3.0 users from configuring these domains, but since there hasn't been a Thunderbird 3.0 new account request in the top 70 misses since the 17th, I think we're probably pretty safe to make this change.

Later,
Blake.
Attachment #466789 - Flags: review?(bwinton) → review+
> This one lists mailx10.mesh.ad.jp as its MX host, not bgmgate1.biglobe.ne.jp,
> so I think we should leave it in this file.

I've not seen a single hit to this domain nor MX server in the log sample I have, so I think it's neglectable.
It probably needs a different POP3 server anyway, given that it goes to a whole different domain. I don't think it worked with the current config anyway.

> So, passwords _really_ go over cleartext on an unsecured socket?  Really?

Kohei?
Comment on attachment 466792 [details] [diff] [review]
Janis, v1 -- done

>+++ janis.or.jp	(Arbeitskopie)
>@@ -1,44 +1,23 @@
> <?xml version="1.0" encoding="UTF-8"?>
> <clientConfig>

'version="1.1"' please.

Other than that, I like it.

Later,
Blake.
Attachment #466792 - Flags: review?(bwinton) → review+
Commited janis as SVN Revision 73035 (sorry, thought that gozer had reviewed, as he did the other one)
> You'll need to add 'version="1.1"' to this element, since no version
> is defined as being version 1.0.

fixed.

> This one lists mailx10.mesh.ad.jp as its MX host, not bgmgate1.biglobe.ne.jp,
> so I think we should leave it in this file.

As discussed, I saw no hits on that in several days.
Also, it probably will *not* work with this config.
So, leaving it out.

> >+    <!-- more than 1400 domains, all with bgmgate1.biglobe.ne.jp as MX -->

> Also, there were exactly 1400 domains, by my count.  :)

Fixed.
Attachment #466789 - Attachment is obsolete: true
Attachment #469290 - Flags: review?(bwinton)
Attachment #466789 - Flags: feedback?(kohei.yoshino.bugs)
Comment on attachment 469290 [details] [diff] [review]
biglobe, v2 -- done

carrying over gozer's r+ for the previous patch.
Attachment #469290 - Flags: review+
Comment on attachment 469290 [details] [diff] [review]
biglobe, v2 -- done

You didn't actually need to get a re-review from me on this one, but since you asked, r=me again.  ;)

Later,
Blake.
Attachment #469290 - Flags: review?(bwinton) → review+
Comment on attachment 469290 [details] [diff] [review]
biglobe, v2 -- done

Thanks.
Commited as SVN Revision 73037
Leaves plala, which is massively annoying, and wind.
Summary: Move the 1500 Japanese .jp domains (for just 2 ISPs biglobene.jp and plala.or.jp) to autoconfig server → Move the 1500 Japanese .jp domains (for just 2 ISPs biglobe.ne.jp and plala.or.jp) to autoconfig server
Attachment #466792 - Attachment description: Janis, v1 → Janis, v1 -- done
Attachment #466792 - Flags: feedback?(kohei.yoshino.bugs)
Attachment #469290 - Attachment description: biglobe, v2 → biglobe, v2 -- done
Blocks: 594665
Component: ispdb → ISPDB Server
Product: Mozilla Messaging → Webtools
I'm a plala.or.jp user.
And I have just noticed that https://autoconfig.thunderbird.net/v1.1/lapis.plala.or.jp is a little outdated.
Now all the plala users share the config.
See https://www.plala.or.jp/support/manual/mail/plala_mail/standard/setup_imap/ (Japanese)

username: %EMAILADDRESS% (not LOCALPART)
incomingserver: (IMAP4) imap.plala.or.jp (port: 993 SSL)
outgoingserver: secure.plala.or.jp (port: 587 TLS or 465 SSL)
Thanks, Tamo!
Component: ISPDB Server → ISPDB Database Entries
You need to log in before you can comment on or make changes to this bug.