524045 - [config] Add Japanese ISP configurations

Status: RESOLVED FIXED

(Webtools :: ISPDB Database Entries, defect)

Description

[u217243]

Mozilla Japan is gathering email configurations directly from Japanese ISPs. We'll submit new configs then add the list here, so please approve them. (Or could you give us approval authority?)

Quick question for you: does the ISP database allow companies, universities and governments to add their configs? Many organizations would like to utilize the DB if anyone could use that.

Comment 1

[u217243]

How can I select STARTTLS for incoming/outgoing socket type, and secure for incoming/outgoing authentication? Also, the Language of the settings page list has only English and French...

Comment 2

[Ludovic Hirlimann [:Usul]]

(In reply to comment #1)
> How can I select STARTTLS for incoming/outgoing socket type, and secure for
> incoming/outgoing authentication? Also, the Language of the settings page list
> has only English and French...

Ispdb.mozillamessaging.com is not on production yet , it's there to gather feedback about it.

Comment 3

[David Ascher (:davida)]

We're not going to delegate authority, certainly not at this early stage.  I think at this point the only real way to get new files into the database would be to submit patches against http://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/

Although I would suggest using preview.ispdb, and then using the 'export as xml' function to then generate the patches.

Comment 4

[u217243]

(In reply to comment #3)
> We're not going to delegate authority, certainly not at this early stage.  I
> think at this point the only real way to get new files into the database would
> be to submit patches against
> http://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/

Understood. We'll make and attach patches...

Comment 5

[u217243]

Attachment: ISP configuration file: IIJ4U

Comment 6

[u217243]

Attachment: ISP configuration file: IIJmio

Comment 7

[u217243]

Attachment: ISP config files

We have collected some configurations. Could you commit these files to the SVN? This attachment (ZIP) contains 86 XML config files. More to come...

Comment 8

[David Ascher (:davida)]

It'd be good to run these configurations through our sanity-checking script.  I'm confident that Kohei did it right, so it'd be to test the script as much as the config files.

Comment 9

[u217243]

Once the configurations are added, the ISPs can test the account setup feature with a Release Candidate build...

Comment 10

[Blake Winton (:bwinton) (:☕️)]

Here are the results of the sanity checks, broken down into two different headings, "Differing Endings" and "Timeouts".

More commentary at the bottom.

------------------
Differing Endings:

bc.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bc.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bc.iij4u.or.jp."

bk.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bk.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bk.iij4u.or.jp."

bp.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bp.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bp.iij4u.or.jp."

bu.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bu.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bu.iij4u.or.jp."

dd.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with dd.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with dd.iij4u.or.jp."

e23.jp:
  "mbox.iijmio-mail.jp doesn’t end with e23.jp."
  "mbox.iijmio-mail.jp doesn’t end with e23.jp."

ff.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with ff.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with ff.iij4u.or.jp."

hh.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with hh.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with hh.iij4u.or.jp."

kk.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with kk.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with kk.iij4u.or.jp."

ma100.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with ma100.tiki.ne.jp."

miobox.jp:
  "mbox.iijmio-mail.jp doesn’t end with miobox.jp."
  "mbox.iijmio-mail.jp doesn’t end with miobox.jp."

miomio.jp:
  "mbox.iijmio-mail.jp doesn’t end with miomio.jp."
  "mbox.iijmio-mail.jp doesn’t end with miomio.jp."

mx1.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx1.tiki.ne.jp."

mx2.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.et.tiki.ne.jp."

mx2.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.tiki.ne.jp."

mx2.wt.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.wt.tiki.ne.jp."

mx21.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx21.tiki.ne.jp."

mx22.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx22.tiki.ne.jp."

mx3.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx3.et.tiki.ne.jp."

mx3.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx3.tiki.ne.jp."

mx31.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx31.tiki.ne.jp."

mx32.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx32.tiki.ne.jp."

mx35.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx35.tiki.ne.jp."

mx36.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx36.tiki.ne.jp."

mx4.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx4.et.tiki.ne.jp."

mx4.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx4.tiki.ne.jp."

mx41.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx41.tiki.ne.jp."

mx5.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx5.et.tiki.ne.jp."

mx5.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx5.tiki.ne.jp."

mx51.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx51.et.tiki.ne.jp."

mx51.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx51.tiki.ne.jp."

mx52.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx52.tiki.ne.jp."

mx6.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx6.et.tiki.ne.jp."

mx6.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx6.tiki.ne.jp."

mx61.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx61.tiki.ne.jp."

mx7.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx7.tiki.ne.jp."

mx71.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx71.tiki.ne.jp."

mx8.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx8.tiki.ne.jp."

mx81.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx81.tiki.ne.jp."

mx82.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx82.tiki.ne.jp."

mx9.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx9.tiki.ne.jp."

mx91.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx91.tiki.ne.jp."

nn.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with nn.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with nn.iij4u.or.jp."

pp.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp."

rr.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with rr.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with rr.iij4u.or.jp."

ss.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with ss.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with ss.iij4u.or.jp."

vm.aikis.or.jp:
  "mail.aikis.or.jp doesn’t end with vm.aikis.or.jp."
  "mail.aikis.or.jp doesn’t end with vm.aikis.or.jp."

vp.tiki.ne.jp:
  "vs.tiki.ne.jp doesn’t end with vp.tiki.ne.jp."

X-IL.jp:
  "mbox.iijmio-mail.jp doesn’t end with X-IL.jp."
  "mbox.iijmio-mail.jp doesn’t end with X-IL.jp."


Timeouts:

azm.janis.or.jp:
  "Couldn’t connect to smtp.azm.janis.or.jp:25, timed out."

cc9.ne.jp:
  "Couldn’t connect to smtp.cc9.ne.jp:25, timed out."

cek.ne.jp:
  "Couldn’t connect to smtp.cek.ne.jp:25, timed out."

ckm.janis.or.jp:
  "Error looking up SRV record for smtp.ckm.janis.or.jp, Timeout"
  "Couldn’t connect to smtp.ckm.janis.or.jp:25, timed out."

dhk.janis.or.jp:
  "Couldn’t connect to smtp.dhk.janis.or.jp:25, timed out."

dia.janis.or.jp:
  "Couldn’t connect to smtp.dia.janis.or.jp:25, timed out."

go.tvm.ne.jp:
  "Couldn’t connect to go.tvm.ne.jp:25, timed out."

grn.janis.or.jp:
  "Couldn’t connect to smtp.grn.janis.or.jp:25, timed out."

iiyama-catv.ne.jp:
  "Couldn’t connect to smtp.iiyama-catv.ne.jp:25, timed out."

ina.janis.or.jp:
  "Couldn’t connect to smtp.ina.janis.or.jp:25, timed out."

janis.or.jp:
  "Couldn’t connect to smtp.janis.or.jp:25, timed out."

kis.janis.or.jp:
  "Couldn’t connect to smtp.kis.janis.or.jp:25, timed out."

kokuyou.ne.jp:
  "Couldn’t connect to smtp.kokuyou.ne.jp:25, timed out."

mhl.janis.or.jp:
  "Couldn’t connect to smtp.mhl.janis.or.jp:25, timed out."

mid.janis.or.jp:
  "Couldn’t connect to smtp.mid.janis.or.jp:25, timed out."

mis.janis.or.jp:
  "Couldn’t connect to smtp.mis.janis.or.jp:25, timed out."

miy.janis.or.jp:
  "Couldn’t connect to smtp.miy.janis.or.jp:25, timed out."

mnet.ne.jp:
  "Error looking up SRV record for mail.mnet.ne.jp, Timeout"
  "Error looking up SRV record for mail.mnet.ne.jp, Timeout"

ngn.janis.or.jp:
  "Couldn’t connect to smtp.ngn.janis.or.jp:25, timed out."

nkn.janis.or.jp:
  "Couldn’t connect to smtp.nkn.janis.or.jp:25, timed out."

osk.janis.or.jp:
  "Couldn’t connect to smtp.osk.janis.or.jp:25, timed out."

pal.kijimadaira.jp:
  "Couldn’t connect to smtp.pal.kijimadaira.jp:25, timed out."

po.dcn.ne.jp:
  "Couldn’t connect to po.dcn.ne.jp:25, timed out."

sakunet.ne.jp:
  "Couldn’t connect to smtp.sakunet.ne.jp:25, timed out."

sas.janis.or.jp:
  "Couldn’t connect to smtp.sas.janis.or.jp:25, timed out."

sko.janis.or.jp:
  "Couldn’t connect to smtp.sko.janis.or.jp:25, timed out."

swk.janis.or.jp:
  "Couldn’t connect to smtp.swk.janis.or.jp:25, timed out."

tgk.janis.or.jp:
  "Couldn’t connect to smtp.tgk.janis.or.jp:25, timed out."

tyt.janis.or.jp:
  "Couldn’t connect to smtp.tyt.janis.or.jp:25, timed out."

ued.janis.or.jp:
  "Couldn’t connect to smtp.ued.janis.or.jp:25, timed out."

ytg.janis.or.jp:
  "Couldn’t connect to smtp.ytg.janis.or.jp:25, timed out."
------------------

So, commentary.

1) I had to massage the data, because the current version of the ISPDB doesn't output it in the same format.  I'll post that patch soon.

2) I think we might want to relax the differing endings to check one level up, so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

3) Are people's addresses really like "[e-mail]", or are they just "[e-mail]" (or better, "[e-mail]")?  And similarly, are they "[e-mail]" or "[e-mail]"?

I guess that's it.  Kohei, if you could check that all those errors (which are really more warnings) are okay, that would be great.

Thanks,
Blake.

Comment 11

[Ben Bucksch (:BenB)]

I think this:
> 2) I think we might want to relax the differing endings to check one level up,
> so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end
> with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

is a symptom of that:

> 3) Are people's addresses really like "[e-mail]", or are they
> just "[e-mail]" (or better, "[e-mail]")?
> And similarly, are they "[e-mail]" or "[e-mail]"?

I'd bet that mx6.et.tiki.ne.jp is not an email domain, but just the MX server  for et.tiki.ne.jp. Even that sounds too long for an email domain, but that may just be a Japanese custom to use many subdomains.

Comment 12

[u217243]

> Differing Endings:

mbox.iij4u.or.jp, mbox.iijmio-mail.jp, smtp-auth.tiki.ne.jp, vs.tiki.ne.jp,
and mail.aikis.or.jp are legitimate SMTP servers.

https://www.iij4u.or.jp/guide/basic/mail/thunderbird/
https://www.iijmio.jp/guide/thunderbird/
http://www.tiki.ne.jp/setup/mail/tb.html
http://www.tiki.ne.jp/setup/mail/vp.html
http://www.aikis.or.jp/aikis/pdf/vi_submission_Winmail.pdf

> Timeouts:

mail.*.janis.or.jp, smtp.*.janis.or.jp and some other JANIS servers seem to 
have been integrated into mail.janis.or.jp and smtp.janis.or.jp this spring. 
I'm checking that with the ISP.

http://www.janis.or.jp/mailsv/svdomain.html

smtp.cc9.ne.jp:25 is an legitimate server. 
http://www.cc9.jp/page.jsp?id=1581
My portscan result:
> Port Scanning host: 202.231.176.3
> Open TCP Port: 25 smtp

go.tvm.ne.jp should be go2.tvm.ne.jp according to the ISP's manual.
I'm checking that with the ISP.

mail.mnet.ne.jp is an legitimate server. 
http://www.mnet.ne.jp/support/Mail/Windows/Netscape7/netscape7_1.htm
My portscan result:
> Port Scanning host: 210.155.224.9
> Open TCP Port: 25 smtp

I couldn't find an online manual for po.dcn.ne.jp but
My portscan result:
> Port Scanning host: 219.105.32.8
> Open TCP Port: 25 smtp

> 2) I think we might want to relax the differing endings to check one level up,
> so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end
> with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

The differing endings are by design and should be allowed as exceptions.

> 3) Are people's addresses really like "[e-mail]", or are they
> just "[e-mail]" (or better, "[e-mail]")?  And similarly,
> are they "[e-mail]" or "[e-mail]"?

mx6.et.tiki.ne.jp and dd.iij4u.or.jp are actual domains for their email addresses.

I'm not sure why Japanese ISPs use so many subdomains, but that is a custom here.

Comment 13

[u217243]

(In reply to comment #12)
> mail.*.janis.or.jp, smtp.*.janis.or.jp and some other JANIS servers seem to 
> have been integrated into mail.janis.or.jp and smtp.janis.or.jp this spring. 
> I'm checking that with the ISP.
> 
> http://www.janis.or.jp/mailsv/svdomain.html

I just contacted the ISP. Those SMTP server names are correct but you can't 
connect the servers from other ISPs. So, the timeouts are OK.

> go.tvm.ne.jp should be go2.tvm.ne.jp according to the ISP's manual.
> I'm checking that with the ISP.

I've confirmed the go.tvm.ne.jp server is correct, not go2.

> I'm not sure why Japanese ISPs use so many subdomains, but that is a custom
> here.

One ISP has over 600 subdomains ;-)

Hope the ISPDB supports wildcard search. I'll file a bug.

Comment 14

[Ben Bucksch (:BenB)]

> Those SMTP server names are correct but you can't 
> connect the servers from other ISPs

We need to document that somehow, and warn users.

FWIW, that's what I added the <authentication>client-ip-based</authentication> for (not supported by 3.0).

Comment 15

[Ben Bucksch (:BenB)]

> Those SMTP server names are correct but you can't 
> connect the servers from other ISPs. So, the timeouts are OK.

Kohei Yoshino, can you please contact the ISP and let them make sure that the servers do not time out, but reject the connection immediately? Timeout will produce very bad usability for our users (and all others) who try the configuration on the road. A flat out rejection (connection refused or host not reachable IMCP) is just as good for them and avoids confusion and frustration for users.
(It also helps our heuristics, if we fall into that.)

> One ISP has over 600 subdomains ;-)
> Hope the ISPDB supports wildcard search. I'll file a bug.

That's what the config fetch from ISP was intended for. We can't possibly cater to all quirks.

Comment 16

[u217243]

(In reply to comment #15)
> Kohei Yoshino, can you please contact the ISP and let them make sure that the
> servers do not time out, but reject the connection immediately? Timeout will
> produce very bad usability for our users (and all others) who try the
> configuration on the road. A flat out rejection (connection refused or host not
> reachable IMCP) is just as good for them and avoids confusion and frustration
> for users.

The ISP, JANIS, has already let all of their customsers know the spec, so it's not a serious problem. It supports SMTP AUTH for customers who'd like to send messages from other ISPs but it's optional.

> > One ISP has over 600 subdomains ;-)
> > Hope the ISPDB supports wildcard search. I'll file a bug.
> 
> That's what the config fetch from ISP was intended for. We can't possibly cater
> to all quirks.

Does the feature work for any ISPs?
http://mxr.mozilla.org/comm-central/source/mailnews/base/prefs/content/accountcreation/fetchConfig.js#59

Comment 17

[Ben Bucksch (:BenB)]

> The ISP, JANIS, has already let all of their customsers know the spec

The fact that customers don't read such specs is the reason for this feature.

> It supports SMTP AUTH for customers who'd like to send
> messages from other ISPs but it's optional.

Is that another server or the same? If the same, how can that work if the server times out?

Either way, we should configure the SMTP AUTH config, so that the same config works also when they are connected to a different network.
We should configure whatever works in most situations (and is most secure).

> Does the feature work for any ISPs?

No, it didn't make 3.0, but I plan to add it for 3.1.

Comment 18

[u217243]

(In reply to comment #17)
> Is that another server or the same? If the same, how can that work if the
> server times out?

A different server.

> Either way, we should configure the SMTP AUTH config, so that the same config
> works also when they are connected to a different network.
> We should configure whatever works in most situations (and is most secure).

The customers have to change their email settings online to use the SMTP AUTH option. I'll ask the ISP about the spec again, but for now, please add the configrations as provided.

> > Does the feature work for any ISPs?
> 
> No, it didn't make 3.0, but I plan to add it for 3.1.

Understood.

Comment 19

[Ben Bucksch (:BenB)]

> The customers have to change their email settings online to use the
> SMTP AUTH option.

That defeats the whole purpose of the feature.
Why can't we just always configure the SMTP AUTH server?

Do they block the latter inside their network? If so, can you contact them to change this, so that it always works?

Comment 20

[Blake Winton (:bwinton) (:☕️)]

Attachment: [checked-in] A patch for the Japanese ISPs.

So, I needed to edit the generated files a little before making this patch, which basically contains all of the configs, since we seem happy with the warnings that the sanity checks reported.

(The edits I needed to make were "s/usernameForm/username/" and "g/<username><\/username>/d".)

Could you double-check that the info in this patch looks correct, and then ask Gozer for a second review, if you're happy, and we can get it committed?

Thanks,
Blake.

Comment 21

[u217243]

Comment on attachment 416110 [details] [diff] [review]
[checked-in] A patch for the Japanese ISPs.

Sorry about the <usernameForm>s. I'll be careful hereafter.

Comment 22

[Blake Winton (:bwinton) (:☕️)]

(In reply to comment #21)
> Sorry about the <usernameForm>s. I'll be careful hereafter.

Oh, it's not your fault.  The ISPDB is generating incorrect XML, at least until we fix bug 522253 (which should be really soon now).  :)

Thanks,
Blake.

Comment 23

[Blake Winton (:bwinton) (:☕️)]

Okay, so if we're all happy with this one, Gozer, would you please check them in to svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/ ?

Thanks,
Blake.

Comment 24

[u217243]

I don't have a check-in privilege to the directory, so please land the patch, then I'll inform the ISPs.

Comment 25

[Mark Banner (:standard8)]

Checked in to trunk: r57651

Please can someone verify the checkin is working (on trunk).

Comment 26

[Blake Winton (:bwinton) (:☕️)]

I tested with bu.iij4u.or.jp, and it seemed to work well.

I'm going to mark it as Resolved Fixed.  Yoshino, once the ISPs are happy, would you mind marking it Resolved Verified?

Thanks,
Blake.

Comment 27

[u217243]

Thank you so much. We're asking the ISPs to verify the actual setup result.

We still gathering configurations, so please keep this open.

Comment 28

[u217243]

Attachment: Japanese ISP configurations - round 2 patch

* Changed authentication settings upon request
* Changed X-IL.jp to x-il.jp upon request - X-IL.jp (uppercase) doesn't work anyway
* Added some new configuration files
* Fixed emailProvider IDs - no effect on the actual setup

Comment 29

[u217243]

Attachment: Japanese ISP configurations - round 2 new files

Comment 30

[u217243]

Attachment: 417439: Japanese ISP configurations - round 2 patch v1.1

Comment 31

[Kohei Yoshino [:kohei]]

I just filed Bug 534604 which affects tiki.ne.jp servers.

Comment 32

[Kohei Yoshino [:kohei]]

Could you run your sanity-checking script against the new 12 files? If you send me the script, I'll try myself.

Comment 33

[Blake Winton (:bwinton) (:☕️)]

Attachment: The results of the sanity tests.

(In reply to comment #32)
> Could you run your sanity-checking script against the new 12 files? If you send
> me the script, I'll try myself.

It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once the patch gets committed.)

Here's the data, in a manually-formatted version.

Thanks,
Blake.

Comment 34

[u217243]

(In reply to comment #33)
> It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once
> the patch gets committed.)

Understood. I'll file a bug for access to the actual ispdb.

> Here's the data, in a manually-formatted version.

* Differing endings look OK
* Connection errors might be OK, as my postscan passed (from Mozilla Japan office)
* X-IL.jp has been renamed x-il.jp

So please land the "round 2 patch v1.1".

Comment 35

[Ludovic Hirlimann [:Usul]]

(In reply to comment #34)
> (In reply to comment #33)
> > It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once
> > the patch gets committed.)
> 
> Understood. I'll file a bug for access to the actual ispdb.

With a OPenID account you laready have access. You just don't get to approve confs, but you can see he data - just go to the webapp.

Comment 36

[u217243]

Attachment: Japanese ISP configurations - round 1 patch fix

* Changed vm.aikis.or.jp's authentication setting upon request.
* Tidied up configurations of tiki.ne.jp and janis.or.jp servers by using %EMAILDOMAIN%s in the <hostname>s.

If looks good, please check-in this along with the "round 2 patch v1.1". Thanks!

Comment 37

[Ben Bucksch (:BenB)]

Comment on attachment 418608 [details] [diff] [review]
Japanese ISP configurations - round 1 patch fix   

     <emailProvider id="mx81.tiki.ne.jp">
+      <domain>ma100.tiki.ne.jp</domain>
+      <domain>mx1.tiki.ne.jp</domain>
+      <domain>mx2.et.tiki.ne.jp</domain>
and:
     <emailProvider id="mx2.wt.tiki.ne.jp">
+      <domain>ma100.tiki.ne.jp</domain>
+      <domain>mx1.tiki.ne.jp</domain>
+      <domain>mx2.et.tiki.ne.jp</domain>

Please make the emailprovider ID then the same as well. The point of that ID is exactly to identify identical configs, and the <domain> shows all the domains it applies for. If that means the patch shrinks to 2-3 configs (which have to be copied to the server 50 times, for 50 domains), all the better!

Comment 38

[Ben Bucksch (:BenB)]

Basically, every ISP should have exactly one emailprovider ID. IDs should be e.g. "tiki.ne.jp" and "janis.or.jp".
(Unless the provider really requires different configs, e.g. offers SSL only on some domains, but that's unlikely.)

Comment 39

[u217243]

Attachment: Japanese ISP configurations: round 2 patch v1.2 + round1 fix

Thank you for the review!

* Now all mx*.tiki.ne.jp config files use the same emailProvider ID and content
* Now all janis.or.jp config files use the same emailProvider ID and content
* Changed IIJ servers' authentication setting upon request
* Changed AIKIS server's authentication setting upon request
* Changed X-IL.jp to x-il.jp upon request. X-IL.jp (uppercase) doesn't work anyway
* Added some new config files

Comment 40

[Ben Bucksch (:BenB)]

I'm away on vacation, moving to bwinton. Thanks for the patch!

Comment 41

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

This patch doesn't apply cleanly against the current version in SVN.

Would you mind getting latest, fixing up the patch, and re-requesting review?

Thanks,
Blake.

Comment 42

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

Oh, wait, my fault I think.  Let me get on with that review.

Sorry about that,
Blake.

Comment 43

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

Okay, it passes all the things I could think of.

I guess I only really have two nits.

First, the emailProvider's id should probably be "tiki.ne.jp", instead of "mx1.tiki.ne.jp".

Second, the indentation is a little off.  It would be nice if we ran the new files through "xmllint --format".  (We should probably leave the old files as they are, at least for this patch.

Thanks,
Blake.

Comment 44

[Kohei Yoshino [:kohei]]

Attachment: [checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted

(In reply to comment #43)
> First, the emailProvider's id should probably be "tiki.ne.jp", instead of
> "mx1.tiki.ne.jp".

"tiki.ne.jp" server itself uses a different config than mx*.tiki.ne.jp servers, so I used "mx1.tiki.ne.jp" instead. I should have mentioned that earlier.

> Second, the indentation is a little off.  It would be nice if we ran the new
> files through "xmllint --format".  (We should probably leave the old files as
> they are, at least for this patch.

Formatted Japanese ISP config files with xmllint.

Comment 45

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 419080 [details] [diff] [review]
[checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted

Okay, I gave it a brief once over, and it looks good to me.

Who are you going to get for your second review?

Later,
Blake.

Comment 46

[Philippe M. Chiasson (:gozer)]

Comment on attachment 419080 [details] [diff] [review]
[checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted

A very time consuming review, but good to go.

In the future, I much prefer review requests for single-changes, instead of jumbo-patches that include many different, unrelated fixes. Especially when one of those change is a large whitespace/reindentation one.

Comment 47

[u217243]

(In reply to comment #46)
> In the future, I much prefer review requests for single-changes, instead of
> jumbo-patches that include many different, unrelated fixes. Especially when one
> of those change is a large whitespace/reindentation one.

Understood. I'll try to keep patches simple.

Please commit the patch to svn.

Comment 48

[u217243]

Could you check in the patch?

Comment 49

[Blake Winton (:bwinton) (:☕️)]

Checked in as http://viewvc.svn.mozilla.org/vc?view=revision&revision=59095

(Sorry that took so long, but I wasn't set up to commit to that tree.  It's fixed now, obviously.)

Thanks,
Blake.

Comment 50

[u217243]

Blake: Thank you!

Comment 51

[u217243]

Attachment: [checked-in] Fix ml.shibata.ne.jp configuration

I had an error in the config for ml.shibata.ne.jp. This patch will fix it.

Comment 52

[u217243]

Attachment: [checked-in] Add wind server configurations

10 configurations, 39 servers.

wind.jp, wind.ne.jp, wind.co.jp, gunmanet.or.jp and gunmanet.ne.jp domains are all owned by the Gunma Internet Corporation aka "wind".

Comment 53

[u217243]

Attachment: [checked-in] Add yahoo.co.jp and ybb.ne.jp configurations

yahoo.co.jp is Japanese version of Yahoo! Mail and almost the same as
https://live.mozillamessaging.com/autoconfig/yahoo.com

ybb.ne.jp is Yahoo! JAPAN's ISP.

Comment 54

[yassan]

Kohei Yoshino san, Shuriken 2010 has goo Mail Advance.
Please watch http://bugzilla.mozilla.gr.jp/show_bug.cgi?id=6709 .

Comment 55

[u217243]

yassan: we already have a config for goo mail advance:
https://live.mozillamessaging.com/autoconfig/goo.jp

Comment 56

[yassan]

(In reply to comment #55)
I'm sorry.

Comment 57

[u217243]

Attachment: [checked-in] Fix aikis configuration

SSL to STARTTLS upon request.

Comment 58

[u217243]

Attachment: [checked-in] Add Plala configurations

95 plala.or.jp subdomains

Comment 59

[u217243]

Attachment: [checked-in] Add Broba configurations

4 broba.cc subdomains (managed by the Plala ISP)

Comment 60

[Philippe M. Chiasson (:gozer)]

Comment on attachment 423483 [details] [diff] [review]
[checked-in] Fix aikis configuration

Verified working

Comment 61

[u217243]

Attachment: Add Biglobe configurations

The Biglobe ISP allows customers to customize their email addresses. There are 1,400 subdomains. Fortunately those configurations are all the same. I have abbreviated the long <domain> list (for reference) as

<domain>xx.biglobe.ne.jp</domain>
<domain>xx.eeyo.jp</domain>
<domain>xx.memail.jp</domain>
<domain>memoad.jp</domain>
<domain>myad.jp</domain>
<domain>xx.gozarre.jp</domain>

Comment 62

[u217243]

Comment on attachment 420504 [details] [diff] [review]
[checked-in] Fix ml.shibata.ne.jp configuration

r61878

Comment 63

[u217243]

Comment on attachment 423483 [details] [diff] [review]
[checked-in] Fix aikis configuration

r61878

Comment 64

[u217243]

Comment on attachment 420523 [details] [diff] [review]
[checked-in] Add wind server configurations

r61879

Comment 65

[u217243]

Comment on attachment 423468 [details] [diff] [review]
[checked-in] Add yahoo.co.jp and ybb.ne.jp configurations

r61880

Comment 66

[u217243]

Comment on attachment 423965 [details] [diff] [review]
[checked-in] Add Plala configurations

r61881

Comment 67

[u217243]

Comment on attachment 423966 [details] [diff] [review]
[checked-in] Add Broba configurations

r61881

Comment 68

[u217243]

Attachment: [checked-in] Add @nifty configuration

1 domain, 1 configuration.

Comment 69

[u217243]

Blake: Biglobe (see my comment 61) want to release their support pages for Thunderbird 3 next Monday, so it would be helpful if you could review the patch by the end of this week. Thank you :)

Comment 70

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 425465 [details] [diff] [review]
Add Biglobe configurations

(In reply to comment #61)
> I have abbreviated the long <domain> list (for reference) as
> <domain>xx.biglobe.ne.jp</domain>

So, because you did that, I can't run the sanity tests on the domains, so I'm going to have to give it an r-.

(But I have verified that one of them passes the sanity checks, and that they're all the same (with no typos), so if you add all the domains, I'll give it an r+.)

Thanks,
Blake.

Comment 71

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 426148 [details] [diff] [review]
[checked-in] Add @nifty configuration

It passes the sanity checks, and matches what gets auto-probed, and it seems to work.

But, it's not secure, and it's using POP3 instead of IMAP, so I would really appreciate it if you could ask them whether they have more secure/better servers available which we could use, but if they don't I guess I'm okay with this.

Later,
Blake.

Comment 72

[u217243]

(In reply to comment #70)
> (But I have verified that one of them passes the sanity checks, and that
> they're all the same (with no typos), so if you add all the domains, I'll give
> it an r+.)

So do we have to add all the 1,400 <domain>s to each configuration files?

Comment 73

[Blake Winton (:bwinton) (:☕️)]

Either that, or break them up into smaller sets somehow (like by sub-domain).

We do realize that this case isn't really handled well by the static autoconfig files, and we are looking into different ways of handling it, but it's highly unlikely that they'll be implemented by the end of this week.

Sorry about that,
Blake.

Comment 74

[u217243]

Comment on attachment 426148 [details] [diff] [review]
[checked-in] Add @nifty configuration

r62070

I'll ask the ISP about the future SSL and IMAP support.

Comment 75

[u217243]

(In reply to comment #73)
I have added all the domains. The new patch is too big (78.5 MB) so I just put the zipped patch (8.3 MB) here. Sorry to trouble you.

https://secure.mozilla-japan.org/temp/ced50b7f017a0eaa4143ca9088662335/biglobe.diff.zip

Comment 76

[u217243]

Blake: could you review the above patch?

Comment 77

[Blake Winton (:bwinton) (:☕️)]

Okay, it took all night to run, but here are the (trimmed down) results of the sanity checks:
Couldn’t get address for mail.biglobe.ne.jp.
Couldn’t find MX record for hiphop.memail.jp.
Couldn’t find MX record for ktd.biglobe.ne.jp.
Couldn’t find MX record for o.memail.jp.
Couldn’t find MX record for tea.biglobe.ne.jp.
Couldn’t find MX record for test.test.biglobe.ne.jp.
Couldn’t find MX record for west.biglobe.ne.jp.

Are those servers all spelled correctly?

I also got a bunch of "mail.biglobe.ne.jp doesn’t end with 007.biglobe.ne.jp." type errors, which I think I'm okay with.

So, I'm going to say r=me, as long as you promise that "test.test.biglobe.ne.jp" is _actually_ a real server that they use for something other than testing.  :)

Later,
Blake.

Comment 78

[u217243]

(In reply to comment #77)
> Okay, it took all night to run, but here are the (trimmed down) results of the
> sanity checks:
> Couldn’t get address for mail.biglobe.ne.jp.

My Network Utility says mail.biglobe.ne.jp = 202.225.89.6

> Are those servers all spelled correctly?
> 
> I also got a bunch of "mail.biglobe.ne.jp doesn’t end with 007.biglobe.ne.jp."
> type errors, which I think I'm okay with.

Those errors should be OK, as the server list has been provided by an email service staff of the ISP. I have confirmed that mail.biglobe.ne.jp is the proper POP/SMTP server name:
http://support.biglobe.ne.jp/settei/mailer/win-tb/tb_001.html

The existing support pages for other domains (memail.jp, eeyo.jp, etc.) say those POP/SMTP servers are mail.memail.jp and mail.eeyo.jp, but Biglobe's staff have said the descriptions would be updated. mail.biglobe.ne.jp, mail.memail.jp and mail.eeyo.jp are all the same server anyway.
http://email.biglobe.ne.jp/myadr/set.html
http://email.biglobe.ne.jp/dokodemo/set2.html

> So, I'm going to say r=me, as long as you promise that
> "test.test.biglobe.ne.jp" is _actually_ a real server that they use for
> something other than testing.  :)

Ha Ha, I haven't noticed there is a such subdomain in the list. Maybe an extra bonus!

Comment 79

[Kohei Yoshino [:kohei]]

Checked in the Biglobe configurations at r62236.

Their support pages are available at
http://support.biglobe.ne.jp/settei/mailer/win-tb/
http://support.biglobe.ne.jp/settei/mailer/mac-tb/

Comment 80

[u217243]

Attachment: Bug 546278 workaround

Until Bug 546278 is fixed, we should use the real hostnames instead of %EMAILDOMAIN% variables.

Comment 81

[Kohei Yoshino [:kohei]]

Comment on attachment 427306 [details] [diff] [review]
Bug 546278 workaround

I just posted patches against Bug 534604 and Bug 546278. Hope those will be landed to the 1.9.1 branch aka Thunderbird 3.0.3. Cleared r? flag.

Comment 82

[Ben Bucksch (:BenB)]

Biglobe now has 1400 (!) domains in our database.
I think that is unreasonable.

Many of these are even vanity domains like "cancer.", "milk.". Some German hosters also have vanity domains like "want-to-wake-up-with-you.de" (in German), "gotaquestion.de" and stuff like this. Total nonsense. Almost nobody is using them. I intentionally did not add them.
If one ISP has 1400 domains, I want to see prove that they are actually used by more than a few hundred users and justify the cost for us, like the slowness that all svn operations now take. Nevermind the inability to browse the list of domains anymore.
I request that the 1400 domains from Biglobe are removed again, please.

Comment 83

[u217243]

Certainly 1,400 domains are too many, but the solutions -- dynamic ISPDB nor fetchConfigFromISP -- have not been implemented at this time, unfortunately. I believe hosting static XML files on the live.momo server is temporary.

Biglobe is the #5 ISP here in Japan, so I guess each domain has hundreds of customers.

Comment 84

[Ben Bucksch (:BenB)]

> the solutions -- dynamic ISPDB nor
> fetchConfigFromISP -- have not been implemented at this time

Ah, yes, I am working on that, in fact just waiting for ui-review and commit approval.

> static XML files [for the many domains] on the live.momo server is temporary.

OK, great. Thanks!

Comment 85

[Ben Bucksch (:BenB)]

Now that we have "fetch from ISP", I filed bug 556655 about this (comment 82).

Comment 86

[u217243]

Closing this bug; we'll file a new bug on demand.