Closed Bug 524045 Opened 10 years ago Closed 9 years ago

[config] Add Japanese ISP configurations

Categories

(Webtools :: ISPDB Database Entries, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: u217243, Assigned: u217243)

Details

(Keywords: jp-critical, Whiteboard: [config])

Attachments

(12 files, 8 obsolete files)

41.47 KB, application/octet-stream
Details
98.19 KB, patch
u217243
: review+
gozer
: review+
Details | Diff | Splinter Review
5.57 KB, application/octet-stream
Details
2.64 KB, text/plain
Details
240.65 KB, patch
gozer
: review+
Details | Diff | Splinter Review
2.12 KB, patch
gozer
: review+
Details | Diff | Splinter Review
46.66 KB, patch
gozer
: review+
Details | Diff | Splinter Review
2.05 KB, patch
gozer
: review+
Details | Diff | Splinter Review
492 bytes, patch
gozer
: review+
Details | Diff | Splinter Review
100.27 KB, patch
gozer
: review+
Details | Diff | Splinter Review
4.53 KB, patch
gozer
: review+
Details | Diff | Splinter Review
1017 bytes, patch
bwinton
: review+
Details | Diff | Splinter Review
Mozilla Japan is gathering email configurations directly from Japanese ISPs. We'll submit new configs then add the list here, so please approve them. (Or could you give us approval authority?)

Quick question for you: does the ISP database allow companies, universities and governments to add their configs? Many organizations would like to utilize the DB if anyone could use that.
How can I select STARTTLS for incoming/outgoing socket type, and secure for incoming/outgoing authentication? Also, the Language of the settings page list has only English and French...
(In reply to comment #1)
> How can I select STARTTLS for incoming/outgoing socket type, and secure for
> incoming/outgoing authentication? Also, the Language of the settings page list
> has only English and French...

Ispdb.mozillamessaging.com is not on production yet , it's there to gather feedback about it.
We're not going to delegate authority, certainly not at this early stage.  I think at this point the only real way to get new files into the database would be to submit patches against http://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/

Although I would suggest using preview.ispdb, and then using the 'export as xml' function to then generate the patches.
(In reply to comment #3)
> We're not going to delegate authority, certainly not at this early stage.  I
> think at this point the only real way to get new files into the database would
> be to submit patches against
> http://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/

Understood. We'll make and attach patches...
Attached file ISP configuration file: IIJ4U (obsolete) —
Attached file ISP configuration file: IIJmio (obsolete) —
Attached file ISP config files
We have collected some configurations. Could you commit these files to the SVN? This attachment (ZIP) contains 86 XML config files. More to come...
It'd be good to run these configurations through our sanity-checking script.  I'm confident that Kohei did it right, so it'd be to test the script as much as the config files.
Once the configurations are added, the ISPs can test the account setup feature with a Release Candidate build...
Assignee: nobody → yoshino
Status: NEW → ASSIGNED
Here are the results of the sanity checks, broken down into two different headings, "Differing Endings" and "Timeouts".

More commentary at the bottom.

------------------
Differing Endings:

bc.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bc.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bc.iij4u.or.jp."

bk.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bk.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bk.iij4u.or.jp."

bp.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bp.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bp.iij4u.or.jp."

bu.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with bu.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with bu.iij4u.or.jp."

dd.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with dd.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with dd.iij4u.or.jp."

e23.jp:
  "mbox.iijmio-mail.jp doesn’t end with e23.jp."
  "mbox.iijmio-mail.jp doesn’t end with e23.jp."

ff.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with ff.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with ff.iij4u.or.jp."

hh.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with hh.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with hh.iij4u.or.jp."

kk.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with kk.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with kk.iij4u.or.jp."

ma100.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with ma100.tiki.ne.jp."

miobox.jp:
  "mbox.iijmio-mail.jp doesn’t end with miobox.jp."
  "mbox.iijmio-mail.jp doesn’t end with miobox.jp."

miomio.jp:
  "mbox.iijmio-mail.jp doesn’t end with miomio.jp."
  "mbox.iijmio-mail.jp doesn’t end with miomio.jp."

mx1.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx1.tiki.ne.jp."

mx2.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.et.tiki.ne.jp."

mx2.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.tiki.ne.jp."

mx2.wt.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx2.wt.tiki.ne.jp."

mx21.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx21.tiki.ne.jp."

mx22.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx22.tiki.ne.jp."

mx3.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx3.et.tiki.ne.jp."

mx3.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx3.tiki.ne.jp."

mx31.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx31.tiki.ne.jp."

mx32.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx32.tiki.ne.jp."

mx35.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx35.tiki.ne.jp."

mx36.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx36.tiki.ne.jp."

mx4.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx4.et.tiki.ne.jp."

mx4.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx4.tiki.ne.jp."

mx41.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx41.tiki.ne.jp."

mx5.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx5.et.tiki.ne.jp."

mx5.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx5.tiki.ne.jp."

mx51.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx51.et.tiki.ne.jp."

mx51.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx51.tiki.ne.jp."

mx52.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx52.tiki.ne.jp."

mx6.et.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx6.et.tiki.ne.jp."

mx6.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx6.tiki.ne.jp."

mx61.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx61.tiki.ne.jp."

mx7.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx7.tiki.ne.jp."

mx71.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx71.tiki.ne.jp."

mx8.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx8.tiki.ne.jp."

mx81.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx81.tiki.ne.jp."

mx82.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx82.tiki.ne.jp."

mx9.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx9.tiki.ne.jp."

mx91.tiki.ne.jp:
  "smtp-auth.tiki.ne.jp doesn’t end with mx91.tiki.ne.jp."

nn.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with nn.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with nn.iij4u.or.jp."

pp.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp."

rr.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with rr.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with rr.iij4u.or.jp."

ss.iij4u.or.jp:
  "mbox.iij4u.or.jp doesn’t end with ss.iij4u.or.jp."
  "mbox.iij4u.or.jp doesn’t end with ss.iij4u.or.jp."

vm.aikis.or.jp:
  "mail.aikis.or.jp doesn’t end with vm.aikis.or.jp."
  "mail.aikis.or.jp doesn’t end with vm.aikis.or.jp."

vp.tiki.ne.jp:
  "vs.tiki.ne.jp doesn’t end with vp.tiki.ne.jp."

X-IL.jp:
  "mbox.iijmio-mail.jp doesn’t end with X-IL.jp."
  "mbox.iijmio-mail.jp doesn’t end with X-IL.jp."


Timeouts:

azm.janis.or.jp:
  "Couldn’t connect to smtp.azm.janis.or.jp:25, timed out."

cc9.ne.jp:
  "Couldn’t connect to smtp.cc9.ne.jp:25, timed out."

cek.ne.jp:
  "Couldn’t connect to smtp.cek.ne.jp:25, timed out."

ckm.janis.or.jp:
  "Error looking up SRV record for smtp.ckm.janis.or.jp, Timeout"
  "Couldn’t connect to smtp.ckm.janis.or.jp:25, timed out."

dhk.janis.or.jp:
  "Couldn’t connect to smtp.dhk.janis.or.jp:25, timed out."

dia.janis.or.jp:
  "Couldn’t connect to smtp.dia.janis.or.jp:25, timed out."

go.tvm.ne.jp:
  "Couldn’t connect to go.tvm.ne.jp:25, timed out."

grn.janis.or.jp:
  "Couldn’t connect to smtp.grn.janis.or.jp:25, timed out."

iiyama-catv.ne.jp:
  "Couldn’t connect to smtp.iiyama-catv.ne.jp:25, timed out."

ina.janis.or.jp:
  "Couldn’t connect to smtp.ina.janis.or.jp:25, timed out."

janis.or.jp:
  "Couldn’t connect to smtp.janis.or.jp:25, timed out."

kis.janis.or.jp:
  "Couldn’t connect to smtp.kis.janis.or.jp:25, timed out."

kokuyou.ne.jp:
  "Couldn’t connect to smtp.kokuyou.ne.jp:25, timed out."

mhl.janis.or.jp:
  "Couldn’t connect to smtp.mhl.janis.or.jp:25, timed out."

mid.janis.or.jp:
  "Couldn’t connect to smtp.mid.janis.or.jp:25, timed out."

mis.janis.or.jp:
  "Couldn’t connect to smtp.mis.janis.or.jp:25, timed out."

miy.janis.or.jp:
  "Couldn’t connect to smtp.miy.janis.or.jp:25, timed out."

mnet.ne.jp:
  "Error looking up SRV record for mail.mnet.ne.jp, Timeout"
  "Error looking up SRV record for mail.mnet.ne.jp, Timeout"

ngn.janis.or.jp:
  "Couldn’t connect to smtp.ngn.janis.or.jp:25, timed out."

nkn.janis.or.jp:
  "Couldn’t connect to smtp.nkn.janis.or.jp:25, timed out."

osk.janis.or.jp:
  "Couldn’t connect to smtp.osk.janis.or.jp:25, timed out."

pal.kijimadaira.jp:
  "Couldn’t connect to smtp.pal.kijimadaira.jp:25, timed out."

po.dcn.ne.jp:
  "Couldn’t connect to po.dcn.ne.jp:25, timed out."

sakunet.ne.jp:
  "Couldn’t connect to smtp.sakunet.ne.jp:25, timed out."

sas.janis.or.jp:
  "Couldn’t connect to smtp.sas.janis.or.jp:25, timed out."

sko.janis.or.jp:
  "Couldn’t connect to smtp.sko.janis.or.jp:25, timed out."

swk.janis.or.jp:
  "Couldn’t connect to smtp.swk.janis.or.jp:25, timed out."

tgk.janis.or.jp:
  "Couldn’t connect to smtp.tgk.janis.or.jp:25, timed out."

tyt.janis.or.jp:
  "Couldn’t connect to smtp.tyt.janis.or.jp:25, timed out."

ued.janis.or.jp:
  "Couldn’t connect to smtp.ued.janis.or.jp:25, timed out."

ytg.janis.or.jp:
  "Couldn’t connect to smtp.ytg.janis.or.jp:25, timed out."
------------------

So, commentary.

1) I had to massage the data, because the current version of the ISPDB doesn't output it in the same format.  I'll post that patch soon.

2) I think we might want to relax the differing endings to check one level up, so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

3) Are people's addresses really like "example@mx6.et.tiki.ne.jp", or are they just "example@et.tiki.ne.jp" (or better, "example@tiki.ne.jp")?  And similarly, are they "example@dd.iij4u.or.jp" or "example@iij4u.or.jp"?

I guess that's it.  Kohei, if you could check that all those errors (which are really more warnings) are okay, that would be great.

Thanks,
Blake.
I think this:
> 2) I think we might want to relax the differing endings to check one level up,
> so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end
> with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

is a symptom of that:

> 3) Are people's addresses really like "example@mx6.et.tiki.ne.jp", or are they
> just "example@et.tiki.ne.jp" (or better, "example@tiki.ne.jp")?
> And similarly, are they "example@dd.iij4u.or.jp" or "example@iij4u.or.jp"?

I'd bet that mx6.et.tiki.ne.jp is not an email domain, but just the MX server  for et.tiki.ne.jp. Even that sounds too long for an email domain, but that may just be a Japanese custom to use many subdomains.
> Differing Endings:

mbox.iij4u.or.jp, mbox.iijmio-mail.jp, smtp-auth.tiki.ne.jp, vs.tiki.ne.jp,
and mail.aikis.or.jp are legitimate SMTP servers.

https://www.iij4u.or.jp/guide/basic/mail/thunderbird/
https://www.iijmio.jp/guide/thunderbird/
http://www.tiki.ne.jp/setup/mail/tb.html
http://www.tiki.ne.jp/setup/mail/vp.html
http://www.aikis.or.jp/aikis/pdf/vi_submission_Winmail.pdf

> Timeouts:

mail.*.janis.or.jp, smtp.*.janis.or.jp and some other JANIS servers seem to 
have been integrated into mail.janis.or.jp and smtp.janis.or.jp this spring. 
I'm checking that with the ISP.

http://www.janis.or.jp/mailsv/svdomain.html

smtp.cc9.ne.jp:25 is an legitimate server. 
http://www.cc9.jp/page.jsp?id=1581
My portscan result:
> Port Scanning host: 202.231.176.3
> Open TCP Port: 25 smtp

go.tvm.ne.jp should be go2.tvm.ne.jp according to the ISP's manual.
I'm checking that with the ISP.

mail.mnet.ne.jp is an legitimate server. 
http://www.mnet.ne.jp/support/Mail/Windows/Netscape7/netscape7_1.htm
My portscan result:
> Port Scanning host: 210.155.224.9
> Open TCP Port: 25 smtp

I couldn't find an online manual for po.dcn.ne.jp but
My portscan result:
> Port Scanning host: 219.105.32.8
> Open TCP Port: 25 smtp

> 2) I think we might want to relax the differing endings to check one level up,
> so that while mbox.iij4u.or.jp doesn’t end with pp.iij4u.or.jp, it does end
> with iij4u.or.jp, and so perhaps should be allowed.  Thoughts?

The differing endings are by design and should be allowed as exceptions.

> 3) Are people's addresses really like "example@mx6.et.tiki.ne.jp", or are they
> just "example@et.tiki.ne.jp" (or better, "example@tiki.ne.jp")?  And similarly,
> are they "example@dd.iij4u.or.jp" or "example@iij4u.or.jp"?

mx6.et.tiki.ne.jp and dd.iij4u.or.jp are actual domains for their email addresses.

I'm not sure why Japanese ISPs use so many subdomains, but that is a custom here.
(In reply to comment #12)
> mail.*.janis.or.jp, smtp.*.janis.or.jp and some other JANIS servers seem to 
> have been integrated into mail.janis.or.jp and smtp.janis.or.jp this spring. 
> I'm checking that with the ISP.
> 
> http://www.janis.or.jp/mailsv/svdomain.html

I just contacted the ISP. Those SMTP server names are correct but you can't 
connect the servers from other ISPs. So, the timeouts are OK.

> go.tvm.ne.jp should be go2.tvm.ne.jp according to the ISP's manual.
> I'm checking that with the ISP.

I've confirmed the go.tvm.ne.jp server is correct, not go2.

> I'm not sure why Japanese ISPs use so many subdomains, but that is a custom
> here.

One ISP has over 600 subdomains ;-)

Hope the ISPDB supports wildcard search. I'll file a bug.
> Those SMTP server names are correct but you can't 
> connect the servers from other ISPs

We need to document that somehow, and warn users.

FWIW, that's what I added the <authentication>client-ip-based</authentication> for (not supported by 3.0).
> Those SMTP server names are correct but you can't 
> connect the servers from other ISPs. So, the timeouts are OK.

Kohei Yoshino, can you please contact the ISP and let them make sure that the servers do not time out, but reject the connection immediately? Timeout will produce very bad usability for our users (and all others) who try the configuration on the road. A flat out rejection (connection refused or host not reachable IMCP) is just as good for them and avoids confusion and frustration for users.
(It also helps our heuristics, if we fall into that.)

> One ISP has over 600 subdomains ;-)
> Hope the ISPDB supports wildcard search. I'll file a bug.

That's what the config fetch from ISP was intended for. We can't possibly cater to all quirks.
(In reply to comment #15)
> Kohei Yoshino, can you please contact the ISP and let them make sure that the
> servers do not time out, but reject the connection immediately? Timeout will
> produce very bad usability for our users (and all others) who try the
> configuration on the road. A flat out rejection (connection refused or host not
> reachable IMCP) is just as good for them and avoids confusion and frustration
> for users.

The ISP, JANIS, has already let all of their customsers know the spec, so it's not a serious problem. It supports SMTP AUTH for customers who'd like to send messages from other ISPs but it's optional.

> > One ISP has over 600 subdomains ;-)
> > Hope the ISPDB supports wildcard search. I'll file a bug.
> 
> That's what the config fetch from ISP was intended for. We can't possibly cater
> to all quirks.

Does the feature work for any ISPs?
http://mxr.mozilla.org/comm-central/source/mailnews/base/prefs/content/accountcreation/fetchConfig.js#59
> The ISP, JANIS, has already let all of their customsers know the spec

The fact that customers don't read such specs is the reason for this feature.

> It supports SMTP AUTH for customers who'd like to send
> messages from other ISPs but it's optional.

Is that another server or the same? If the same, how can that work if the server times out?

Either way, we should configure the SMTP AUTH config, so that the same config works also when they are connected to a different network.
We should configure whatever works in most situations (and is most secure).

> Does the feature work for any ISPs?

No, it didn't make 3.0, but I plan to add it for 3.1.
(In reply to comment #17)
> Is that another server or the same? If the same, how can that work if the
> server times out?

A different server.

> Either way, we should configure the SMTP AUTH config, so that the same config
> works also when they are connected to a different network.
> We should configure whatever works in most situations (and is most secure).

The customers have to change their email settings online to use the SMTP AUTH option. I'll ask the ISP about the spec again, but for now, please add the configrations as provided.

> > Does the feature work for any ISPs?
> 
> No, it didn't make 3.0, but I plan to add it for 3.1.

Understood.
> The customers have to change their email settings online to use the
> SMTP AUTH option.

That defeats the whole purpose of the feature.
Why can't we just always configure the SMTP AUTH server?

Do they block the latter inside their network? If so, can you contact them to change this, so that it always works?
So, I needed to edit the generated files a little before making this patch, which basically contains all of the configs, since we seem happy with the warnings that the sanity checks reported.

(The edits I needed to make were "s/usernameForm/username/" and "g/<username><\/username>/d".)

Could you double-check that the info in this patch looks correct, and then ask Gozer for a second review, if you're happy, and we can get it committed?

Thanks,
Blake.
Attachment #416110 - Flags: review?(yoshino)
Comment on attachment 416110 [details] [diff] [review]
[checked-in] A patch for the Japanese ISPs.

Sorry about the <usernameForm>s. I'll be careful hereafter.
Attachment #416110 - Flags: review?(yoshino)
Attachment #416110 - Flags: review?(gozer)
Attachment #416110 - Flags: review+
(In reply to comment #21)
> Sorry about the <usernameForm>s. I'll be careful hereafter.

Oh, it's not your fault.  The ISPDB is generating incorrect XML, at least until we fix bug 522253 (which should be really soon now).  :)

Thanks,
Blake.
Attachment #416110 - Flags: review?(gozer) → review+
Okay, so if we're all happy with this one, Gozer, would you please check them in to svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/ ?

Thanks,
Blake.
Keywords: checkin-needed
I don't have a check-in privilege to the directory, so please land the patch, then I'll inform the ISPs.
Checked in to trunk: r57651

Please can someone verify the checkin is working (on trunk).
I tested with bu.iij4u.or.jp, and it seemed to work well.

I'm going to mark it as Resolved Fixed.  Yoshino, once the ISPs are happy, would you mind marking it Resolved Verified?

Thanks,
Blake.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Thank you so much. We're asking the ISPs to verify the actual setup result.

We still gathering configurations, so please keep this open.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
* Changed authentication settings upon request
* Changed X-IL.jp to x-il.jp upon request - X-IL.jp (uppercase) doesn't work anyway
* Added some new configuration files
* Fixed emailProvider IDs - no effect on the actual setup
Attachment #408014 - Attachment is obsolete: true
Attachment #408015 - Attachment is obsolete: true
Attachment #417441 - Attachment is patch: true
Attachment #417441 - Attachment mime type: application/octet-stream → text/plain
Attachment #417441 - Flags: review?(gozer)
Attachment #417439 - Attachment is obsolete: true
I just filed Bug 534604 which affects tiki.ne.jp servers.
Status: REOPENED → ASSIGNED
Attachment #417441 - Flags: review?(gozer) → review+
Could you run your sanity-checking script against the new 12 files? If you send me the script, I'll try myself.
(In reply to comment #32)
> Could you run your sanity-checking script against the new 12 files? If you send
> me the script, I'll try myself.

It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once the patch gets committed.)

Here's the data, in a manually-formatted version.

Thanks,
Blake.
(In reply to comment #33)
> It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once
> the patch gets committed.)

Understood. I'll file a bug for access to the actual ispdb.

> Here's the data, in a manually-formatted version.

* Differing endings look OK
* Connection errors might be OK, as my postscan passed (from Mozilla Japan office)
* X-IL.jp has been renamed x-il.jp

So please land the "round 2 patch v1.1".
(In reply to comment #34)
> (In reply to comment #33)
> > It's not a script, but is instead built in to the ISDPB.  (Uh, or will be once
> > the patch gets committed.)
> 
> Understood. I'll file a bug for access to the actual ispdb.

With a OPenID account you laready have access. You just don't get to approve confs, but you can see he data - just go to the webapp.
* Changed vm.aikis.or.jp's authentication setting upon request.
* Tidied up configurations of tiki.ne.jp and janis.or.jp servers by using %EMAILDOMAIN%s in the <hostname>s.

If looks good, please check-in this along with the "round 2 patch v1.1". Thanks!
Attachment #418608 - Flags: review?(gozer)
Comment on attachment 418608 [details] [diff] [review]
Japanese ISP configurations - round 1 patch fix   

     <emailProvider id="mx81.tiki.ne.jp">
+      <domain>ma100.tiki.ne.jp</domain>
+      <domain>mx1.tiki.ne.jp</domain>
+      <domain>mx2.et.tiki.ne.jp</domain>
and:
     <emailProvider id="mx2.wt.tiki.ne.jp">
+      <domain>ma100.tiki.ne.jp</domain>
+      <domain>mx1.tiki.ne.jp</domain>
+      <domain>mx2.et.tiki.ne.jp</domain>

Please make the emailprovider ID then the same as well. The point of that ID is exactly to identify identical configs, and the <domain> shows all the domains it applies for. If that means the patch shrinks to 2-3 configs (which have to be copied to the server 50 times, for 50 domains), all the better!
Attachment #418608 - Flags: review?(gozer) → review-
Basically, every ISP should have exactly one emailprovider ID. IDs should be e.g. "tiki.ne.jp" and "janis.or.jp".
(Unless the provider really requires different configs, e.g. offers SSL only on some domains, but that's unlikely.)
Attachment #417441 - Attachment is obsolete: true
Attachment #418608 - Attachment is obsolete: true
Thank you for the review!

* Now all mx*.tiki.ne.jp config files use the same emailProvider ID and content
* Now all janis.or.jp config files use the same emailProvider ID and content
* Changed IIJ servers' authentication setting upon request
* Changed AIKIS server's authentication setting upon request
* Changed X-IL.jp to x-il.jp upon request. X-IL.jp (uppercase) doesn't work anyway
* Added some new config files
Attachment #418798 - Flags: review?(ben.bucksch)
Attachment #418798 - Flags: review?(ben.bucksch) → review?(bwinton)
I'm away on vacation, moving to bwinton. Thanks for the patch!
Attachment #416110 - Attachment description: A patch for the Japanese ISPs. → [checked-in] A patch for the Japanese ISPs.
Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

This patch doesn't apply cleanly against the current version in SVN.

Would you mind getting latest, fixing up the patch, and re-requesting review?

Thanks,
Blake.
Attachment #418798 - Flags: review?(bwinton) → review-
Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

Oh, wait, my fault I think.  Let me get on with that review.

Sorry about that,
Blake.
Attachment #418798 - Flags: review- → review?(bwinton)
Comment on attachment 418798 [details] [diff] [review]
Japanese ISP configurations: round 2 patch v1.2 + round1 fix

Okay, it passes all the things I could think of.

I guess I only really have two nits.

First, the emailProvider's id should probably be "tiki.ne.jp", instead of "mx1.tiki.ne.jp".

Second, the indentation is a little off.  It would be nice if we ran the new files through "xmllint --format".  (We should probably leave the old files as they are, at least for this patch.

Thanks,
Blake.
Attachment #418798 - Flags: review?(bwinton) → review+
(In reply to comment #43)
> First, the emailProvider's id should probably be "tiki.ne.jp", instead of
> "mx1.tiki.ne.jp".

"tiki.ne.jp" server itself uses a different config than mx*.tiki.ne.jp servers, so I used "mx1.tiki.ne.jp" instead. I should have mentioned that earlier.

> Second, the indentation is a little off.  It would be nice if we ran the new
> files through "xmllint --format".  (We should probably leave the old files as
> they are, at least for this patch.

Formatted Japanese ISP config files with xmllint.
Attachment #418798 - Attachment is obsolete: true
Attachment #419080 - Flags: review?(bwinton)
Comment on attachment 419080 [details] [diff] [review]
[checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted

Okay, I gave it a brief once over, and it looks good to me.

Who are you going to get for your second review?

Later,
Blake.
Attachment #419080 - Flags: review?(bwinton) → review+
Attachment #419080 - Flags: review+ → review?(gozer)
Comment on attachment 419080 [details] [diff] [review]
[checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted

A very time consuming review, but good to go.

In the future, I much prefer review requests for single-changes, instead of jumbo-patches that include many different, unrelated fixes. Especially when one of those change is a large whitespace/reindentation one.
Attachment #419080 - Flags: review?(gozer) → review+
(In reply to comment #46)
> In the future, I much prefer review requests for single-changes, instead of
> jumbo-patches that include many different, unrelated fixes. Especially when one
> of those change is a large whitespace/reindentation one.

Understood. I'll try to keep patches simple.

Please commit the patch to svn.
Could you check in the patch?
Checked in as http://viewvc.svn.mozilla.org/vc?view=revision&revision=59095

(Sorry that took so long, but I wasn't set up to commit to that tree.  It's fixed now, obviously.)

Thanks,
Blake.
Attachment #419080 - Attachment description: Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted → [checked-in] Japanese ISP configurations: round 2 patch v1.2 + round1 fix + formatted
Blake: Thank you!
I had an error in the config for ml.shibata.ne.jp. This patch will fix it.
Attachment #420504 - Flags: review?(gozer)
10 configurations, 39 servers.

wind.jp, wind.ne.jp, wind.co.jp, gunmanet.or.jp and gunmanet.ne.jp domains are all owned by the Gunma Internet Corporation aka "wind".
Attachment #420523 - Flags: review?(gozer)
Attachment #420504 - Flags: review?(gozer) → review+
Attachment #420523 - Flags: review?(gozer) → review+
Keywords: checkin-needed
yahoo.co.jp is Japanese version of Yahoo! Mail and almost the same as
https://live.mozillamessaging.com/autoconfig/yahoo.com

ybb.ne.jp is Yahoo! JAPAN's ISP.
Attachment #423468 - Flags: review?
Attachment #423468 - Flags: review? → review?(gozer)
Attachment #423468 - Flags: review?(gozer)
Attachment #423468 - Flags: review?(gozer)
Kohei Yoshino san, Shuriken 2010 has goo Mail Advance.
Please watch http://bugzilla.mozilla.gr.jp/show_bug.cgi?id=6709 .
yassan: we already have a config for goo mail advance:
https://live.mozillamessaging.com/autoconfig/goo.jp
(In reply to comment #55)
I'm sorry.
SSL to STARTTLS upon request.
Attachment #423483 - Flags: review?(gozer)
95 plala.or.jp subdomains
Attachment #423965 - Flags: review?(gozer)
4 broba.cc subdomains (managed by the Plala ISP)
Attachment #423966 - Flags: review?(gozer)
Attachment #423966 - Flags: review?(gozer) → review+
Comment on attachment 423483 [details] [diff] [review]
[checked-in] Fix aikis configuration

Verified working
Attachment #423483 - Flags: review?(gozer) → review+
Attachment #423468 - Flags: review?(gozer) → review+
Attachment #423965 - Flags: review?(gozer) → review+
Attached patch Add Biglobe configurations (obsolete) — Splinter Review
The Biglobe ISP allows customers to customize their email addresses. There are 1,400 subdomains. Fortunately those configurations are all the same. I have abbreviated the long <domain> list (for reference) as

<domain>xx.biglobe.ne.jp</domain>
<domain>xx.eeyo.jp</domain>
<domain>xx.memail.jp</domain>
<domain>memoad.jp</domain>
<domain>myad.jp</domain>
<domain>xx.gozarre.jp</domain>
Attachment #425465 - Flags: review?(gozer)
Attachment #425465 - Flags: review?(gozer) → review?(bwinton)
Comment on attachment 420504 [details] [diff] [review]
[checked-in] Fix ml.shibata.ne.jp configuration

r61878
Attachment #420504 - Attachment description: Fix ml.shibata.ne.jp configuration → [checked-in] Fix ml.shibata.ne.jp configuration
Comment on attachment 423483 [details] [diff] [review]
[checked-in] Fix aikis configuration

r61878
Attachment #423483 - Attachment description: Fix aikis configuration → [checked-in] Fix aikis configuration
Comment on attachment 420523 [details] [diff] [review]
[checked-in] Add wind server configurations

r61879
Attachment #420523 - Attachment description: Add wind server configurations → [checked-in] Add wind server configurations
Comment on attachment 423468 [details] [diff] [review]
[checked-in] Add yahoo.co.jp and ybb.ne.jp configurations

r61880
Attachment #423468 - Attachment description: Add yahoo.co.jp and ybb.ne.jp configurations → [checked-in] Add yahoo.co.jp and ybb.ne.jp configurations
Comment on attachment 423965 [details] [diff] [review]
[checked-in] Add Plala configurations

r61881
Attachment #423965 - Attachment description: Add Plala configurations → [checked-in] Add Plala configurations
Comment on attachment 423966 [details] [diff] [review]
[checked-in] Add Broba configurations

r61881
Attachment #423966 - Attachment description: Add Broba configurations → [checked-in] Add Broba configurations
1 domain, 1 configuration.
Attachment #426148 - Flags: review?(bwinton)
Blake: Biglobe (see my comment 61) want to release their support pages for Thunderbird 3 next Monday, so it would be helpful if you could review the patch by the end of this week. Thank you :)
Comment on attachment 425465 [details] [diff] [review]
Add Biglobe configurations

(In reply to comment #61)
> I have abbreviated the long <domain> list (for reference) as
> <domain>xx.biglobe.ne.jp</domain>

So, because you did that, I can't run the sanity tests on the domains, so I'm going to have to give it an r-.

(But I have verified that one of them passes the sanity checks, and that they're all the same (with no typos), so if you add all the domains, I'll give it an r+.)

Thanks,
Blake.
Attachment #425465 - Flags: review?(bwinton) → review-
Comment on attachment 426148 [details] [diff] [review]
[checked-in] Add @nifty configuration

It passes the sanity checks, and matches what gets auto-probed, and it seems to work.

But, it's not secure, and it's using POP3 instead of IMAP, so I would really appreciate it if you could ask them whether they have more secure/better servers available which we could use, but if they don't I guess I'm okay with this.

Later,
Blake.
Attachment #426148 - Flags: review?(bwinton) → review+
(In reply to comment #70)
> (But I have verified that one of them passes the sanity checks, and that
> they're all the same (with no typos), so if you add all the domains, I'll give
> it an r+.)

So do we have to add all the 1,400 <domain>s to each configuration files?
Either that, or break them up into smaller sets somehow (like by sub-domain).

We do realize that this case isn't really handled well by the static autoconfig files, and we are looking into different ways of handling it, but it's highly unlikely that they'll be implemented by the end of this week.

Sorry about that,
Blake.
Comment on attachment 426148 [details] [diff] [review]
[checked-in] Add @nifty configuration

r62070

I'll ask the ISP about the future SSL and IMAP support.
Attachment #426148 - Attachment description: Add @nifty configuration → [checked-in] Add @nifty configuration
(In reply to comment #73)
I have added all the domains. The new patch is too big (78.5 MB) so I just put the zipped patch (8.3 MB) here. Sorry to trouble you.

https://secure.mozilla-japan.org/temp/ced50b7f017a0eaa4143ca9088662335/biglobe.diff.zip
Attachment #425465 - Attachment is obsolete: true
Blake: could you review the above patch?
Okay, it took all night to run, but here are the (trimmed down) results of the sanity checks:
Couldn’t get address for mail.biglobe.ne.jp.
Couldn’t find MX record for hiphop.memail.jp.
Couldn’t find MX record for ktd.biglobe.ne.jp.
Couldn’t find MX record for o.memail.jp.
Couldn’t find MX record for tea.biglobe.ne.jp.
Couldn’t find MX record for test.test.biglobe.ne.jp.
Couldn’t find MX record for west.biglobe.ne.jp.

Are those servers all spelled correctly?

I also got a bunch of "mail.biglobe.ne.jp doesn’t end with 007.biglobe.ne.jp." type errors, which I think I'm okay with.

So, I'm going to say r=me, as long as you promise that "test.test.biglobe.ne.jp" is _actually_ a real server that they use for something other than testing.  :)

Later,
Blake.
(In reply to comment #77)
> Okay, it took all night to run, but here are the (trimmed down) results of the
> sanity checks:
> Couldn’t get address for mail.biglobe.ne.jp.

My Network Utility says mail.biglobe.ne.jp = 202.225.89.6

> Are those servers all spelled correctly?
> 
> I also got a bunch of "mail.biglobe.ne.jp doesn’t end with 007.biglobe.ne.jp."
> type errors, which I think I'm okay with.

Those errors should be OK, as the server list has been provided by an email service staff of the ISP. I have confirmed that mail.biglobe.ne.jp is the proper POP/SMTP server name:
http://support.biglobe.ne.jp/settei/mailer/win-tb/tb_001.html

The existing support pages for other domains (memail.jp, eeyo.jp, etc.) say those POP/SMTP servers are mail.memail.jp and mail.eeyo.jp, but Biglobe's staff have said the descriptions would be updated. mail.biglobe.ne.jp, mail.memail.jp and mail.eeyo.jp are all the same server anyway.
http://email.biglobe.ne.jp/myadr/set.html
http://email.biglobe.ne.jp/dokodemo/set2.html

> So, I'm going to say r=me, as long as you promise that
> "test.test.biglobe.ne.jp" is _actually_ a real server that they use for
> something other than testing.  :)

Ha Ha, I haven't noticed there is a such subdomain in the list. Maybe an extra bonus!
Checked in the Biglobe configurations at r62236.

Their support pages are available at
http://support.biglobe.ne.jp/settei/mailer/win-tb/
http://support.biglobe.ne.jp/settei/mailer/mac-tb/
Attached patch Bug 546278 workaround (obsolete) — Splinter Review
Until Bug 546278 is fixed, we should use the real hostnames instead of %EMAILDOMAIN% variables.
Attachment #427306 - Flags: review?(bwinton)
Attachment #427306 - Attachment is obsolete: true
Attachment #427306 - Flags: review?(bwinton)
Comment on attachment 427306 [details] [diff] [review]
Bug 546278 workaround

I just posted patches against Bug 534604 and Bug 546278. Hope those will be landed to the 1.9.1 branch aka Thunderbird 3.0.3. Cleared r? flag.
Biglobe now has 1400 (!) domains in our database.
I think that is unreasonable.

Many of these are even vanity domains like "cancer.", "milk.". Some German hosters also have vanity domains like "want-to-wake-up-with-you.de" (in German), "gotaquestion.de" and stuff like this. Total nonsense. Almost nobody is using them. I intentionally did not add them.
If one ISP has 1400 domains, I want to see prove that they are actually used by more than a few hundred users and justify the cost for us, like the slowness that all svn operations now take. Nevermind the inability to browse the list of domains anymore.
I request that the 1400 domains from Biglobe are removed again, please.
Certainly 1,400 domains are too many, but the solutions -- dynamic ISPDB nor fetchConfigFromISP -- have not been implemented at this time, unfortunately. I believe hosting static XML files on the live.momo server is temporary.

Biglobe is the #5 ISP here in Japan, so I guess each domain has hundreds of customers.
> the solutions -- dynamic ISPDB nor
> fetchConfigFromISP -- have not been implemented at this time

Ah, yes, I am working on that, in fact just waiting for ui-review and commit approval.

> static XML files [for the many domains] on the live.momo server is temporary.

OK, great. Thanks!
Now that we have "fetch from ISP", I filed bug 556655 about this (comment 82).
Whiteboard: [config]
Summary: Add Japanese ISP configurations → [config] Add Japanese ISP configurations
Closing this bug; we'll file a new bug on demand.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago9 years ago
Resolution: --- → FIXED
Component: ispdb → ISPDB Database Entries
Product: Mozilla Messaging → Webtools
You need to log in before you can comment on or make changes to this bug.