779845 - Inconsistencies in the PSL test data and Example

Status: RESOLVED FIXED

(Core Graveyard :: Networking: Domain Lists, defect)

Description

[Rob Stradling]

The changes to the .jp PSL rules in Bug 763208 invalidated some of the test data [1] and parts of the Example [2].

1. I suggest (as mentioned in Bug 763208 comment 9) replacing the .jp section of the test data as follows:
# More complex TLD.
checkPublicSuffix('jp', NULL);
checkPublicSuffix('test.jp', 'test.jp');
checkPublicSuffix('www.test.jp', 'test.jp');
checkPublicSuffix('ac.jp', NULL);
checkPublicSuffix('test.ac.jp', 'test.ac.jp');
checkPublicSuffix('www.test.ac.jp', 'test.ac.jp');
checkPublicSuffix('kyoto.jp', NULL);
checkPublicSuffix('test.kyoto.jp', 'test.kyoto.jp');
checkPublicSuffix('ide.kyoto.jp', NULL);
checkPublicSuffix('b.ide.kyoto.jp', 'b.ide.kyoto.jp');
checkPublicSuffix('a.b.ide.kyoto.jp', 'b.ide.kyoto.jp');
checkPublicSuffix('c.kobe.jp', NULL);
checkPublicSuffix('b.c.kobe.jp', 'b.c.kobe.jp');
checkPublicSuffix('a.b.c.kobe.jp', 'b.c.kobe.jp');
checkPublicSuffix('city.kobe.jp', 'city.kobe.jp');
checkPublicSuffix('www.city.kobe.jp', 'city.kobe.jp');

2. Petar Bogdanovic noticed (in Bug 763208 comment 10) that the "Unlisted TLD" test data is actually invalid, because 'If no rules match, the prevailing rule is "*" '.  So the following test data should be either removed or fixed:
# Unlisted TLD.
checkPublicSuffix('example', NULL);
checkPublicSuffix('example.example', NULL);
checkPublicSuffix('b.example.example', NULL);
checkPublicSuffix('a.b.example.example', NULL);

3. Also, I think the following test data could also be removed:
# Listed, but non-Internet, TLD.
#checkPublicSuffix('local', NULL);
#checkPublicSuffix('example.local', NULL);
#checkPublicSuffix('b.example.local', NULL);
#checkPublicSuffix('a.b.example.local', NULL);

4. I'll defer to Gerv to come up with a fix for the .jp parts of the Example.


[1] http://publicsuffix.org/list/test.txt
[2] http://publicsuffix.org/list/

Comment 1

[Masatoshi Kimura [:emk]]

Attachment: Part 1: Import PSL test from http://publicsuffix.org/list/test.txt

I'll incorporate the test into our tree to ensure the test is up-to-date.

Comment 2

[Masatoshi Kimura [:emk]]

Attachment: Part 2: Update the PSL test to catch up with the .jp PSL update and fix test's bug about unlisted TLD

I set the user name of these two patches to Rob Stradling because they do not contain my creative part at all. Rob, please verify the test is imported correctly and your name and mail address is correct. They will be displayed on the commit log.
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml

Comment 3

[Masatoshi Kimura [:emk]]

Attachment: Part 3: Incorporate the PSL test into our test system

Gerv, please forward to a appropriate reviewer if you are not.

Comment 4

[Gervase Markham [:gerv]]

Comment on attachment 651320 [details] [diff] [review]
Part 1: Import PSL test from http://publicsuffix.org/list/test.txt

r=gerv on the idea of having these tests as part of our build tree.

Gerv

Comment 5

[Gervase Markham [:gerv]]

Comment on attachment 651321 [details] [diff] [review]
Part 2: Update the PSL test to catch up with the .jp PSL update and fix test's bug about unlisted TLD

r=gerv on fixes.

Gerv

Comment 6

[Gervase Markham [:gerv]]

Comment on attachment 651322 [details] [diff] [review]
Part 3: Incorporate the PSL test into our test system

Brian: as a member of the networking team, can you rubber-stamp this?

Thanks :-)

Gerv

Comment 7

[Rob Stradling]

(In reply to Gervase Markham [:gerv] from comment #4)
> Comment on attachment 651320 [details] [diff] [review]
> Part 1: Import PSL test from http://publicsuffix.org/list/test.txt
> 
> r=gerv on the idea of having these tests as part of our build tree.

+1, although I suggest renaming test.txt as part of the import.  How about netwerk/test/unit/data/test_psl.txt ?

Comment 8

[Masatoshi Kimura [:emk]]

Attachment: Part 1: Import PSL test from <http://publicsuffix.org/list/test.txt>. r=gerv, feedback=rob

(In reply to Rob Stradling from comment #7)
> +1, although I suggest renaming test.txt as part of the import.  How about
> netwerk/test/unit/data/test_psl.txt ?
Fair enough. I'll attach updated patches.

Comment 9

[Masatoshi Kimura [:emk]]

Attachment: Part 2: Update the PSL test to catch up with the .jp PSL update and fix test's bug about unlisted TLD. r=gerv, feedback=rob

Comment 10

[Masatoshi Kimura [:emk]]

Attachment: Part 3: Incorporate the PSL test into our test system

Comment 11

[Masatoshi Kimura [:emk]]

ping?

Comment 12

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Comment on attachment 651433 [details] [diff] [review]
Part 3: Incorporate the PSL test into our test system

LGTM.

Comment 13

[Gervase Markham [:gerv]]

Pushed to mozilla-inbound:

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/44f88d64b116
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/d3a2125835be
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/8406983b121c

Gerv

Comment 14

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

It's buuuurning

https://hg.mozilla.org/integration/mozilla-inbound/rev/46365c62935c

Comment 15

[Gervase Markham [:gerv]]

Looks like this is my screw-up; I forgot to hg add a file. :-|

Gerv

Comment 16

[Gervase Markham [:gerv]]

Attempt 2:

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/1bd1a16af051
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/1ec081fd2580
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/3e1fa8a59f8d

Gerv

Comment 17

[Masatoshi Kimura [:emk]]

Shouldn't be closed until http://publicsuffix.org/list/ is updated so that it points the test data on our tree.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/1bd1a16af051
https://hg.mozilla.org/mozilla-central/rev/1ec081fd2580
https://hg.mozilla.org/mozilla-central/rev/3e1fa8a59f8d

Comment 19

[Masatoshi Kimura [:emk]]

Please update a link on [1] from [2] to [3] (or put a redirect on [2]).
[1] http://publicsuffix.org/list/
[2] http://publicsuffix.org/list/test.txt
[3] http://mxr.mozilla.org/mozilla-central/source/netwerk/test/unit/data/test_psl.txt?raw=1

Comment 20

[Gervase Markham [:gerv]]

emk: Done.

Gerv