987668 - Add hyphenation resources for Polish

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, enhancement)

Description

[Mike de Boer [:mikedeboer]]

Polish hyphenation resources are available here: http://mirrors.ctan.org/language/hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-pl.tex

The license details suggest public domain/ non license encumbered, but I'm asking Gerv nonetheless.

Comment 1

[Mike de Boer [:mikedeboer]]

Gerv, what do you make of the license/ copyright information as stated in the linked file? Can we use it to include in the tree and if not, what needs to be changed to be able to add it?

Comment 2

[Mike de Boer [:mikedeboer]]

FWIW, the French hyphenation rules seem to have the same text wrt licensing: https://mxr.mozilla.org/mozilla-central/source/intl/locales/fr/hyphenation/LICENSE

Comment 3

[Gervase Markham [:gerv]]

<sigh>

The license gives us a right to "do with this file whatever needs to be done" as long as it's "for the sake of a better world". I hereby declare that I believe that for the sake of a better world, what we need to do is treat this file as if it's under a free-software-compatible liberal license which requires the preservation of license and copyright notices ("respects the copyright") but has no other requirements of any kind, including usage restrictions.

Let us proceed on that basis.

Gerv

Comment 4

[Gervase Markham [:gerv]]

I further declare that this reasoning applies to all files with an equivalent licensing statement. We are doing a lot for the sake of a better world today :-)

Gerv

Comment 5

[Jonathan Kew [:jfkthame]]

(In reply to Mike de Boer [:mikedeboer] from comment #2)
> FWIW, the French hyphenation rules seem to have the same text wrt licensing:
> https://mxr.mozilla.org/mozilla-central/source/intl/locales/fr/hyphenation/
> LICENSE

The situation is slightly different for the French patterns, I think, in that the original French file stated:

83 % This file is available for free and can used and redistributed
84 % asis for free. Modified versions should have another name.

which, although over-simplified by today's standards, is clearly intended to allow free use, redistribution, and modification (provided the file name is changed).


The Polish file does not include any such statement from the original authors; it gives a history of their development, but nothing explicit regarding permission. There's a hint in the comment:

% to be used for the inclusion of Polish hyphenation patterns in any format,
% not necessarily in the MeX or LaMeX ones.

that IMO suggests the authors intended their work to be widely useful and not restricted to whatever specific systems they were aware of at the time, but that's about all we have to go on.

The "do with this file..." paragraph was added during adaptation for newer TeX systems and formats, and intends to allow such work to continue as needed, but does not in itself constitute a license for the pattern data.

Comment 6

[Mike de Boer [:mikedeboer]]

(In reply to Jonathan Kew (:jfkthame) from comment #5)
> The "do with this file..." paragraph was added during adaptation for newer
> TeX systems and formats, and intends to allow such work to continue as
> needed, but does not in itself constitute a license for the pattern data.

But as we nothing more to go on and as Gerv pointed out in comment 3, the text "has no other requirements of any kind, including usage restrictions."

I think that comment 3 still stands regardless, and we're ok with including the Polish pattern data. To be safe, n-i'ing Gerv again to reply with comment 5 taken into consideration.

Comment 7

[:Gijs (he/him)]

(In reply to Mike de Boer [:mikedeboer] from comment #6)
> (In reply to Jonathan Kew (:jfkthame) from comment #5)
> > The "do with this file..." paragraph was added during adaptation for newer
> > TeX systems and formats, and intends to allow such work to continue as
> > needed, but does not in itself constitute a license for the pattern data.
> 
> But as we nothing more to go on and as Gerv pointed out in comment 3, the
> text "has no other requirements of any kind, including usage restrictions."
> 
> I think that comment 3 still stands regardless, and we're ok with including
> the Polish pattern data. To be safe, n-i'ing Gerv again to reply with
> comment 5 taken into consideration.

Actually setting needinfo flag now. :-)

Comment 8

[Gervase Markham [:gerv]]

:mikedeboer: you are slightly misinterpreting my words, but never mind.

I think that in the case of the Polish patterns, the history shows that the author intended to allow modification - because after the original person wrote it, different people modified it, and then the original person joined back in again, thereby implicitly endorsing their modifying of his original file. That, together with the "use it anywhere" statement Jonathan refers to, suggests to me that it implies the rights we need.

So you should go ahead and use it.

Gerv

Comment 9

[Mike de Boer [:mikedeboer]]

(In reply to Gervase Markham [:gerv] from comment #8)
> :mikedeboer: you are slightly misinterpreting my words, but never mind.

Whoops! Not my intention, apologies.

Jonathan, would you like to have the honor of adding this locale? I'm just asking because you're quite familiar with the process, as in you did this before. I can also try, but I'm just thinking of the most efficient route...

Comment 10

[Jonathan Kew [:jfkthame]]

OK, I'll take this, and try to put a patch up soon.

(Note that patterns from the TeX world need some preprocessing before we use them, otherwise they won't work as intended. There's information about this in the various READMEs in the libhyphen package; in particular, we need to apply the "substrings" tool there.)

Comment 11

[Jonathan Kew [:jfkthame]]

Attachment: add hyphenation patterns for Polish.

Comment 12

[Jonathan Kew [:jfkthame]]

Attachment: reftest for Polish hyphenation.

Comment 13

[Jonathan Kew [:jfkthame]]

Comment on attachment 8397279 [details] [diff] [review]
add hyphenation patterns for Polish.

Adding r?gerv to check the LICENSE file seems reasonable for this case.

Comment 14

[Jonathan Kew [:jfkthame]]

Tryserver says that the reftest fails (with just a few differing pixels) on several platforms: https://tbpl.mozilla.org/?tree=Try&rev=917fdfb10372.

This is happening because there's kerning between <letter><hyphen> pairs in certain cases, which affects positioning of the hyphen in the reference file; but we fail to apply the kerning on automatic hyphens (or those resulting from &shy;). I'll file a separate bug about that. We can work around the issue here by simply disabling kerning altogether in this testcase.

Pushed a new try run, to check this works as expected: https://tbpl.mozilla.org/?tree=Try&rev=15684681dfcc.

Comment 15

[Jonathan Kew [:jfkthame]]

Gerv, could you confirm you're happy with the simple LICENSE file here? Is there any further explanation or clarification we should give?

Thanks!

Comment 16

[Gervase Markham [:gerv]]

Please add a sentence to the part at the top which you wrote:

"The modifiers of this file give you the right to use, modify and distribute their changes in any way and for any purpose."

That way, we aren't compounding any problem there might be.

Gerv

Comment 17

[Gervase Markham [:gerv]]

Comment on attachment 8397279 [details] [diff] [review]
add hyphenation patterns for Polish.

r=gerv with suggested change.

Gerv

Comment 18

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e713344051f0
https://hg.mozilla.org/integration/mozilla-inbound/rev/5a58c89a71a3

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e713344051f0
https://hg.mozilla.org/mozilla-central/rev/5a58c89a71a3

Comment 20

[Jean-Yves Perrier [:teoli]]

Doc updated:
https://developer.mozilla.org/en-US/Firefox/Releases/31
and
https://developer.mozilla.org/en-US/docs/Web/CSS/hyphens