512247 - Remove license notice from js test files

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Jason Orendorff [:jorendorff]]

The license notices are 25% of the tests. It's crazy.

Comment 1

[Jason Orendorff [:jorendorff]]

Attachment: v1

This patch is automatically generated and it is large.

$ grep '^-' rm-test-licenses.patch | grep -v '^--- ' | wc
  109621 1050875 5378041

So the license notices take up about 5MB on disk.  That's a quarter of the total size of the js/tests suite.

A simple way to check this patch is just to run the tests. This patch
causes failures in two tests that test Error object line numbers. I have a
quick follow-up patch to fix those.

But many tests fail in trunk js or are spidermonkey-n tests, so it's
important to check the patch in other ways.

One way is to look at the lines it removes.
$ grep '^-' rm-test-licenses.patch | grep -v '^--- ' | LC_ALL=C sort | LC_ALL=C uniq

The only surprises I found were a few existing typos in the tests.

In js1_8_1/regress/regress-452498-117.js:
>-03/* -*- Mode: C++; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
>+03

(A typo. Looks like my script removed the modeline but left the
typo. I'll remove it in the follow-up patch.)

In js/tests/js1_7/extensions/regress-346642-06.js:
>- * the provisions above, a recipient may use your version of this file under    let (x=3) ((++x)())
>-
>- * the terms of any one of the MPL, the GPL or the LGPL.

In js/tests/js1_5/Array/regress-101964.js:
>- * ***** END LICENSE BLOCK ***** /


Another way is to check the lines it adds.
  $ grep '^\+' rm-test-licenses.patch | grep -v '+++' 
  +/* 
  +/*  */
  +03
  +
  +

(That "03" is the typo mentioned above.)


Another way is to check where within each test file the edits are.
$ grep @@ rm-test-licenses.patch | sort | uniq -c
   7 @@ -1,36 +0,0 @@
   7 @@ -1,37 +0,0 @@
   2 @@ -1,39 +1,2 @@
   2 @@ -1,39 +1,3 @@
   1 @@ -1,39 +1,5 @@
   1 @@ -1,40 +1,1 @@
   2 @@ -1,40 +1,2 @@
1460 @@ -1,40 +1,3 @@
   1 @@ -1,40 +1,5 @@
   1 @@ -1,41 +1,2 @@
1219 @@ -1,41 +1,3 @@
  32 @@ -1,42 +1,3 @@
  40 @@ -1,43 +1,3 @@
   9 @@ -1,44 +1,3 @@
 116 @@ -1,45 +1,3 @@
   5 @@ -1,46 +1,3 @@
   2 @@ -1,48 +1,3 @@
   1 @@ -217,7 +172,7 @@ function check(desc, actual, expected, o

They differ because the license blocks have different numbers of Contributor(s).

That last one is an emacs mode line in the middle of a file.

Comment 2

[Bob Clary [:bc] (inactive)]

gerv?

Comment 3

[Jason Orendorff [:jorendorff]]

Attachment: fixups v1

After applying v1, apply this.

(They're two separate patches because the gigantic one is entirely autogenerated and I don't expect anyone to review a 6MB patch line by line.)

Comment 4

[Jason Orendorff [:jorendorff]]

Oh, I forgot to mention.

I also need to add a license.txt file to js/src.  I have all the Contributor(s) lines scraped out of the tests, just need to throw it together.  I'll get it in this afternoon, I promise.

Comment 5

[Jason Orendorff [:jorendorff]]

Attachment: license.txt v1

Comment 6

[Bob Clary [:bc] (inactive)]

In general I'm ok with removing the license boiler plate, but these go back to the original check ins circa 1998 and are not something I decided upon. I want someone who knows more about the licensing requirements to make the decision about whether it is appropriate or not.

As for the license.txt file, I would really rather not have to maintain such a monolithic list of names/emails. If we go this route, lets just drop attribution altogether.

Comment 7

[Jason Orendorff [:jorendorff]]

(In reply to comment #6)
> As for the license.txt file, I would really rather not have to maintain such a
> monolithic list of names/emails. If we go this route, lets just drop
> attribution altogether.

I completely agree. Gerv, do we have to keep the list of contributors? Is it ok to just keep a single list (in the license file) of all the contributors, without the filenames they contributed to?

Comment 8

[Blake Kaplan (:mrbkap) (inactive)]

Could we keep a small license block in the files like:

See js/tests/license.txt for license information.
This file was contributed by: A, B, C

?

Comment 9

[Gervase Markham [:gerv]]

Whoa, there.

The MPL section 3.5 says:

"You must duplicate the notice in Exhibit A in each file of the Source Code. If it is not possible to put such notice in a particular Source Code file due to its structure, then You must include such notice in a location (such as a relevant directory) where a user would be likely to look for such a notice."

There are no structural restrictions here, so we need to keep the notices. Yes, not all files in the repo have them, but that's no reason to take steps backwards.

This bug opened with:

> The license notices are 25% of the tests. It's crazy.

Crazy in what sense? More space on disk? Hardly noticeable. More download time? Everything is compressed these days, and most of the blocks are identical. 

I agree that if we rewrote the MPL, we might be more flexible in this area. Hindsight is great. But I don't see it as a massive burden. Or have I missed something?

Gerv

Comment 10

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

25% crap that people have to read and scroll through when working with or analyzing the tests.  If it's easier, we can just relicense all the Mozilla-owned tests (approximately 98% of them, I would guess) to something other than MPL, so that we can get rid of the cruft.  We don't care about getting changes to existing tests back very much at all, and the MPL doesn't compel the new test files that we might actually find interesting.  The license requirements here are solving a non-problem, at non-zero cost to our actual work.

Alternatively, we could just move them to the end, but then "cat test.js" becomes useless as well on most terminals.

That you don't see it as a burden is likely due to the fact that you don't have to wade through it every time you're diagnosing a test failure, or get false hits on words in it when searching the test repository. :-P

Comment 11

[Gervase Markham [:gerv]]

:-P

Last time this came up, the solution was as codified in the current licensing policy
http://www.mozilla.org/MPL/license-policy.html
That is, we approved the use of:

Any copyright is dedicated to the Public Domain.
http://creativecommons.org/licenses/publicdomain/

If you know all the contributors, feel free to relicense individual tests. And feel free to encourage people to do this for new tests.

Gerv

Comment 12

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Should be easy.

I would be surprised if we had many JS tests that listed any contributors other than Mozilla employees and contractors; I would not surprised if we had none.

Up for a big bang, jorendorff?

Comment 13

[Brendan Eich [:brendan]]

Honestly I can think of many more important things for jorendorff to work on than finding non-Mozilla-or-Netscape-employed test contributors (there have been more than a few over the years), getting their permission to relicense PD, and doing the grunt work. *Many* more important things.

/be

Comment 14

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

I am just looking to go by the set of people who have been listed as Contributors in the files, not doing archaeology.  For these trivial bits of code, I don't think that additions that the authors didn't feel warranted following the letter of the license (and adding their names to the header) should impede such relicensing, but I'll check with counsel.  Ones that have other contributors we can leave with the noisy boilerplate, since there will I suspect be few.

If nothing else, we can use the cleaner text for *future* tests.

Comment 15

[Brendan Eich [:brendan]]

Yes, PD for all new tests sounds great -- maximizes sharing upside, minimizes the overhead/boilerplate.

/be

Comment 16

[Jason Orendorff [:jorendorff]]

(In reply to comment #14)
> [...] I'll check with counsel.

OK, I'll wait for this.  Whatever the rules are, implementing them is the trivial part.

Comment 17

[Robert Sayre]

Comment on attachment 396361 [details]
license.txt v1

(In reply to comment #9)
>
> There are no structural restrictions here, so we need to keep the notices.

The structural restriction is obvious. It hurts our testability on the code we do ship, since we do tons of unnecessary i/o to run the test suite (under valgrind, too). So, the properly licensed code we do ship is hurt by the over-licensed code we don't ship. Gerv, please r+ this.

Comment 18

[Gervase Markham [:gerv]]

Hang on, this isn't what I thought we'd agreed... I thought the plan was to relicense the tests as PD. If we stick with the MPL, then my objection in the first two paras of comment #9 still applies. I'd love it if the MPL allowed externalizing license information like this, but it doesn't. (One more thing to fix, if and when we ever get around to it.)

If we are going to relicense them as PD, then we need to put the PD declaration from the license policy at the top of each, and make sure we have permission from all non-Mozilla contributors. (assuming brendan and shaver are happy that Mozilla can speak for 'Netscape' in this matter).

As a third option, either of those gentlemen could declare that we are ignoring my comment #9 reading of the MPL, and we could go ahead with the current patch.

Gerv

Comment 19

[Robert Sayre]

(In reply to comment #18)
> I'd love it if the MPL allowed
> externalizing license information like this, but it doesn't.

It does! It says it right in the paragraph you quoted. :)

Comment 20

[Gervase Markham [:gerv]]

No, it doesn't. It only allows you to put the notice in the directory if it's not possible to put it in the file due to its structure. That is clearly not the case with these files, because that's where the notice is now! :-)

Files which you might not be able to put it in due to structure might be e.g. PNGs or GIFs.

Gerv

Comment 21

[Bob Clary [:bc] (inactive)]

Attachment: replace license boilerplate with public domain in template.js

Lets stop making it worse by replacing the template.js license boilerplate with the pd declaration.

How about I contact everyone I can in the contributors list and ask them if we can replace the license with the pd declaration. Although the older tests will still be left with license boilerplate, pretty much anything added in the last 4-5 years will be able to be released under the pd with the listed contributor's permission.

For example:

jorendorff: y/n ?
brendan   : y/n ?
mrbkap    : y/n ?
sayrer    : y/n ?

etc.

Comment 22

[Jason Orendorff [:jorendorff]]

Comment on attachment 407840 [details] [diff] [review]
replace license boilerplate with public domain in template.js

Two thumbs up.

And yes, please feel free to relicense any test contributions of mine with the public domain notice.

Comment 23

[Blake Kaplan (:mrbkap) (inactive)]

I, too, am fine (happy, even) seeing my test contributions in the public domain.

Comment 24

[Robert Sayre]

yeah public domain for my stuff is the way to go.

Though we're only having this discussion because of amateur lawyering regarding files we don't ship in Firefox. Waste of time.

Comment 25

[Mitchell Baker]

Rob:  if by "amateur lawyering" you mean Gerv (or maybe in any case) your comment is not remotely helpful.

Assume that a professional lawyer says the same thing -- that doesn't make the problem go away. 

And your assertion that this is the *only* reason for such a discussion strikes me as plain wrong.  As well as nasty.

Comment 26

[Mitchell Baker]

The long term solution is to update the MPL; there's a bunch of language that we can do without.  That will take a while though, it involves the open source licensing community and re-certification.  Makes sense to me to use another method for these test cases.

Comment 27

[Robert Sayre]

(In reply to comment #25)
> 
> Assume that a professional lawyer says the same thing -- that doesn't make the
> problem go away. 

It's not clear to me that there is a problem.

> And your assertion that this is the *only* reason for such a discussion strikes
> me as plain wrong.  As well as nasty.

I've never seen a project license every test, including Mozilla. So, I'm not sure what our goal is. For example, source/toolkit/components/feeds/test/xml/ is full of tests I wrote that aren't licensed per-file. While I would be happy to license them any which way, I have to wonder what the value of undertaking such an effort would be.

Comment 28

[Mitchell Baker]

it sounds like the problem is to much license text in the files for ease of use.  The public domain solution reduces that.  Saying nothing also does that, but doesn't tell others if they can reuse them.   In other places where the goal is maximum reuse we've used a BSD or Apache license.  Those may have text requirements as well.

Comment 29

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

I talked to our counsel about it, and I thought commented in here with the response -- apparently not.  I'm pretty sure that the recommendation was that we were clear to make stuff PD if we checked with people in the Contributors: line, but I'll check again.

We aren't in compliance with the letter of the MPL universally (sec 3.3, f.e., which I believe would require us to have a ridiculously huge changelog accompanying every chunk that might be considered different Covered Code!) and I don't think that moving to license-by-reference would be likely to cause us problems.  I didn't check that specifically with counsel, but I will do so if I remember in the morning, since it would let us reduce license noise a lot.

BSD and Apache do both have text requirements, at least from my reading; PD is fine here by my reading of our license policies, so we should go ahead with it.  I certainly grant permission for my own contributions to be thusly relicensed, if there are any in the test suite not covered by the Mozilla copyright.

(The "problem" occurs when strict reading of license requirements is treated as a *barrier* to action, rather than something to be weighed against gains (incl practicality and productivity).  My experience is that professional lawyers are more likely to say "this is the exposure/liability" and amateurs more likely to say "you cannot do X", where there is a difference between the two groups.)

Comment 30

[Bob Clary [:bc] (inactive)]

I am responsible for the license text in almost all tests added since early 2005 up until recently. I just followed the pattern already in use. Early on I got explicit permission to add the test to the tree from the person who discovered the basic test but stopped doing that after N+1 tests. I added the person responsible for discovering the test to the contributor line when I could probably have done without. Sorry for making the issue worse, but as I said. I can fix all of the tests from active contributors and most since I started doing this.

Comment 31

[Gervase Markham [:gerv]]

shaver: I think we've generally thought that CVS or Hg logs are what we do to meet the requirement of sentence 1 of 3.3. And, as you can see from about:licence, I go to a fair amount of trouble to meet the requirement in sentence 2. Even if part of the motivation is showing that it really is unnecessary.

Your last paragraph is a fair and observant criticism, although the amateurs don't have professional liability insurance ;-)

Gerv

Comment 32

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

(In reply to comment #31)
> shaver: I think we've generally thought that CVS or Hg logs are what we do to
> meet the requirement of sentence 1 of 3.3.

I have trouble reconciling these positions, I admit:

- it's OK to replace a requirement to document changes in the source tree _and_ make clear mention of the modification in our executable with "go to another service that we don't even document to find out what changed, when those changes might not even apply to the source you have"; but
- it's NOT OK to include the licensing information for tests by reference to a file that's in the same directory.

Not that I think we should do more to comply with the letter of 3.3 (I don't even really want to know what it means in the context of our mixed-contribution environment), I just think the proposal here is a lot less of a divergence from the LETTER OF THE LAW than is our behaviour WRT 3.3.

Comment 33

[Gervase Markham [:gerv]]

shaver: another reasonable point. I should make it clear that if you, Mitchell or any of our lawyers think that license-by-reference is cool, then I'm not going to argue. Although wide use of it would give us a question about what to do with Contributors: lines.

Gerv

Comment 34

[Blake Kaplan (:mrbkap) (inactive)]

Would it be possible to use the CVS/hg/svn log as the definitive changelog/authors log?

Comment 36

[Bob Clary [:bc] (inactive)]

http://hg.mozilla.org/tracemonkey/rev/c7e7f1c963f6
replace license headers in template.js with public domain notice. I added a line for Contributor.

http://hg.mozilla.org/tracemonkey/rev/8cdade53b923
replace license headers in tests by Jason Orendorff, Blake Kaplan, Robert Sayre, Bob Clary with public domain notice; by permission.

brendan, shaver, igor, jesse and gary: you're next. Do I have permission to replace the license boilerplate for the tests where you are listed as contributors with the public domain grant?

Comment 37

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Sure.

Comment 38

[Brendan Eich [:brendan]]

Yes!

/be

Comment 39

[Jesse Ruderman]

Go for it.

Comment 40

[Nochum Sossonko [:Natch]]

These bugs are all part of a search I made for js bugs that are getting lost in transit:

http://tinyurl.com/jsDeadEndBugs

They all have a review+'ed, non-obsoleted patch and are not marked fixed-in-tracemonkey or checkin-needed but have not seen any activity in 300 days. Some of these got lost simply because the assignee/patch provider never requested a checkin, or just because they were forgotten about.

Comment 41

[Bob Clary [:bc] (inactive)]

Gerv removed most of the original license block text in Bug 716478. Calling this fixed.