506082 - New top-level repository: nanojit-central, plus tinderboxes

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Graydon Hoare :graydon]

We're in the process of promoting nanojit to a top-level project with an independent makefile, testsuite, set of tinderboxes, checkin policy, etc.

I'd like to start this off by getting the repository seeded and, if possible, tinderboxes running for x86-linux, arm-linux, x86-osx, and x86-win32.

The repository will require the same sort of push-hook we have on other repositories, such that it enforces single-headed pushes only. Initial push permission set should include everyone who can push to tamarin-redux and everyone who can push to tracemonkey.

If the tinderboxes require separate bugs, that's fine, I'll file those; just focus this one on getting the repository going. I'll push the initial history to it whenever it's active.

Comment 1

[Reed Loden [:reed]]

You should file a bug against Release Engineering for the tinderboxes. IT only deals with the Hg repository in this case.

http://hg.mozilla.org/nanojit-central/ ready to go. _Anybody_ with Hg access (of any form) can commit to it.

Comment 2

[Graydon Hoare :graydon]

Will file a releng bug for tinderboxes, thanks. Curious about a couple unaddressed points:

  - Can the commit rights be tightened as per the initial request? I got the impression we had management on a repo-by-repo basis. All hg writers is too broad.

  - More pressingly: is there, or can there be, a push-hook to require single-headedness? I'm not sure if this is the default on all such repositories, but it's something all teams involved want to try.

This is a somewhat awkward / delicate task (sharing code with Adobe, automatically transplanting revisions into two downstream projects on a fine-grained basis) and much of the goal here is to mess with policy to try to get the process running smoothly.

Comment 3

[Reed Loden [:reed]]

(In reply to comment #2)
>   - Can the commit rights be tightened as per the initial request? I got the
> impression we had management on a repo-by-repo basis. All hg writers is too
> broad.

We have no such group that encompasses all requested people. If you get me a list of people, I can create a group and put them in it. Note that both tamarin-redux and tracemonkey both are currently wide open to anybody with an Hg account.

>   - More pressingly: is there, or can there be, a push-hook to require
> single-headedness? I'm not sure if this is the default on all such
> repositories, but it's something all teams involved want to try.

I already added that, as per your original request in comment #0.

Comment 4

[Gervase Markham [:gerv]]

graydon: the knob we currently have is "mozilla-central committers". If nanojit-central code is going to be merged into mozilla-central without close review at merge time, then it seems to make sense to restrict it to this set of people also. Would that exclude some of our external collaborators at Adobe?

As Reed says, tamarin-redux and tracemonkey are currently both open to anyone with Hg access. Were you under the impression that they were restricted to a more limited set? If so, who?

Gerv

Comment 5

[Brendan Eich [:brendan]]

Nanojit is open source -- it isn't just "us" and "our colleagues at Adobe". There are people at Intel. There are volunteers.

/be

Comment 6

[Gervase Markham [:gerv]]

Right. I didn't mean to be exclusive. Let me rephrase the two questions in the most general way:

"If we locked down nanojit-central to the group of people who have mozilla-central access, would we exclude anyone we don't want to exclude?"
and conversely:
"If we don't lock down nanojit-central to the group of people who have mozilla-central access, doesn't that mean that people without the mozilla-central "trust level" or whatever you call it will have the ability to influence the mozilla-central tree?"

I'm currently working for Mitchell on harmonizing our commit access policies, and one principle I think is wise, and which is in the draft, is that all trees from which code is merged to mozilla-central without careful merge-time review should be restricted to the same set of committers that mozilla-central is. Otherwise, restricting mozilla-central starts to mean less. Does that sound like a reasonable principle?

Gerv

Comment 7

[Brendan Eich [:brendan]]

(In reply to comment #6)
> I'm currently working for Mitchell on harmonizing our commit access policies,
> and one principle I think is wise, and which is in the draft, is that all trees
> from which code is merged to mozilla-central without careful merge-time review
> should be restricted to the same set of committers that mozilla-central is.

That does not describe nanojit / m-c because graydon et al. do careful review as they go in nj and therefore need not merging back into m-c.

> Otherwise, restricting mozilla-central starts to mean less. Does that sound
> like a reasonable principle?

It sounds like it requires redundant review. If you want graydon, et al., to review changes going from nj into m-c that they did not already review, then it makes more sense. I'm not sure how to formalize this.

I suspect graydon et al. (sorry, tedious and I should cite sewardj and njn in turn, others too) will re-review anyway. I'm sure they'll speak up. But I don't believe we can formalize this in some airtight way, except to require review by m-c committers at some point in the lifecycle of a patch that makes it into m-c.

/be

Comment 8

[Mike Shaver (:shaver emeritus)]

I think you are combining unlike things.  Someone merging code from nanojit or tracemonkey to mozilla-central will be in a "safer" position than someone committing a patch attached to bugzilla: the patch has already been integrated and run through various of our tests, in addition to whatever review is necessary to get into the subsidiary tree.  Restricting mozilla-central is at least as much about understanding of our tree rules and test requirements as about whether someone's code needs review.  All code needs review, and we should trust module owners to ensure that their modules are maintained in keeping with the process and quality standards for our project.

This is similar to what we do with upstream libraries like cairo, sqlite, even NSS -- no additional merge-time review requirement is imposed.  That we host nanojit-central in hg instead of putting it on github or google code shouldn't make it _harder_ for that code to get in -- but if we make it onerous to put the code where we can easily pull it to our world then people will just do that, and Mozilla will be less productive for it.

Comment 9

[Gervase Markham [:gerv]]

OK... I'll outline the scenario I'm worried about, and you can tell me if I don't understand the situation and the tools.

1) Person X applies for Hg access to work on e.g. contribute.mozilla.org website. This requires one voucher, who may well be someone who's just glad of the help. And person X may well be helpful. Point is, the requirements for getting some sort of access to Hg are much less than those for m-c access.
 
2) Person X checks in malicious code to a non-m-c tree which gets merged to m-c. Because those trees can be less in the glare of publicity (pushlog, tinderbox etc.), no-one notices the checkin, or perhaps they don't notice the malicious part of it.

3) The code eventually gets merged across to m-c.

If you think 2) is impossible, then OK. I'll take off my tinfoil hat. Another formulation might be that we should restrict to m-c committers those trees from which code is directly pulled for the products. Would that be just m-c, c-c, m-1.9.1 and other branches?

And if that's what we are doing, what sort of tightening is Graydon looking for in comment #2, and why?

Gerv

Comment 10

[[e-mail] (:bhearsum)]

Can we move this discussion to somewhere else?

Comment 11

[Graydon Hoare :graydon]

Heh. We have stumbled into the dreaded bog of "transitive trust attacks", that neither hg nor git built any architecture to help mitigate. It's OK though; most of the conventional (tarball and patch) world of free software is vulnerable too.

Anyway, the plan is to semi-automatically import from nanojit-central to tracemonkey and hence mozilla-central, with only minimal "batch" re-review at the point of importing. So if you're worried about this, tighten up that tinfoil hat. It's vulnerability city!

This avenue of attack is IMO comparable to upstream code we "blindly" import. I'm pretty sure we don't (re-)audit all the changesets in sqlite when we pull in updated copies. A bad sqlite version could easily carry a firefox-specific attack. We trust "their" development community to be self-regulating and non-malicious when we copy in their code. Possibly misplaced trust, but there you have it. We depend on lots of libraries this way, implicitly. We could even wind up attacked because someone committed code to gcc or python or gnu make that corrupted firefox when it was compiling it. All equally and entirely plausible.

The tightening that I wanted to do was simply that of setting up a "nanojit upstream community": a group of people we're going to say we trust as the source of nanojit code, the same way we trust a group of people to be sqlite authors or gnu make maintainers. I'm happy to compose such a list, but if you think this is a "new policy" issue we should take it outside :)

Comment 12

[Brendan Eich [:brendan]]

Shaver mentioned NSS, and my knee jerked into the libpkix table (ow!).

Gerv, there's no code review cure for what you are worried about in comment 9.

Graydon's mentioning gcc or python or gnu make reminded me of Ken Thompson's Turing Award lecture:

http://cm.bell-labs.com/who/ken/trust.html

/be

Comment 13

[Gervase Markham [:gerv]]

OK.

Graydon: I don't think there's any significant policy implications about creating an "upstream nanojit community" with their own hg_nanojit permissions bit. (Are we happy that this community is nanojit-only, rather than a wider one?) The only thing we need is a list of people who are in it, plus the name of the person or persons who can authorize additions.

Gerv

Comment 14

[Nicholas Nethercote [inactive]]

(In reply to comment #4)
> graydon: the knob we currently have is "mozilla-central committers". If
> nanojit-central code is going to be merged into mozilla-central without close
> review at merge time, then it seems to make sense to restrict it to this set of
> people also. Would that exclude some of our external collaborators at Adobe?

I think it would exclude me.  I have tracemonkey commit access but not mozilla-central commit access (AFAIK).

Comment 15

[Graydon Hoare :graydon]

(In reply to comment #12)

> http://cm.bell-labs.com/who/ken/trust.html

Heh. Don't forget Samuel King's proof of concept from last year:

http://www.usenix.org/event/leet08/tech/full_papers/king/king_html/

(In reply to comment #13)

> creating an "upstream nanojit community" with their own hg_nanojit permissions
> bit. (Are we happy that this community is nanojit-only, rather than a wider
> one?) The only thing we need is a list of people who are in it, plus the name
> of the person or persons who can authorize additions.

I think it's small enough that managing it independently won't be hard.

Initial list is, erm, just from glancing over the commit logs:

[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]

I've probably left a few people out, but if someone finds themselves omitted they can apply to have the bit extended to them, yes?

Authorize-new-people person? Probably Edwin, it's originally and mostly his code. Or myself since I seem to have drifted into a nanojit-coordinating role inside Mozilla. Or anyone else who feels somewhat more managerial (?)

Comment 16

[Graydon Hoare :graydon]

Er, actually, it just occurred to me that I can't really tell from looking at the hg logs here whether those commits were pushed by the author, in each case. I suppose you can omit anyone on that list who doesn't have push access to our 'wide open' hg repositories in the first place!

Comment 17

[Reed Loden [:reed]]

(In reply to comment #15)
Added everybody who had an Hg account from your list. The following people didn't have an Hg account, so I couldn't add them:
* [e-mail]
* [e-mail]
* [e-mail]
* [e-mail]

> I've probably left a few people out, but if someone finds themselves omitted
> they can apply to have the bit extended to them, yes?

Yep.

> Authorize-new-people person? Probably Edwin, it's originally and mostly his
> code. Or myself since I seem to have drifted into a nanojit-coordinating role
> inside Mozilla. Or anyone else who feels somewhat more managerial (?)

Ok, sounds good.

nanojit-central modified to require hg_nanogit.

Comment 18

[Gervase Markham [:gerv]]

> nanojit-central modified to require hg_nanogit.

Are we using hg or git? Make up your mind! ;-)

Gerv