1410453 - Assess use of external addon netlify.com in Mozilla's GitHub organization 'mozilla' and 'mozilla-mobile'

Status: RESOLVED FIXED

(mozilla.org :: Github: Administration, task)

Description

[Blair MacIntyre]

I want to give netlify.com access to mozilla and mozilla-mobile for the following reasons:

Below are my answers to your stock questions:

** Which repositories do you want to have access? (all or list)

mozilla/webxr-api
mozilla/webxr-polyfill
mozilla-mobile/webxr-ios

(may be other repos, probably all starting with "webxr-", in the future)

** Are any of those repositories private?

no.

** Provide link to vendor's description of permissions needed and why

https://www.netlify.com/docs/github-permissions/

netlify watches repos and when there is a commit, pulls the repo and builds the website from it.

Comment 1

[edunham]

Andre, 

https://www.netlify.com/docs/webhooks/ leads me to believe that it's possible to configure Netlify with per-repo access tokens. Would this be sufficient for the webhooks you're trying to set up? If so, it would be a good way to avoid giving Netlify access to all repos within the Mozilla org while still having it work on the webxr-* repos. As documented in https://mana.mozilla.org/wiki/display/POLICIES/Standard%3A+GitHub+repositories+and+organizations, it's preferred to minimize integrations' access to the entire organization.

Comment 2

[edunham]

Note to self: I need to update https://github.com/mozilla/admin_for_mozilla_private/wiki/3rd-Party-App-Integration-Hook-Lore#netlify once we arrive at a solution in this bug.

Comment 3

[Blair MacIntyre]

FWIW, I'm completely agnostic on how we get this to work;  obviously, I'd prefer a solution that (a) works and (b) works for the org.

I admit I'm not sure exactly what is needed with webhooks vs other access.  There are only two real features I needs, one require, one desired:
- required: when I commit to the repo, it triggers a new build on netlify.  I think webhooks and the ability to pull from the repo is all that is needed for this this. 
- desired: PR integration, particularly the example they give on that page of setting a comment on a PR with link to the URL where the PR build is hosted.  This is amazingly useful for testing PRs, but not strictly needed.

The extend of my use of netlify was in orgs/repos I controlled, so I've never had to dig in at this level before.

Comment 4

[edunham]

I've contacted Netlify support to figure out if and how we can set it up per-repo without handing out full organization access.

Comment 5

[edunham]

I got the following back from Netlify support and will try it on the requested repos: 

Hi, 
It is possible to link a repository without granting any permissions other than ones you put in place manually.
To do this, first make sure you've signed up for a Netlify account.  This will be needed for the next steps to work :)
On a machine with a browser (e.g. your development laptop, rather than your CI system), install our CLI:

npm install netlify-cli -g

With that cli, you'll be able to do a lot of things, one of which is prepare a site for deployment and get the necessary authentication information to enter for linking your repository to our system for continuous deployment.
Next, change to the directory that contains a checkout of your source code.
Then run

netlify init -m

This will first use the browser to authenticate with Netlify, and store an access token which will enable other command line operations.  This token is in ~/.netlify/config in case you ever need to remove it.
After authenticating, that command will print a deploy key.  This will need to be installed in the repository you wish to deploy.  The key is unique to this repository and the netlify site that you are about to deploy - you cannot use it for other sites or repositories!
The CLI prompts you to install it before continuing, so once you've done so, you'll continue and we'll provide you a webhook as well, which you'll also need to put in place & configure in your repo settings.  I'd suggest setting the webhook events yourself ("Let me select individual events") and choosing "Push" as well as "Pull Request".   That webhook may not work with a self-hosted repo (apologies - I don't have one handy to test to confirm), in this case, you can use one that you setup yourself on the main settings page for the site after deployment.
Once you have all that setup, you should be able to deploy just by pushing to git.
You've granted us only the permissions you've installed yourself to any of your repositories, and can remove those permissions entirely from your repository at will, so we won't retain any permissions if you decide not to use our service.

Comment 6

[Blair MacIntyre]

Do you just want me to try it, since I want to set up the repos and site?

Comment 7

[edunham]

That's fine too. You're admin on both of the mozilla/webxr-* repos and it seems I do not have perms on mozilla-mobile.

Comment 8

[hwine]

fwiw, this is typical of many "OAuth" (old style) 3rd party integrations. They want more access to do the initial setup, reducing friction for new customers. Doing it the "hard way" (comment 5) also works. As is our usual process with such 3rd party apps, I've denied the permission request -- it will still work for you.

For updating PR status, the authors should switch to the newer "GitHub App" approach, which does give the fine grained access they mention on their page. On the Mozilla org, we restrict GitHub Apps to specific repositories, so the granting of "PR write" permissions is not an org wide concern.

If they do that, please open a new request, listing the repos you want included.

:bmacintyre as :edunham mentioned, we don't have admin rights on the mozilla-mobile organization. You'll need to find out who does.

Comment 9

[Blair MacIntyre]

I set up the basic stuff, and it works.

But, as expected, I can't update PR's because I can't send info back from netlify.  

:hwine by "Github App" approach, do you mean we need to add/create a Github App integration?  How do we do that?   Is that what you mean by opening a new Bug?   What info do you need?  

This is what I'd like to enable https://www.netlify.com/blog/2016/07/20/introducing-deploy-previews-in-netlify/

Comment 10

[hwine]

(In reply to Blair MacIntyre from comment #9)
> I set up the basic stuff, and it works.

good!
 
> :hwine by "Github App" approach, do you mean we need to add/create a Github
> App integration?  How do we do that?  

No, unfortunately, this would take work by netify. There are 2 ways 3rd party apps can integrate with GitHub to write to repositories: OAuth (legacy, what netify is using), and "GitHub Application" (aka "integrations") which have been available for about a year. OAuth has very course grain permissions - it we let an OAuth app write to your repository, it can write to anyone's repository. "GitHub app" lets us grant much finder grained permissions, so it much preferred.

> Is that what you mean by opening a
> new Bug?   What info do you need?  

_If_ Netify created a new GitHub App, you'd file a bug similar to this one, so we can grant access.

> 
> This is what I'd like to enable
> https://www.netlify.com/blog/2016/07/20/introducing-deploy-previews-in-
> netlify/

Yeah -- that would be nice. And it could be implemented via the "GitHub App" authorization approach.

Comment 11

[Blair MacIntyre]

Ok, I'll ask.

Comment 12

[Blair MacIntyre]

Talked to the netlify support folks, they escalated to their CTO: the report back is "that would be a lot of work, it's not on our map for the foreseeable future".

Comment 13

[hwine]

Bummer. 

1) For the mozilla org, does the manual setup work for you? Or do you want to ask for a security review of 'netlify'? (Then we'd engage EIS.)

2) For the mozilla-mobile org, the decision is up to the owners of that org.

Blair - please let us know your decision on (1) and we can close or re-route this ticket.

Comment 14

[Blair MacIntyre]

For (1) the manual setup currently provide the base functionality (when we push to github, we get the site re-depolyed), but I would really like the additional functionality.  Being able to automatically preview PRs (for example) is shockingly useful when we start having folks submit more PRs against the web libraries.

So if it's possible to make this happen, that would be awesome.  Even if only for public repos.

I'm not sure we need it on mozilla-mobile right now, I've moved the Javascript code back into the webxr-polyfill library.

Comment 15

[hwine]

:gene -- first one of these we do want to do the deeper sec review on. Is everything you need in this bug?

Comment 16

[Gene Wood [:gene]]

> So if it's possible to make this happen, that would be awesome.  Even if only for public repos.

Unfortunately there's no way with a GitHub integration like this to constrain which types of organization repos (public vs private), a user can authorize netlify to access.

> first one of these we do want to do the deeper sec review on

Blair, as you've got the functionality you need currently, would you be open to holding on the approval of netlify across the entire github.com/mozilla org for a bit longer as the organization owners of github.com/mozilla are currently in the process of coming up with a new, more well defined, strategy for dealing with the inherent conflict in getting user (like yourself) the functionality you want while at the same time protecting sensitive code bases hosted in github.com/mozilla (like firefox accounts).

The challenge with enabling netlify now is that we've not yet communicated to the repo admins of the 1200+ repos in github.com/mozilla about the security implications of enabling all users in the github.com/mozilla org to authorize netlify to act on those users behalfs and in doing so potentially grant netlify access to their repos without their knowledge.

We worked on this at the Austin work week and I'm hoping we'll have something by the first two weeks of January.

If this is acceptable I'll come back to you in the first couple weeks of January with an update.

Comment 17

[Blair MacIntyre]

That's fine.  I'd be open to anything you all think is reasonable.  Thanks for the update.

Comment 18

[Armen [:armenzg]]

I've wanted to use Netlify, however, I'm now making sure I leave the repos under my user or a different org than 'mozilla'.

I don't know if it applies to this conversation but I've noticed a checkmark to "Limit GitHub access to public repositories."
https://cl.ly/0Q3y333m2M0N
The doc is here:
https://www.netlify.com/docs/github-permissions/

Comment 19

[hwine]

I'll give my view of where we are with this:
 - we're in the process of encouraging all "owners" to have day-to-day accounts. That greatly reduces any risk.
 - we're in the process of informing all "admins" of sensitive repos that it's their responsibility to protect their repos (and provide some tooling to help) Being in the org doesn't improve security in any meaningful way.

Once we have acceptable traction on both of those efforts, we will have addressed the concerns enough to approve apps like netlify.

:gene is ^^ close enough?

Comment 20

[Gene Wood [:gene]]

Yes, that's a good summary. We haven't communicated to the repo admins of the 2000 repos yet nor built something to enable them to audit the users to whom they've granted read/write or admin permissions on their repo, to discover what third parties those users have delegated rights to.

As a result if we enable netlify, the attack surface of all repos increases, repo admins don't know this, and we don't have any way to enable them to discover this.

I met with GitHub 2 days ago and talked about this specific problem (though there's no solution yet).

I would recommend that we continue to hold off on enabling netlify on github.com/mozilla and hopefully in the next month or month and a half be better positioned to enable it.

Comment 21

[Ed Morley [:emorley]]

(In reply to Gene Wood [:gene] from comment #20)
> I would recommend that we continue to hold off on enabling netlify on
> github.com/mozilla and hopefully in the next month or month and a half be
> better positioned to enable it.

I don't suppose there's been any news for this? :-)

I'd like to overhaul the Treeherder docs, but my plan for doing so involves mkdocs+third party theme, which isn't supported on Read the Docs. My hope was to use Netlify to host the docs (since GitHub pages has a whole host of issues, and Netlify comes with nice things like PR deploy previews), however the manual setup approach mentioned above doesn't allow for deploy previews and other features. 

The only workaround would be to move the Treeherder repo outside of the Mozilla org, but that seems a bit excessive for just wanting to overhaul the docs (albeit we'd then be able to use the tool in bug 1440615 too) - and would also annoyingly mean we'd lose the TravisCI paid org plan, so would have to pay for our own to speed up CI again.

Comment 22

[hwine]

With a fresh cup of coffee in hand, I re-read this, and I think there is a workaround.

(In reply to Armen [:armenzg] from comment #18)
> The doc is here:
> https://www.netlify.com/docs/github-permissions/

At the bottom of that page, there is an item 4:
> If you or the organization owner would prefer not to grant organization-level access, our Support team can provide you the necessary information to connect a repository manually. (This includes a deploy key specific to your Netlify site and a webhook to notify us of your new commits).
All of that work can be done by any repository admin -- no organization owner scopes are required.

Will that process work for y'all? It appears to only be this one-time setup that Netlify is trying to make more convenient for less techy folks.

:emorley - could you let us know if this is a viable solution?

Comment 23

[Ed Morley [:emorley]]

That workaround is the what I'm referring to in comment 21 by "manual setup" (see also comment 5, comment 14), which doesn't allow for the PR preview feature (since it needs GitHub API keys to update PR statuses and/or post comments) - so unfortunately wouldn't help. Was worth checking though :-)

Comment 24

[hwine]

Ed - sorry about that -- clearly it's a 2 cups of coffee day.

That does mean Netlify's docs are out of date, though, since they claim to discard the token after installation. Something is not adding up here.

Can you clarify what Netlify support has told you regarding this issue?

From a brief read of their docs at:
  https://www.netlify.com/docs/webhooks/
It appears that individual users would be generating the GitHub tokens to allow the commit status updates. What has your experience with that interface?

Those docs imply that after the manual setup, anyone with write access to repo should be able to generate a token to permit status updates -- no owner scopes needed.

Comment 25

[Ed Morley [:emorley]]

I believe that https://www.netlify.com/docs/github-permissions/ 's reference to discarding tokens might be about the access token from signing into Netlify as an end-user ("log in with" on https://app.netlify.com/), rather than the access given to Netlify from the their oauth app when added to a repo/org. 

However I agree that page could be clearer - would you mind emailing them at the link at the bottom of that page mentioning the confusion?

Comment 26

[:Eli Perelman]

With the Privacy review in CASA complete and now on to Terms of Service, maybe this decision could be re-evaluated?

Comment 27

[hwine]

(In reply to :Eli Perelman from comment #26)
> With the Privacy review in CASA complete and now on to Terms of Service,
> maybe this decision could be re-evaluated?

The concern isn't about Netlify-as-a-vendor, it's how we handle the security for any OAuth app. We never enable an OAuth app that requires an owner to grant access to the OAuth application.

While that is the "default" process for the install, Netlify has provided another method, as documented on their site.

(In reply to Ed Morley [:emorley] from comment #25)
> However I agree that page could be clearer

Please re-read comment 24. After you manually configure the webhooks for a repo (you need to be a repo admin for that step), all the commit/PR/whatever notification is configured via Netlify (not GitHub) web site. An admin can do that for sure, perhaps even someone with write permissions. (The Netlify page is technically correct, if a bit awkwardly written.)

Bottom line: no owner access is required to install or activate any feature of Netlify. The admin for the repository can configure everything they deem appropriate for their repository by following the directions on the Netlify site.

Comment 28

[Ed Morley [:emorley]]

Have you verified all functionality works, or is this inferring?
Testing now locally shows that the commit statuses are not being updated, even when manually set up.

The set up process is also pretty tedious compared to the GitHub app method (there's more than just listed above), so this really needs to be a stop-gap rather than a long term solution. Reopening the bug for the long term solution (eg asking people with sensitive repositories to move them to another org, per comment 16 and comment 20).

Comment 29

[Ed Morley [:emorley]]

Netlify now have a new-style GitHub app (which is surprising given comment 12 gave the impression it wasn't happening any time soon), so it's now possible to configure Netlify on `mozilla` org repositories :-)

See:
https://www.netlify.com/docs/github-permissions/#installing-the-netlify-app-on-github

(If the user creating the Netlify site is an admin for the repo, they can even enable the integration themselves - otherwise it triggers the request process)