1330320 - Migrate jenkins-dxr.mozilla.org behind auth0

Status: RESOLVED FIXED

(Webtools Graveyard :: DXR, defect)

Description

[Julien Vehent]

The jenkins instance at jenkins-dxr.mozilla.org is currently public. Given Jenkins' poor security track record and repeated vulnerability that lead to remote code execution, we ask that all jenkins instances are hosted privately, either behind the corp VPN or some SSH bastion.

Users who require access to this jenkins instance should be identified and added to the appropriate LDAP ACLs.

Comment 1

[Erik Rose [:erik][:erikrose]]

We currently rely on Jenkins as the public interface to the status of DXR indexing jobs: when they'll be done or what build error is foiling them. I often direct Firefox developers there. Can we show from the logs that nobody actually checks it?

Comment 2

[Julien Vehent]

I would suggest getting the statuses needed out of the jenkins UI into a separate dashboard [1, 2] so public users never hit the actual jenkins server. There really isn't any safe way to host public jenkins instances without exposing ourselves to RCEs.

[1] https://jenkins.io/blog/2016/01/10/beautiful-jenkins-dashboard/
[2] https://github.com/arcturial/jenkins-dash

Comment 3

[Erik Rose [:erik][:erikrose]]

A nice middle-of-the-road solution that wouldn't require a bunch of programming or setting up additional servers might be to put Jenkins behind an LDAP authz prompt. What do you think about that?

Comment 4

[Julien Vehent]

Not really. Jenkins' access controls, pre-authentication, have been bypassed before so we learned the hard way not to trust them. The whole jenkins server needs to be hosted privately and data that needs to be displayed publicly should be served by a separate application. I know it's a drag, but there really isn't a safe way to host a public jenkins instance (we tried).

Comment 5

[Erik Rose [:erik][:erikrose]]

I wasn't talking about using Jenkins' ACLs but rather having the web server itself throw an auth in front of it. I wouldn't trust Jenkins' own ACLs either.

If that can't be done, then we'll have to do without any sort of build transparency. DXR has only a handful of hours a week from me at the moment, and the off-the-shelf dashboards don't look to give access to console output, which is a major use case for us.

Comment 6

[Julien Vehent]

Ha, sure, that would work. Maybe use auth0 (replacement of Okta) instead of LDAP to get SSO.

I was under the impression you wanted to keep a public page for contributors, but if the target population can be limited to LDAP users, putting the entire service behind auth0 is a fine solution.

Comment 7

[Erik Rose [:erik][:erikrose]]

Sure, auth0 would be fine.

A public page is the ideal, of course, but let's see if anybody screams before spending the hours building one. Cheers!

Comment 8

[Julien Vehent]

:kang - could you share technical details on adding auth0 auth to Apache?

Comment 9

[Guillaume Destuynder [:kang] [this account is no longer active]]

Here's a live demo of Apache + Auth0: https://apache.testrp.security.allizom.org/
Here's the configuration + instructions: https://github.com/mozilla-iam/testrp.security.allizom.org/tree/master/webserver_configurations/OpenID_Connect/Apache
Here's the higher-level "how this work": https://wiki.mozilla.org/Security/Guidelines/OpenID_Connect
Here's how to request Mozilla's auth0 tokens for your app/relying party: https://mana.mozilla.org/wiki/display/SECURITY/SSO+Request+Form

In particular the configuration ensures that the app/relying party (you) will use Mozilla's hosted lock (the login page) and properly check the user session.

Hope this helps - but feel free to ping, mail, or need-info if you need help or have questions

Comment 10

[Kendall Libby [:fubar] (he/him)]

Starting to poke at this again as there's another round of jenkins updates. Filed RITM005249 (https://mozilla.service-now.com/sp?id=ticket&table=sc_req_item&sys_id=eff23635db8632006c3fb1c0ef961930) to get it moving.

Comment 11

[Kendall Libby [:fubar] (he/him)]

Got back to this and after much wailing and gnashing of teeth it's working. https://github.com/mozilla-platform-ops/dxr-infra/pull/39