558856 - Sixgill web server setup

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Brian Hackett [Laid off!]]

To continue deploying Sixgill, we need web hosting to store tool reports, ideally at sixgill.mozilla.org.  This requires:

1. Storage for 1GB+ of HTML pages, which are updated daily.  These do not need to be backed up.
2. The build machine dm-sixgill01 (bug 548427) needs to be able to write HTML pages to the web site.
3. The server needs to be able to run CGI scripts, and communicate information from these scripts to the build machine.  This could be done by having the scripts write to a local directory which the build machine can read from.

Comment 1

[Brian Hackett [Laid off!]]

Hi, has there been any progress on this?  Is there other information/authorization needed?

Comment 2

[matthew zeier [:mrz]]

Is there any reason why the current dm-sixgill01 can't be this server?

Comment 3

[Brian Hackett [Laid off!]]

It would be fine (and very convenient!) if dm-sixgill01 was this server.

Comment 4

[Shyam Mani [:fox2mike]]

(In reply to comment #0)
> To continue deploying Sixgill, we need web hosting to store tool reports,
> ideally at sixgill.mozilla.org.  This requires:
> 
> 1. Storage for 1GB+ of HTML pages, which are updated daily.  These do not need
> to be backed up.
> 2. The build machine dm-sixgill01 (bug 548427) needs to be able to write HTML
> pages to the web site.

[root@dm-sixgill01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      246G  9.2G  224G   4% /

Sounds like a GB isn't an issue at all :)

> 3. The server needs to be able to run CGI scripts, and communicate information
> from these scripts to the build machine.  This could be done by having the
> scripts write to a local directory which the build machine can read from.

Allocate a directory or tell us which one you're using for us to change the apache configs accordingly?

Comment 5

[Brian Hackett [Laid off!]]

Hi, /build/www (does not currently exist) would be good to use as the server's root directory.  One subdirectory, /build/www/scripts would need to run CGI scripts.

Comment 6

[Shyam Mani [:fox2mike]]

Brian,

Cool. So can I set this up when you have content for it? How do you plan to push stuff into /build?

Comment 7

[Brian Hackett [Laid off!]]

I put up an initial site at /build/www, this should be enough for now (more subdirectories will be added to /build/www in the future, don't think that will impact anything).

I'll be setting up a cron job to do a daily run that gets the latest source from mozilla-central, runs the sixgill tool to generate HTML reports from that source, and pushes the reports into /build/www/*.  The scripts are needed so that people using the website can have the tool reanalyze particular parts of the source and generate new reports.

Comment 8

[Shyam Mani [:fox2mike]]

Brian,

Do you have access to the VPN? I see you're able to login to the box, so I guess you do. Please hit http://dm-sixgill01.mozilla.org/ and let me know if everything works as expected.

Once this looks okay, I might have to pass over the app to our Security team for a review before I can make this public aka set this up as sixgill.mozilla.org, let me know if that's okay with you.

Comment 9

[Brian Hackett [Laid off!]]

Hi, it doesn't look like the CGI scripts are working, e.g. if I visit http://dm-sixgill01.mozilla.org/scripts/login.cgi I just get the script's text.  Otherwise looks good.

Having this reviewed by the security team is a very good idea.

Comment 10

[Shyam Mani [:fox2mike]]

(In reply to comment #9)
> Hi, it doesn't look like the CGI scripts are working, e.g. if I visit
> http://dm-sixgill01.mozilla.org/scripts/login.cgi I just get the script's text.
>  Otherwise looks good.

Guess I left out a step in the config, now when I hit that I get :

document.getElementById('login_server').innerHTML = 'Need CGI parameter: mail';

> Having this reviewed by the security team is a very good idea.

Cool.

CC'ing clyon.

Comment 11

[Brian Hackett [Laid off!]]

Cool, looks like things are working now.  How do I disallow access to certain files/directories? (.htaccess files don't seem to be enabled).  The following shouldn't be accessible:

/build/www/scripts/users
/build/www/scripts/jobs/*
/build/www/scripts/jobs_todo/*

Comment 12

[Shyam Mani [:fox2mike]]

I can enable htaccess for /scripts and you drop in a file?

Comment 13

[Shyam Mani [:fox2mike]]

I've added AllowOverride Limit to the scripts section, let me know if you need anything else.

Comment 14

[Shyam Mani [:fox2mike]]

Chris,

Can infrasec poke the app and approve it so I can make it public? Code is in /build (on dm-sixgill01) and dm-sixgill01.m.o is accessible via the VPN.

Thanks!

Comment 15

[Shyam Mani [:fox2mike]]

Over to infrasec, re-assign to me when they're done, so I'll do the rest of the setup.

Comment 16

[Chris Lyon [:clyon]]

(In reply to comment #15)
> Over to infrasec, re-assign to me when they're done, so I'll do the rest of the
> setup.

Adding in mcoates to take a look at this over the next few days.

Comment 17

[Michael Coates [:mcoates] (acct no longer active)]

I'm starting to take a look at the web app.  A couple of quick questions:

1. Is everything located at: http://dm-sixgill01.mozilla.org in scope for testing?  (e.g. are any other unrelated web apps also available via this starting URL)

2. What does this app do?  This will help me better understand the potential attack vectors.

Comment 18

[Brian Hackett [Laid off!]]

#1: Yes, everything at this site is in scope.  The only scripts are the scripts folder (/build/www/scripts on the machine).

#2: The app is a web interface for adding annotations for use by the sixgill static analysis tool (http://www.sixgill.org). (adding annotations this way reduces the lag in seeing the effect of an annotation from several hours to a few seconds).  The website user writes an annotation (function precondition/postcondition, assertion, etc.), the addjob cgi puts a description of this job is put in the /build/www/scripts/jobs_todo folder, another process (/build/run-new/run_fetchjobs) picks up this job and does work based on it:

- Runs gcc with a plugin on some file in the local mozilla-central tree, picking up the effects of the annotation.
- Runs the sixgill binaries to regenerate any new HTML report, and writes that to /build/www/...

Comment 19

[Michael Coates [:mcoates] (acct no longer active)]

Brian,

I've taken a look at the app and will file some security bugs shortly. I'll cc you on each of those.

Another question regarding the app. I noticed users can create a new account on the webpage and then perform all actions within the site.  How are you planning to prevent a malicious user from creating an account and then making unwanted updates?  Should users be vetted by someone before access is granted?

Second question, the data seems to be sensitive (related to buffer overflows). Is this information already public? What are the impacts of making this information publicly available?

Comment 20

[Michael Coates [:mcoates] (acct no longer active)]

Brian,

One more question. Where do you want the security bugs to be filed in bugzilla?  Which product/component etc?

Comment 21

[Brian Hackett [Laid off!]]

Hi, there isn't any prevention for actions from a malicious user --- rubbish
annotations are easy to roll back (by hand), and otherwise the only thing
someone can do is bog down the dm-sixgill01 machine and keep other people from
using the site.  My plan was to wait and see if malicious or pain-in-the-neck
users show up, and if so to use an email-based authentication mechanism.  If
you think malicious users are inevitable, it might make sense to put the
authentication in place now.

All this data has been publicly available at sixgill.org for several months,
plus the tool itself is open source (low publicity, though) so anyone can
download it and run it across any codebase.

Can you file the bugs under Core / Rewriting and Analysis ?

Comment 22

[Brendan Eich [:brendan]]

Should we start out with requiring LDAP entries for all users (hg push access covers this)? Sorry if that's already the plan.

/be

Comment 23

[Reed Loden [:reed]]

(In reply to comment #21)
> Can you file the bugs under Core / Rewriting and Analysis ?

Core is for Gecko code-only issues. I recommend something like Websites :: Other instead.

Comment 24

[Brian Hackett [Laid off!]]

(In reply to comment #22)
> Should we start out with requiring LDAP entries for all users (hg push access
> covers this)? Sorry if that's already the plan.
> 
> /be

Right now there isn't really a plan, anyone can make an 'account' and submit annotations (at least once the website is no longer behind the VPN).  Is there a good way for the server to check LDAP authentication, instead of having a separate login/password?  (no idea how this stuff works).  If so that does sound like the best solution.

Brian

Comment 25

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I think LDAP would be a great approach here.

Comment 26

[Michael Coates [:mcoates] (acct no longer active)]

The unrestricted account creation issue has been captured in bug 567756.

Comment 27

[Michael Coates [:mcoates] (acct no longer active)]

We are finished with our security review of sixgill. Once bug 567465 is resolved (tracker bug which is blocked by all individual issues) then please feel free to move forward.

Comment 28

[Shyam Mani [:fox2mike]]

Punt back when it's good to go :)

Comment 29

[Michael Coates [:mcoates] (acct no longer active)]

Nothing left for infrasec to do here. We're waiting for the q2-secreview-sixgill to be resolved (bug 567465) which has several dependent security bug fixes.

Comment 30

[Chris Lyon [:clyon]]

doesn't belong in our queue, we already have a tracker.

Comment 31

[Shyam Mani [:fox2mike]]

Moving to projects, pending secreview bugs being fixed.

Comment 32

[Shyam Mani [:fox2mike]]

Reopen this when you have patched the security issues.

Comment 33

[Brian Hackett [Laid off!]]

The security issues were fixed over half a year ago, and are still waiting on review to confirm they are fixed.

Comment 34

[Shyam Mani [:fox2mike]]

Reopen when the review has confirmed they've been fixed and there is action for IT to take :)

Comment 35

[Brian Hackett [Laid off!]]

(In reply to comment #34)
> Reopen when the review has confirmed they've been fixed and there is action
> for IT to take :)

How does that happen?  Back in October I commented in all the bugs hanging off bug 567465 that they were fixed, since then they have sat untouched.

Comment 36

[Shyam Mani [:fox2mike]]

(In reply to comment #35)

> How does that happen?  Back in October I commented in all the bugs hanging
> off bug 567465 that they were fixed, since then they have sat untouched.

Poke them again? Email people involved asking them for help? There is a lot of stuff that goes on and people tend to forget :) There is no harm in asking multiple times or just dropping an email asking them if they have some spare time to look at things.

I see you won't let me close this bug as INCOMPLETE, which is what it is at this point and has been for months, so I'm just going to leave it open and not care (because if it was close INCOMPLETE and REOPENED when there was actual action to be taken, it would have made more sense).

Comment 37

[Brian Hackett [Laid off!]]

Oops, meant to leave it INCOMPLETE (bugzilla....)

Comment 38

[Michael Coates [:mcoates] (acct no longer active)]

Sec review items reviewed and closed.

Comment 39

[Damon Sicore (:damons)]

OK.  So, what's next here?  Seriously.  Can we get this going?

Comment 40

[Shyam Mani [:fox2mike]]

(In reply to comment #38)
> Sec review items reviewed and closed.

Coates, good to take this live as is then? 

Brian, it's been so long - What do you need me to do for you to get this going? It should be accessible at sixgill.mozilla.org ?

Comment 41

[Brian Hackett [Laid off!]]

Yeah, the web server on the dm-sixgill01 machine (which seems to be down atm) should be available at sixgill.mozilla.org, with everything behind LDAP.

Comment 42

[Michael Coates [:mcoates] (acct no longer active)]

(In reply to comment #40)
> (In reply to comment #38)
> > Sec review items reviewed and closed.
> 
> Coates, good to take this live as is then? 

Yes.

Comment 43

[Shyam Mani [:fox2mike]]

(In reply to comment #41)
> Yeah, the web server on the dm-sixgill01 machine (which seems to be down
> atm) should be available at sixgill.mozilla.org, with everything behind LDAP.

And https I suppose? 

The box was up, I'm doing a full system upgrade. Let me know if you need SSL, I'll have to grab a cert.

Comment 44

[Brian Hackett [Laid off!]]

(In reply to comment #43)
> (In reply to comment #41)
> > Yeah, the web server on the dm-sixgill01 machine (which seems to be down
> > atm) should be available at sixgill.mozilla.org, with everything behind LDAP.
> 
> And https I suppose? 
> 
> The box was up, I'm doing a full system upgrade. Let me know if you need
> SSL, I'll have to grab a cert.

Yeah, this should be https.  What would SSL be needed for, if all authentication is going through LDAP?

Comment 45

[Michael Coates [:mcoates] (acct no longer active)]

Should be SSL only. Need to protect ldap credentials that are being passed

Comment 46

[Shyam Mani [:fox2mike]]

Party on lads, this is now live and has it's own SSL cert and is fronted by Zeus.

SSH will only work on the internal interface as before. 

I took the opportunity to upgrade the OS and kernel on the machine and reboot it.

If you have any issues with how it works now, please re-open this bug. File new bugs for new requests/features/config changes.

Apologies on behalf of IT/Infrasec that this took so long. The dependent bug in case you're curious is for Nagios monitoring and will be fixed in a few hours.

Comment 47

[Shyam Mani [:fox2mike]]

PS : One hell of a beefy server that. Hope it's put to good use :)