554655 - Daily reporting of security bugs accessed

Status: NEW

(bugzilla.mozilla.org :: General, enhancement)

Description

[Johnathan Nightingale [:johnath]]

We should do what we can to avoid compromise of credentials for any security group member, but we should also have mechanisms in place for discovering and containing compromises after the fact.

Seems like a simple (to describe, dunno what the implementation hit would be) mechanism to help with that would be a once-daily email sent to any individual who accesses security bugs, listing the ones accessed. E.g.

===
Hello [e-mail],

Today your account was used to access these 3 security bugs:

123456 - Exploitable crash at ...
123457 - A malicious page can cause us to...
123211 - Security is neat, lets have more of it

If you did not access these bugs and suspect your account may have been compromised, please forward this email to [e-mail] with details.
===

This does mean that a credential which is compromised might take a day to detect (or even longer if the email isn't responded to promptly), but overall I think that would still represent a net improvement in our ability to identify and contain breaches. In the worst case, a dedicated attacker pulls down the full DB before we detect things, and we are not a lot better off (though at least we can kill follow up access). But in many cases, particularly breaches where the attacker doesn't realize what they have immediately, or where they lack bugzilla-fu, our situation is improved for the cost of a once-daily spam to frequent sec-bug workers.

How do we make it go?

Comment 1

[Michael Coates [:mcoates] (acct no longer active)]

I think this is a great idea. I'll add one more item to it. A few security people should also receive the master list. That way if attacker Joe creates a new account somehow and accesses all the security bugs, we will know too.

Comment 2

[timeless]

Note that some accounts shouldn't work this way. Please respect accounts which are set not to deliver *any* email.

fwiw, I agree w/ comment 1.

i've filed bug 554870 about a related item.

Comment 3

[Gervase Markham [:gerv]]

Bugzilla doesn't keep track of who _accesses_ each bug, only who changes it. So a tool which did this would need to be built on top of web server log parsing.

We would need to change BMO Apache logging to log user ID of user in access.log (I told clyon how to do this a year ago, but I don't know if he's done it.)

Then, each day:

- Use API to get list of all open secure bugs, with titles
- Search for all bug accesses in access.log
- Filter out the ones which are to secure bugs
- Aggregate the list by user ID
- Create and send the relevant emails

Gerv

Comment 4

[Chris Lyon [:clyon]]

(In reply to comment #3)
> Bugzilla doesn't keep track of who _accesses_ each bug, only who changes it. So
> a tool which did this would need to be built on top of web server log parsing.
> 
> We would need to change BMO Apache logging to log user ID of user in access.log
> (I told clyon how to do this a year ago, but I don't know if he's done it.)

Unless there is something I am missing here, user ID in the access.log can only be done via http basic auth. If there is a way to get Bugzilla to add this to the logs, this will solve most of the tracking issues. 


> Then, each day:
> 
> - Use API to get list of all open secure bugs, with titles
> - Search for all bug accesses in access.log
> - Filter out the ones which are to secure bugs
> - Aggregate the list by user ID
> - Create and send the relevant emails

I can't see doing this every day since most of the time we need real time information. We do gather secure bugs and feed them into our monitoring system for tracking but there are elements such as user id we would love to have.

Comment 5

[Gervase Markham [:gerv]]

(In reply to comment #4)
> Unless there is something I am missing here, user ID in the access.log can only
> be done via http basic auth. If there is a way to get Bugzilla to add this to
> the logs, this will solve most of the tracking issues. 

You are missing something :-) The value of the Bugzilla_login cookie is the userid of the user - 9911 in my case. And you can get Apache to log the value of particular cookies to access.log.
http://httpd.apache.org/docs/current/mod/mod_log_config.html
You want %{Bugzilla_login}C

Now I seem to remember there was a weird truncation bug with cookie logging a while ago, but it might have been fixed now. Try it and see :-)

Gerv

Comment 6

[Chris Lyon [:clyon]]

(In reply to comment #5)
> (In reply to comment #4)
> > Unless there is something I am missing here, user ID in the access.log can only
> > be done via http basic auth. If there is a way to get Bugzilla to add this to
> > the logs, this will solve most of the tracking issues. 
> 
> You are missing something :-) The value of the Bugzilla_login cookie is the
> userid of the user - 9911 in my case. And you can get Apache to log the value
> of particular cookies to access.log.
> http://httpd.apache.org/docs/current/mod/mod_log_config.html
> You want %{Bugzilla_login}C

That is good news actually because we can do many things with info

@justdave, can we get this working on bmo?

Comment 7

[Nathan Miller [:nmiller]]

@justdave would this bug cover what we were emailing about the other day or should I still open another bug for adding Bugzilla_login to the standard apache access logs?

I can create another bug to remove the forensic logging and to scrub those other files with the cookies.

Comment 8

[Dave Miller [:justdave]]

The userid is already included in the logs provided by Zeus, and has been since we set it up when we switched over to it from the Netscaler.  It was also included in the logs from the Netscaler before that.  I'm guessing you want this in the actual apache logs (since that's what you're forwarding to arcsight, I imagine), and that's probably the source of my prior confusion if so.  I previously kept replying with something along the lines of "that's already in there, why do you keep asking?".

Comment 9

[Dave Miller [:justdave]]

If I specify %{Bugzilla_login}C in the log format, and the user is not currently authenticated, so there is no Bugzilla_login cookie, does it print a -?

Comment 10

[Dave Miller [:justdave]]

(In reply to comment #9)
> If I specify %{Bugzilla_login}C in the log format, and the user is not
> currently authenticated, so there is no Bugzilla_login cookie, does it print a
> -?

It does, I just tested it on one of the staging servers.

Works nicely if I replace the %u in the usual log format, actually, and that's the obvious place to put it. :)

Comment 11

[Dave Miller [:justdave]]

If the bug summaries are included in the above emails, the mail should probably be encrypted when it's mailed out (once we get that stuff set up - it's coming).  Right now it wouldn't be any different than sending normal bugmail about the bug.

Comment 12

[Dylan Hardison [:dylan] (he/him)]

We have sufficient data in the database for this feature.

Comment 13

[Dylan Hardison [:dylan] (he/him)]

This is really trivial, I'm just going to do it.

Comment 14

[Daniel Veditz [:dveditz]]

This proposed feature would have detected the 2015 bugzilla compromise at least a year before we actually found out about it because of a 0-day