Improve the description of the "Security Sensitive" checkbox for new bugs

RESOLVED FIXED

Status

()

RESOLVED FIXED
10 years ago
7 years ago

People

(Reporter: nelson, Assigned: reed)

Tracking

Details

Attachments

(1 attachment)

Far less than half of the new bugs that have the "security sensitive"
(security group) flag set are even close to qualifying for the intended 
meaning of that bug flag.   

The current description seems to be understood as "If you'd like to see
this bug report kept under wraps for any reason, check this".  

This causes every person who watches security@mozilla.org to waste a lot
of time reading bug reports that don't qualify.  

We can do better.  The description can make it clearer that this check box
is for product vulnerabilities, and is NOT for
- enhancement requests of any kind  (including new CA certificates)
- XSS errors and other web site errors
- other common (but wrong) reasons for checking that box
(Reporter)

Updated

10 years ago
Summary: Improve the description of the "Security Sensitive" checkbox for new bogs → Improve the description of the "Security Sensitive" checkbox for new bugs

Comment 1

10 years ago
yeah, I think part of the problem is in the guided bug form... 

https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&format=guided

if we add some clearity that this check box is for undisclosed or publically known security vulnerabilities or exploits then we might reduce the traffic significantly.

We also might thing about removing the security check box from the guided form altogether, although that has a bit of risk.

(Assignee)

Comment 2

10 years ago
(In reply to comment #1)
> if we add some clearity that this check box is for undisclosed or publically
> known security vulnerabilities or exploits then we might reduce the traffic
> significantly.

I think that's fine. Do you have recommendations on the wording?

The current wording is "This is a security problem that should be kept confidential until addressed (see the mozilla.org security policy for more details)."

> We also might thing about removing the security check box from the guided form
> altogether, although that has a bit of risk.

I don't think this is a good idea at all. How will people file security bugs in Bugzilla that aren't members of the security group? Bad idea.

Comment 3

10 years ago
(In reply to comment #2)
> 
> I think that's fine. Do you have recommendations on the wording?
> 
> The current wording is "This is a security problem that should be kept
> confidential until addressed (see the mozilla.org security policy for more
> details)."

Something like "this check box is for security researchers that are reporting serious security exploits and vulnerabilities. (see the mozilla.org security policy for more details)."

Its pretty clear from the reports that no one is taking the time to find or read the more details on the security policy page, so we should say who this field is for.  Its also not clear on the mozilla.org site where to find the "security policy."   I think most of the traffic will go here http://www.mozilla.org/security/ and that page encourages reporting to security@mozilla.org and has nothing on filing bugs.

 
> 
> > We also might thing about removing the security check box from the guided form
> > altogether, although that has a bit of risk.
> 
> I don't think this is a good idea at all. How will people file security bugs in
> Bugzilla that aren't members of the security group? Bad idea.
> 

They would use security@mozilla.org mailing list to report if they don't have a bugzilla account, or they would use the the full bug entry form...

https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox

not the guided form.   Admittedly this has some risk, but I think some research would show that 99.9% of the "not a security bug" reports are coming from the guided form, and that the bulk of valid security issues are coming from the "non-guided" form.  Those numbers are just my impression from looking at the incoming flow of bugs, but some more detailed research might be valuable if anyone wanted to step up...

I'm ok with not removing the secruity check boz from the guided form until someone is able to confirm that this is the primary/sole source of the extra noise...

Comment 4

10 years ago
I don't think a lot of people watching for these kind of bugs mark then consistently but here is a query that turns up 25 bugs (not including this one( that have the comment "not a security bug"

https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=substring&long_desc=not+a+security+bug&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&resolution=---&emailassigned_to1=1&emailtype1=exact&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=exact&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=noop&type0-0-0=noop&value0-0-0=

All but 7 of those bugs have the markings that indicate they were filed using the guided form.  That's not exactly the data that we are after, but gives you an idea that a high pct. (+70%) of incorrectly marked bugs are coming in via the guided form.

(Assignee)

Comment 5

10 years ago
(In reply to comment #3)
> Something like "this check box is for security researchers that are reporting
> serious security exploits and vulnerabilities. (see the mozilla.org security
> policy for more details)."

I think that's probably a good start... do note that the text is used for all products in Bugzilla, including Websites, Webtools, Bugzilla itself, etc., so we don't need to make it too-product specific.

> Its also not clear on the mozilla.org site where to find the
> "security policy."   I think most of the traffic will go here
> http://www.mozilla.org/security/ and that page encourages reporting to
> security@mozilla.org and has nothing on filing bugs.

The text "mozilla.org security policy" links to the policy.

> They would use security@mozilla.org mailing list to report if they don't have a
> bugzilla account, or they would use the the full bug entry form...
> https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox
> not the guided form.

No, users without canconfirm will _always_ get the guided form (by default), so that's not a good idea, and it just makes it harder for people to report security bugs to us.

Comment 6

10 years ago
>  users without canconfirm will _always_ get the guided form (by default), so
that's not a good idea, and it just makes it harder for people to report
security bugs to us.


maybe we should have a special guided form for reporting security bugs that makes it really easy and correct make the right markings on bugs that are security bugs, and includes information on the bounty program and other security program bug policies.

That would allow us to remove the security flag from the general guide form.

At any rate, here is the current text:  "This is a security problem that should be kept confidential until addressed (see the mozilla.org security policy for more details). "

I think a lot of people are getting tripped up on the word "confidential."  In particular I think this happens when bug filer might not be familiar with open source bug filing systems, and that value of having as many eyes as possible on the general bug.

A lot of the bugs I've read say something like "I found something that really looks bad to me, let's keep it confidential until you can get it fixed so you really don't look bad..."   

Many of these turn out to be configuration specific kinds of issues where having a lot of people look at the bug leads to faster resolution.   

Maybe there is also a way to also say what kinds of bugs are *not* security bugs and include that in the suggested text for the checkbox that we have started in comment 3 and 5.
(Assignee)

Comment 7

10 years ago
Just a short note: I'm all for better wording the text concerning the security checkbox, but I'm not for making it harder to report security bugs.

Comment 8

10 years ago
> but I'm not for making it harder to report security bugs.

I don't think anyone is, but at any rate, lets get the better description of what a security bug is inserted on the guide form as soon as possible.

looks like at least 13 or 14 more "not security bugs" found their way into the critical evaluation queue since the last comment on bug , wasting more time and effort.
(Reporter)

Comment 9

10 years ago
How can we get forward motion on this problem?
Whose approval is needed?
Is a draft new description for the checkbox needed?

I think that just a little improvement might eliminate security sensitive 
bug like bug 457365.  Let's try to at least raise the bar *a little*.
(Assignee)

Comment 10

10 years ago
(In reply to comment #9)
> Whose approval is needed?

Nobody's, really...

> Is a draft new description for the checkbox needed?

Yes.
(Reporter)

Comment 11

10 years ago
Suggested new working: 
  This is a serious security vulnerability that could hurt many browser users 
  and needs to be kept secret to keep evildoers from exploiting it until it 
  is fixed.
(Assignee)

Comment 12

10 years ago
(In reply to comment #11)
> Suggested new working: 
>   This is a serious security vulnerability that could hurt many browser users 
>   and needs to be kept secret to keep evildoers from exploiting it until it 
>   is fixed.

See my comment #5 -- Bugzilla isn't just about Firefox, so that text won't work.
(Reporter)

Comment 13

10 years ago
OK

  This is a serious security vulnerability that could hurt many users of
  Mozilla software and needs to be kept secret to keep evildoers from 
  exploiting it until it is fixed.
(Assignee)

Comment 14

10 years ago
Sounds ok to me.

dveditz, what do you think?
(In reply to comment #13)
>   This is a serious security vulnerability that could hurt many users of
>   Mozilla software and needs to be kept secret to keep evildoers from 
>   exploiting it until it is fixed.

Sure. I'm a little dubious of the premise though. I'd rather have too many bugs filed as security sensitive than not enough. Not because someone might see them, but because the real problems are likely to get lost in the noise if the security flag isn't set.

But something like this change seems like it clarifies the purpose rather than being an attempt to scare people away so that's fine. I think it's a little long, and don't like the kept/keep repetition so close together.

  "This is a security problem that evildoers could use to hurt many users of
  Mozilla software. It should be kept confidential until it is fixed."

"evildoers" is a little too Texan for my taste, but does get the point across. The only real substitute would be "hackers" but I don't want to get embroiled in a hackers/crackers debate. And no, we can't use "crackers"--that will mystify 95% of the people using the form.

Arg, and now I've got use/users too close together. Avoid "users"?

  "This is a security problem that evildoers could abuse to hurt people.
  It should be kept confidential until it is fixed."

or spread them out?

  "Evildoers could abuse this security problem to hurt many users of
  Mozilla software. It should be kept confidential until it is fixed."

Go for silly?

  "You could use this to abuse users."
How about something simpler:

 [ ] This is a security problem that should be kept hidden from the public to protect users.

The "protect users" part is really the reason we keep bugs hidden in the first place, and I think that's the real problem with the current description - it doesn't make it clear why you'd want the bug to be kept hidden.
Or perhaps:

[ ] This is a security problem that could be used to harm users, and should be kept hidden from the public until it is resolved.

"could be used to harm users" is more explicit, and "until it is resolved" is more accurate.
We need something like "temporarily" or "until it's fixed" in there for sure. Before that was added we got even more of the internal access passwords and phone numbers than we do now.

I worry that users see all these as
  This is a security problem blah blah blah
   -- yes it is

If that's true it really doesn't matter what the blah blah is, they've stopped reading already. "Yes, it's a security problem" check the box. I try not to let it bother me too much, they mean well.

That said comment 17 might be the best version yet (though I'd ditch the comma).
(In reply to comment #18)
> I worry that users see all these as
>   This is a security problem blah blah blah
>    -- yes it is
> 
> If that's true it really doesn't matter what the blah blah is, they've stopped
> reading already. "Yes, it's a security problem" check the box. I try not to let
> it bother me too much, they mean well.

Yeah, and I think wording is not going to get that number down to zero, but I think this wording will improve the situation by some non-zero amount, so hooray for iterative advances.  :)
 
> That said comment 17 might be the best version yet (though I'd ditch the
> comma).

Yeah, I agree.
(Reporter)

Comment 20

10 years ago
> I worry that users see all these as
>   This is a security problem blah blah blah

OK, then delay the words "security problem" until much later in the sentence.
e.g. 

[ ] Many users could be harmed by this security problem, so it should be
kept hidden from the public until it is resolved.

or 

[ ] If this became known, attackers could harm many users with this security 
problem, so it should be kept hidden from the public until it is resolved.
> [ ] Many users could be harmed by this security problem, so it should be
> kept hidden from the public until it is resolved.
> 
> or 
> 
> [ ] If this became known, attackers could harm many users with this security 
> problem, so it should be kept hidden from the public until it is resolved.

Both of those sounds good, I prefer the second (but delete the comma after "known,").

Both of those need more of a break after "problem":
  "problem, so it"  --> 
      "problem; it" or "problem. It"

I prefer the second because "If this became known" may help signal to people that they don't need to use the flag for a bug they got off a public web site--it's already known. On the other hand the flag does alert us to issues that might otherwise sit unnoticed in bugzilla given the high volume of UNCONFIRMED bugs.
(Reporter)

Comment 22

10 years ago
> they don't need to use the flag for a bug they got off a public web site
> -- it's already known.

That's a good point, so here's yet another suggestion.

[ ] This problem has not previously been publicly reported.  If it became 
known attackers could use it to harm many users; so it should be kept 
hidden from the public until it is resolved.
I actually prefer your original version (minus the commas). It's quite possible the filer is the first person to report the problem _to us_ and I wouldn't want to discourage that.
(Assignee)

Comment 24

10 years ago
(In reply to comment #23)
> I actually prefer your original version (minus the commas).

What's the deal about the commas? All the commas in comment #20's proposed sentences are correct and needed.
The comma after "known" is wrong. The comma in "problem, so" is correct usage, but on purely aesthetic grounds I feel it gives the text more punch to split the sentence with a semi-colon (or period).

For your comma-reading pleasure:
http://www.webupon.com/Web-Talk/Comma-Overuse-in-the-Internet-Generation.34012
(Assignee)

Comment 26

10 years ago
(In reply to comment #25)
> The comma after "known" is wrong.

No, it's not wrong...

See http://owl.english.purdue.edu/handouts/grammar/g_comma.html, specifically rule #2.

> The comma in "problem, so" is correct usage,
> but on purely aesthetic grounds I feel it gives the text more punch to split
> the sentence with a semi-colon (or period).

Using a semicolon would be wrong here, but a full-stop period would work just fine.

See http://owl.english.purdue.edu/handouts/grammar/g_commacomp.html.

> For your comma-reading pleasure:
> http://www.webupon.com/Web-Talk/Comma-Overuse-in-the-Internet-Generation.34012

Commas are fine when used correctly. The "Internet Generation" just doesn't understand basic grammar rules; therefore, misuse of the comma is much more commonplace, which can and does cause correct usage to be mistaken as incorrect usage.
(Reporter)

Comment 28

10 years ago
Can we get some traction on this?
Can we get any of the existing proposals rolled out on b.m.o?

The bugs being submitted with the security box checked are getting 
downright ridiculous.
(Assignee)

Comment 29

8 years ago
Created attachment 455804 [details] [diff] [review]
patch - v1
Assignee: nobody → reed
Status: NEW → ASSIGNED
(Assignee)

Comment 30

8 years ago
Committing to: bzr+ssh://bzr.mozilla.org/bmo/3.4/
modified template/en/default/bug/create/create-guided.html.tmpl
modified template/en/default/bug/create/create.html.tmpl
Committed revision 6799.

This will go live with the next bmo upgrade.
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
sweet!  Thanks, Reed.
Component: Bugzilla: Other b.m.o Issues → General
Product: mozilla.org → bugzilla.mozilla.org
You need to log in before you can comment on or make changes to this bug.