897236 - Request for Web bounty bugzilla form

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Extensions, defect)

Description

[David Chan [:dchan]]

The security assurance team has a quarterly goal to improve the reporting / recording process for web bounty bugs. The current process is that a researcher either sends the bug report to [e-mail] or they file a bug and notify [e-mail] that they have filed such a bug.

This process worked relatively well until we started tracking more metrics for reports. Current bounty bugs require the following metadata
whiteboard tags
[reporter-external]
[site:VULNERABLE SITE] - (there may be more than one)
[verif?] - for triage purposes

keywords
wsec-* [1]

flags
sec-bounty

bug dependencies
One from the list in bug 835434


The desire for a form is to pre-populate / aid in completing some of these fields. Ideally this would reduce the amount of extra checking required to make sure a bounty bug
has the required information. We have had bugs fall through the cracks due to not being nominated for bounty.

A form would also allow us to extend / modify the bounty process without requiring researchers to constantly check the bounty page. There is an internal and external desire to create a "Hall of Fame" page, but the big issue is attribution / releases. I could see the form having an option to include / exclude the bug for "Hall of Fame" purposes.


Could the Bugzilla team comment on the feasibility of creating such a form? Alternatively, do you believe there is a better way to approach this problem, a webapp? I can gather a list of all the information we wish to capture. 


[1] - https://wiki.mozilla.org/Security_Severity_Ratings#Group_Keywords

Comment 1

[David Chan [:dchan]]

:glob

Who should I talk to to get this started?

Comment 2

[David Lawrence [:dkl]]

Sorry for the delay in getting to this. We can definitely help you with the creation of a custom bug entry form to suit your needs.

We can create field such as drop downs that have some predetermined values like the whiteboard tags, etc. Also we can include a few free form text fields such as for summary and description. 

Just list out the fields you want to display, how you want them ordered, and the wording you need around each field. We can then map them to normal Bugzilla fields in the background and well as choose proper defaults for the fields not displayed.

Then we put up a working draft on our devel instance at bugzilla-dev.allizom.org for you to try out and give feedback.

dkl

Comment 3

[David Chan [:dchan]]

I got some feedback from Curtis regarding the form. We want to start with simplified version of the normal Bugzilla submission form.

When a user views the page, they will only be able to see and manipulate the following information
Summary / Title
Comment
Attachments
URL

The following fields / options will be set automatically and not surfaced to the user
Whiteboard tags
[reporter-external]
[verif?]

flags
sec-bounty?

Component
Security Assurance

Groups
Security-Sensitive Websites bug


Our team will handle the triaging of the bugs to the correct team / product once we receive the report. We will also set the correct blockers / dependencies. If possible we would also like to send an autoreply / formletter response to the user saying that we have received their bug. The Bugzilla e-mail may be sufficient.

Is there an easy way to see whether a bug was created through a Bugzilla form? We would like to track that for metrics purposes.

Comment 4

[David Lawrence [:dkl]]

Sorry for the delay. Will work on this tomorrow and should be able to get a form up for your feedback on our test instance.

dkl

Comment 5

[David Lawrence [:dkl]]

(In reply to David Chan [:dchan] from comment #3)
> I got some feedback from Curtis regarding the form. We want to start with
> simplified version of the normal Bugzilla submission form.
> 
> When a user views the page, they will only be able to see and manipulate the
> following information
> Summary / Title
> Comment
> Attachments
> URL

Which ones should be required? I assume summary and comment but are the others required fields as well?
 
> The following fields / options will be set automatically and not surfaced to
> the user
> Whiteboard tags
> [reporter-external]
> [verif?]

So no need for any keywords to be set in this version?
 
> flags
> sec-bounty?
> 
> Component
> Security Assurance

If we file the bug into mozilla.org :: Security Assurance by default, we currently do not have the sec-bounty flag enabled for that combination. Thoughts? Also the sec-bounty group currently requires a user to be in the core-security permission group to set sec-bounty? so we would need to relax that if you want anyone to be able to use this form and have the flag created automatically.

> If possible we would also like to send an autoreply /
> formletter response to the user saying that we have received their bug. The
> Bugzilla e-mail may be sufficient.

Normal email preferences, users do not receive the email notification for the bug *they* just created unless they have changed their preferences to receive all email. So we would need to add some code to add them back
as a recipient of the web-bounty submission form was used.
 
> Is there an easy way to see whether a bug was created through a Bugzilla
> form? We would like to track that for metrics purposes.

Aside from setting some whiteboard entry, etc. with [web-bounty-form] or similar, the only other way is to grep the logs from time to time.

dkl

Comment 6

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

(In reply to David Lawrence [:dkl] from comment #5)
> (In reply to David Chan [:dchan] from comment #3)
> > I got some feedback from Curtis regarding the form. We want to start with
> > simplified version of the normal Bugzilla submission form.
> > 
> > When a user views the page, they will only be able to see and manipulate the
> > following information
> > Summary / Title
> > Comment
> > Attachments
> > URL
> 
> Which ones should be required? I assume summary and comment but are the
> others required fields as well?

I think attachments is the only one that should not be required, we may not get a PoC or screen shot but the others we definitely need.

>  
> > The following fields / options will be set automatically and not surfaced to
> > the user
> > Whiteboard tags
> > [reporter-external]
> > [verif?]
> 
> So no need for any keywords to be set in this version?
>  

No, no keywords on this version. Just those 2 tags (for now)

> > flags
> > sec-bounty?
> > 
> > Component
> > Security Assurance
> 
> If we file the bug into mozilla.org :: Security Assurance by default, we
> currently do not have the sec-bounty flag enabled for that combination.
> Thoughts? Also the sec-bounty group currently requires a user to be in the
> core-security permission group to set sec-bounty? so we would need to relax
> that if you want anyone to be able to use this form and have the flag
> created automatically.

I don't think we are comfortable relaxing that as if they can set ? they could also incorrectly set + which has larger implications. Since I am a "watcher" of the component I will just have to set the process up to set the ? flag by hand when reviewing incoming items.

> 
> > If possible we would also like to send an autoreply /
> > formletter response to the user saying that we have received their bug. The
> > Bugzilla e-mail may be sufficient.
> 
> Normal email preferences, users do not receive the email notification for
> the bug *they* just created unless they have changed their preferences to
> receive all email. So we would need to add some code to add them back
> as a recipient of the web-bounty submission form was used.
>  
> > Is there an easy way to see whether a bug was created through a Bugzilla
> > form? We would like to track that for metrics purposes.
> 
> Aside from setting some whiteboard entry, etc. with [web-bounty-form] or
> similar, the only other way is to grep the logs from time to time.

Sounds like we might want to set this whiteboard tag as well for now then as I think we need some tracking on this.

> 
> dkl

Before the summary section is it possible to have this link as the help link or text to help get better bugs? 
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines?redirectlocale=en-US&redirectslug=Bug_writing_guidelines

Or can we have some preloaded form style text in the box to help guide good bug filing?

Comment 7

[David Chan [:dchan]]

+1 to what curtis said.

We don't want people to inadvertently set sec-bounty+, so I guess curtis or another PM will have to handle setting the ? flag.

Perhaps we can move the email reply to something that happens in triage. The gap we want to address is making sure that bug reporters gets acknowledgement from the security team that we have seen their bug. Replies to security@ mail sometimes fall through the cracks.


dkl:
I'm wary of using a whiteboard tag or anything else that can be changed by normal users for metrics purposes. Are there bugzilla components in which new bugs can be created, but only moved out of by privileged accounts? If so, we could create a new component in Security Assurance just for bounty requests.

I don't think our team needs continuous tracking of number of bounty submissions, perhaps once / twice a quarter for metrics. grepping logs is adequate in my opinion.

Comment 8

[Reed Loden [:reed]]

(In reply to Curtis Koenig [:curtisk] from comment #6)
> > Also the sec-bounty group currently requires a user to be in the
> > core-security permission group to set sec-bounty? so we would need to relax
> > that if you want anyone to be able to use this form and have the flag
> > created automatically.
> 
> I don't think we are comfortable relaxing that as if they can set ? they
> could also incorrectly set + which has larger implications. Since I am a
> "watcher" of the component I will just have to set the process up to set the
> ? flag by hand when reviewing incoming items.

No, we can make it so anybody can request sec-bounty, but only members of security-group can grant it. Does that work?

Comment 9

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

That would be fine, our concern is people changing ? to -/+ if all they can do is ? while that could get annoying it would be appropriate.

Comment 10

[David Lawrence [:dkl]]

(In reply to Curtis Koenig [:curtisk] from comment #6)
> Before the summary section is it possible to have this link as the help link
> or text to help get better bugs? 
> https://developer.mozilla.org/en-US/docs/Mozilla/QA/
> Bug_writing_guidelines?redirectlocale=en-
> US&redirectslug=Bug_writing_guidelines
> 
> Or can we have some preloaded form style text in the box to help guide good
> bug filing?

I have added the guidelines link. We may want to add some paragraph of text at the beginning explaining what the form is for and some other paragraph at the end above Submit explaining what will happen after the bug is submitted. Also we can add some brief explanation of each field above the form fields. You can use as an example of what I mean: https://bugzilla.mozilla.org/form.creative If you give me the text I can add it in.

dkl

Comment 11

[David Lawrence [:dkl]]

(In reply to David Chan [:dchan] from comment #7)
> dkl:
> I'm wary of using a whiteboard tag or anything else that can be changed by
> normal users for metrics purposes. Are there bugzilla components in which
> new bugs can be created, but only moved out of by privileged accounts? If
> so, we could create a new component in Security Assurance just for bounty
> requests.

We can create a new specific component for new bugs and then enforce the component changing
in the code itself. Any tweaks to the logic would require future code pushes as well.
 
> I don't think our team needs continuous tracking of number of bounty
> submissions, perhaps once / twice a quarter for metrics. grepping logs is
> adequate in my opinion.

grepping logs is always an option.

dkl

Comment 12

[David Lawrence [:dkl]]

(In reply to Curtis Koenig [:curtisk] from comment #9)
> That would be fine, our concern is people changing ? to -/+ if all they can
> do is ? while that could get annoying it would be appropriate.

There is also the issue with sec-bounty not currently being available to the mozilla.org :: Security Assurance component. Should I do the above as well as add the flag to the component?

dkl

Comment 13

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

(In reply to David Lawrence [:dkl] from comment #10)
> (In reply to Curtis Koenig [:curtisk] from comment #6)
> > Before the summary section is it possible to have this link as the help link
> > or text to help get better bugs? 
> > https://developer.mozilla.org/en-US/docs/Mozilla/QA/
> > Bug_writing_guidelines?redirectlocale=en-
> > US&redirectslug=Bug_writing_guidelines
> > 
> > Or can we have some preloaded form style text in the box to help guide good
> > bug filing?
> 
> I have added the guidelines link. We may want to add some paragraph of text
> at the beginning explaining what the form is for and some other paragraph at
> the end above Submit explaining what will happen after the bug is submitted.
> Also we can add some brief explanation of each field above the form fields.
> You can use as an example of what I mean:
> https://bugzilla.mozilla.org/form.creative If you give me the text I can add
> it in.
> 
> dkl

That is a great idea, I'll put that in a separate comment for cleanliness in a bit (need to think it out).

(In reply to David Lawrence [:dkl] from comment #12)
> (In reply to Curtis Koenig [:curtisk] from comment #9)
> > That would be fine, our concern is people changing ? to -/+ if all they can
> > do is ? while that could get annoying it would be appropriate.
> 
> There is also the issue with sec-bounty not currently being available to the
> mozilla.org :: Security Assurance component. Should I do the above as well
> as add the flag to the component?
> 
> dkl

That is an interesting question I don't think we've fully thought through. If we file these in mozilla.org :: Security Assurance component then we are going to have to change the component/product every time a bug is filled, because these bugs are really against websites. We may be better off making a Websites::Security component for these to minimize churn and to make the bug filling more like what happens for Core/Firefox (people file bugs in Core::Security and we move it where it needs to go). I also believe the flag is already there for this component.
Thoughts?

Comment 14

[David Lawrence [:dkl]]

Here is what I have so far (minus the sec-bounty? flag which I am waiting on response).

https://bugzilla-dev.allizom.org/form.web.bounty

dkl

Comment 15

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

dchan, how does this look to you?

Summary/Title
A short description of the issue being reported including the host name for the website on which it exists (example xss in blarg.foo.mozilla.org)

Comment
How was this issue discovered, include the steps, tools or other information that will help reproduce and diagnose the issue. A good primer on what to include can be found at: https://developer.mozilla.org/en-US/docs/Mozilla/QA/

Attachments
Any files that add context to the report, such as screen shots or code blocks for reproduction purposes.

URL
The full URL (hostname/subpage) where the issue exists (if the URL is especially long please just include it in the comments)

Comment 16

[David Chan [:dchan]]

(In reply to David Lawrence [:dkl] from comment #14)
> Here is what I have so far (minus the sec-bounty? flag which I am waiting on
> response).
> 
> https://bugzilla-dev.allizom.org/form.web.bounty
> 
> dkl

The form looks good to me. It does deviate from the Bug Writing guidelines slightly since the user isn't selecting the Product / Component. I think that is an improvement since some sites have Components, some sites are under other and other sites are in a completely different product like the Sync website under Services


(In reply to Curtis Koenig [:curtisk] from comment #15)
> dchan, how does this look to you?
> 
> Summary/Title
> A short description of the issue being reported including the host name for
> the website on which it exists (example xss in blarg.foo.mozilla.org)
> 
> Comment
> How was this issue discovered, include the steps, tools or other information
> that will help reproduce and diagnose the issue. A good primer on what to
> include can be found at: https://developer.mozilla.org/en-US/docs/Mozilla/QA/
> 
> Attachments
> Any files that add context to the report, such as screen shots or code
> blocks for reproduction purposes.
> 
> URL
> The full URL (hostname/subpage) where the issue exists (if the URL is
> especially long please just include it in the comments)

You bring up a good point about really long POC. We should make URL optional


A Websites::Security component makes sense to me, with our team moving the bug to proper Product / Component afterwards.

Comment 17

[David Lawrence [:dkl]]

New version pushed to dev

https://bugzilla-dev.allizom.org/form.web.bounty

dkl

Comment 18

[David Lawrence [:dkl]]

(In reply to David Lawrence [:dkl] from comment #17)
> New version pushed to dev
> 
> https://bugzilla-dev.allizom.org/form.web.bounty
> 
> dkl

The DB on bugzilla-dev.allizom.org was recently refreshed. Please ping me or glob on IRC to reset your password so you can see the test form for feedback.

dkl

Comment 19

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

I don't think I have an account at all, do I just create an account like I did for bugzilla or do you all do that?

Comment 20

[David Lawrence [:dkl]]

(In reply to Curtis Koenig [:curtisk] from comment #19)
> I don't think I have an account at all, do I just create an account like I
> did for bugzilla or do you all do that?

I have emailed you your reset password. Feel free to login and change it.

dkl

Comment 21

[David Lawrence [:dkl]]

dchan, I also reset your password and emailed it to you since I need your feedback on the form as well.

dkl

Comment 22

[David Chan [:dchan]]

(In reply to David Lawrence [:dkl] from comment #21)
> dchan, I also reset your password and emailed it to you since I need your
> feedback on the form as well.
> 
> dkl

Thanks for resetting my password. I've looked at the form and it seems good to me. Summary of remaining work as I understand it

- change sec-bounty flag so that it can be set by anyone, but only +/- by core-security
- decide where to place the bug initially. Given the current restrictions in the Security Assurance component, I propose the bug be filed under "Websites :: Other". The bug can then be moved to the appropriate product / component during triage or by the reporter.
- add a hall of fame checkbox. rforbes can describe more in detail what he needed.

The first two would be required before launching the form. Hall of fame has some other dependencies before it is needed.

Comment 23

[David Lawrence [:dkl]]

(In reply to David Chan [:dchan] from comment #22)
> (In reply to David Lawrence [:dkl] from comment #21)
> > dchan, I also reset your password and emailed it to you since I need your
> > feedback on the form as well.
> > 
> > dkl
> 
> Thanks for resetting my password. I've looked at the form and it seems good
> to me. Summary of remaining work as I understand it
> 
> - change sec-bounty flag so that it can be set by anyone, but only +/- by
> core-security
> - decide where to place the bug initially. Given the current restrictions in
> the Security Assurance component, I propose the bug be filed under "Websites
> :: Other". The bug can then be moved to the appropriate product / component
> during triage or by the reporter.
> - add a hall of fame checkbox. rforbes can describe more in detail what he
> needed.
> 
> The first two would be required before launching the form. Hall of fame has
> some other dependencies before it is needed.

Ok. I have pushed the final version for testing.

https://bugzilla-dev.allizom.org/form.web.bounty

Now files into Websites :: Other and sets the sec-bounty? flag automatically. Once you verify we can
push it to production. 

Please file a separate bug about the hall of fame checkbox and we can address it then.

dkl

Comment 24

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

:dkl - just fyi and not a huge deal but I am not getting reset mail and it's not stuck in my postini or personal junk folders

Comment 25

[David Chan [:dchan]]

Curtis and I looked at the form on my laptop and most of it looks good.

- Can we move the [verif?] whiteboard tag to be the rightmost / last tag. This tag will be removed once our team has looked at the bug.
- It appears that a non core-security member can change the sec-bounty flag from ? to + . Is this a side effect of a reporter being able to change flags they set? I filed a bug with a new account and was able to change the flag.

I'll talk with Raymond and file a followup for the hall of fame stuff. Thanks for all your help.

Comment 26

[:glob ✱]

(In reply to David Chan [:dchan] from comment #25)
> - It appears that a non core-security member can change the sec-bounty flag
> from ? to + . Is this a side effect of a reporter being able to change flags
> they set? I filed a bug with a new account and was able to change the flag.

this is a side-effect of the database sanitisation routine for bugzilla-dev - the groups are removed and had to be re-added.  things will behave as expected on production.

Comment 27

[David Lawrence [:dkl]]

(In reply to Curtis Koenig [:curtisk] from comment #24)
> :dkl - just fyi and not a huge deal but I am not getting reset mail and it's
> not stuck in my postini or personal junk folders

My bad. Email is disabled from bugzilla-dev.allizom.org which is why you are not receiving the notifications.
If you PM me, I can manually reset your password to your choosing if you need to change it.

dkl

Comment 28

[David Lawrence [:dkl]]

(In reply to David Chan [:dchan] from comment #25)
> Curtis and I looked at the form on my laptop and most of it looks good.
> 
> - Can we move the [verif?] whiteboard tag to be the rightmost / last tag.
> This tag will be removed once our team has looked at the bug.

Once this is done, I will commit then and the new form will be in next weeks push.

> - It appears that a non core-security member can change the sec-bounty flag
> from ? to + . Is this a side effect of a reporter being able to change flags
> they set? I filed a bug with a new account and was able to change the flag.

As glob mentioned, this will be set correctly on production and the group configurations on 
bugzilla-dev do not always match production.

dkl

Comment 29

[David Lawrence [:dkl]]

Committing to: bzr+ssh://[e-mail]/bmo/4.2
modified .htaccess
added extensions/BMO/template/en/default/bug/create/create-web-bounty.html.tmpl
Committed revision 9024.