180813 - create "clean-report" keyword for QA classification of bugs

Status: VERIFIED FIXED

(bugzilla.mozilla.org :: Administration, task)

Description

[benc]

The main networking compontent had been using "helpwanted" to refer to bugs that
were ready for engineering to assign, prioritize and target milestone.

Bugs that were submitted in the NEW or UNCONFIRMED state, which were owned by
the default ("new-network-bugs" were to be reviewed for accuracy and validity
before being sent to engineering.

That was sort of a stretch of the current definition, which reads:

"Bugs or features which require coding assistance to fix or implement."

This was originally gagan's suggestion. I think this process works well, but
dougt has asked for a return to the stricter definition of "helpwanted".

Since using helpwanted was never meant to be a permanent solution, I am
requesting that we create a "assignme" keyword.

In short, putting this keyword on a bug means:

1- The summary is clear, searchable, and consisten with the problem.
2- Clear steps for reproduction are provided.
3- Proper keywords and platform values have been set.

The person that puts the "assignme" keyword on a bug is essentially vouching for
the quality of the bug report. The goal is to allow engineers to query quickly
for a list of bugs that are "high quality".

If approved, I will take care of writing the bug description. If a lot of
process questions arise, I will write a longer document for mozilla.org, and put
a link into the description as well.

Comment 1

[Doug Turner (:dougt)]

there is no need for the keyword.

Comment 2

[Bradley Baetz (:bbaetz)]

We have the 'confirmed' state for this..

Comment 3

[Hixie (not reading bugmail)]

yeah... how is this different from "NEW" vs "CONFIRMED"?

If there are bugs that are being put in the NEW state that should not be, let me
know, and I will remove the relevant privileges from the offending users.

Comment 4

[Asa Dotzler [:asa]]

Hixie, it's not "bugs that are being put into the New state taht should not be"
it's bugs that start out New that shouldn't. Over half of the bugs in the system
start out as New because they're filed by people who either earned the privs or
were grandfathered in when Unconf was created. 
New doesn't mean confirmed, and Unconfirmed doesn't even mean unconfirmed.
Unless we have the ability to move bugs from New to Unconfirmed or all bugs
start out as Unconfirmed then we have a situation where New bugs are a mess of
"really confirmed and ready to for fixing" and "reported as New but who knows".
It would be nice to be able to note that a bug, even though it started out as
New, that isn't really ready for developer action, or that it is.
Developers can make this distinction by moving a bug from New to Assigned. That
to me has always suggested something like "this is a bug and it's probably one
I'm capable of fixing". QA triage doesn't have any decent mechanism for saying
"this is really a bug and should be looked at by a developer".  
Reopening for further consideration. 

Comment 5

[benc]

Doug: 

I am assuming you are speaking soley from an engineering perspective. From a QA
and bug management perspective, there are sound reasons for us to use this.

Using this keyword would decrease the workload of your team and increase he
quality of the bug reports you look at. However, even if engineering does not
want to use the results of a keyword query, QA still has uses for such a keyword.

You asked us to stop using the "helpwanted" keyword with a secondary meaning, so
this bug is an attempt to accomodate your request.

Bradley:

There is no real "confirmed" state because New is messed up... I would like
bugzilla to be changed, but since we don't have any to implement that, this is
the interim solution.

Comment 6

[timeless]

i'd prefer to fix unco/new so that they're more useful.

Comment 7

[Doug Turner (:dougt)]

>Using this keyword would decrease the workload of your team and increase he
quality of the bug reports you look at.

You should have been in sales!  I find this statement untrue at best.  My team
is still going to have to review all of the incoming bugs with or without this
keyword.  Exactly how will this keyword decrease my workload?

I ask that the QA team (you and tever) confirms bugs, try to reproduce problems,
and verify bugs.  I will take care of the triage.  

Comment 8

[benc]

If your or your team wants to read every single bug that appears in Networking,
I am certainly not trying to stop you. This keyword would not preclude that.

However, this is not scalable in several dimensions:

1- Number of bugs - we can expect the number of bugs to increase, probably on a
linear basis as the number of downloads (hence contributors) increases.

2- Other components - not all engineering teams want to read all incoming bugs,
and if this is a system that is generally a best practice, then the usage is
stil a good idea.

3- Many bugs in Networking bugs are not Networking. Why should you have to read
those?

4- Many bugs in Networking are dulicates. Why should you spend time marking
duplicates when other people can do that? Looking at the compreg.dat duplicates
(200+), I don't think more thant 10% were marked by a netscape account, much
less by the people in XPCOM (Bug 171441).

The underlying idea here is:

as the community grows in size, and contributes more bugs, it should also grow
in (for lack of a better phrase) "wisdom and stature", and learn to take care of
 its own. I think there are enough people who can and will help with problem
descriptions of submitted bugs to have an open system.

When you are talking about "triage", I am assuming that you are talking about
the more specific meaning, which is taking a well-described bug, and deciding if
and when it needs to be fixed. Having a clear problem description is the
pre-processing of what engineering has to do.

Comment 9

[Bradley Baetz (:bbaetz)]

If you find people routinly filing useless bugs as NEW, then I'm sure you can
get canconfirm removed from them.

Rather than adding an 'assignme' keyword, couldn't you just assign it to the
relevent person? I don't see what this will help - someone stil has to go
through and look at every incoming bug.

ASSIGNED is meant to mean 'I have a plan to work on this', although its not used
that way be all people, I know.

Yes, not all imcoming bugs in networking are really netowkring. But _someone_
has to read them, and move them to the appropriate component. If someone files a
mailnews bug in one of the networking components, what benefit does a pre-triage
stage have? Move it to the right product, and be done with.

For anyone to use this keyword on a bug, they must have determined if the bug is
valid. If it is, confirm it/reassign it to someone who can do something about
it. If not, then dupe it, mark is as INVALID, move it, or whatever. Where is the
benefit in someone having to look at a bug twice?

If you really want to, you can use the 'assigned to new-network-bugs' as the
"hasn't been triaged" state. That was, IIRC, the originally purpose of the dummy
assignee.

Comment 10

[Hixie (not reading bugmail)]

This sounds to me like a Bugzilla bug. I am wary of adding keywords to work
around limitations in the bug system that really should be fixed at the lowest
level.

In the meantime I propose that the peaple who want this keyword use a status
whiteboard marker and see if it really does help them. If it does indeed help
increase the quality of bug reports, then we can consider this further.

Comment 11

[Doug Turner (:dougt)]

benc - I personally do not care what you do.  if you think that this keyword
will be useful, knock yourself out.

Comment 12

[Hixie (not reading bugmail)]

Having re-read all the comments here with an open mind, and had discussions on
IRC with some of the people commenting on this bug, I believe the real need, and
the solution that would most benefit the community at large on the short to
medium term, is not a keyword asking for a bug to be assigned, but a keyword
indicating that a bug has:
   1. a clear summary and problem description
   2. steps to reproduce
   3. been reproduced by someone other than the reporter
   4. a valid, neat, and useful testcase, if appropriate
...and is generally in an ideal state as far as the QA Contact is concerned.
This is effectively the opposite of the "qawanted" keyword.

I have decided to call this keyword "clean-report". I do not expect it to be
used by most Bugzilla users, indeed only those who most frequently perform
preliminary triage and bug confirmation should use it.

On the long term, I think the UNCONFIRMED vs NEW state would be the correct way
to solve this. Part of this would require that Bugzilla have the ability to send
a bug back to UNCONFIRMED. Another part of this is the fact that we need to
reduce the number of poorly written bugs being filed in the NEW state. To help
counteract this, I ask that if you see anyone who frequently reports poorly
written bug reports you inform me of them so that I may speak with them about
the problem and potentially reduce their Bugzilla privileges such that their
subsequent bugs start out in the UNCONFIRMED state.

FIXED. Please re-open if you do not consider this an adequate solution.

Comment 13

[benc]

V