Add a QA product to Bugzilla



17 years ago
7 years ago


(Reporter: lchiang, Assigned: Hixie (not reading bugmail))





17 years ago
Add a QA product to Bugzilla

Asa, per our conversation, here is the bug to add a QA product to Bugzilla for
QA activities.  Please don't implement yet until I've posted a list of
components and descriptions for your review.  

I'm logging this bug to track this creation.

Comment 1

16 years ago
Here is what I'm thinking:

Product:  QA
Description:  Tasks relating to the testing of any projects contributed to

Component:  Test Plan - New
Description:  A test plan describes the approach/methodology to test a
particular new major feature or group of features.  File a bug against this
component when there is a new feature being implemented which need a test plan.
 The bug report is closed when the test plan has been posted to's
Quality section.


Component:  Test Cases - New
Description:  A test case contains the steps for verifying an outcome of a
feature.  Normally, there is one test case per outcome.  File a bug against this
component where there is a new feature being implemented which need test cases
(or there are existing features which don't currently have test cases
available). All new features should have corresponding test cases.  The bug
report is closed when all test cases for a feature have been posted to's Quality section.


Component:  Test Cases - Modify
Description:  File a bug against this component when any existing test cases
need to be modified or to be built upon.  The bug report is closed when all test
cases needing modification under the bug report have been completed.


Component:  Investigation
Description:  There are times when QA gets asked to investigate and recommend
default values or other items for various features.  File a bug against this
component to track and record the results of the investigations.  For example,
investigate the recommended default cache size.

I was debating whether to break this down by functional area or by QA tasks.  I
decided to do the latter because areas are always changing.  I think that the
traffic here will be low enough that I can be the default QA contact / assignee.
 I assume that whomever files the bug will be able to reassign (usually to
him/herself) right after filing the bug.

Any other suggestions?

Severity: normal → enhancement

Comment 2

16 years ago
Some more thoughts:

Component:  Test Execution (Help Needed)
Description:  The URL listed in this bug report needs an owner (outside of the
default QA contact) to run it.  The bug report will list the frequency and
duration.  If an owner is found, then this bug report can be closed.

Comment 3

16 years ago
additional data point. bug is
getting close to fixed and the demo installation at shows the "Type:" field. Bug
types might cover some of the same ground. I still think a Product is a good
idea. Will do more thinking.

Comment 4

16 years ago
A "Type" field is useful, but may be hard to query on and still too broad.  For
example, if a "type" field is used, I assume most of the items I list above will
become "tasks".  It may be hard then to distinguish between development tasks
and QA tasks for a particular feature unless a sort is also done by owner.  

The types field (defects, feature, inquiry, task, patch) can all apply to tests
as well.

Will continue thinking....
looking at comment 2, shouldn't there also be a "Test Plan - Modify" component?
there are times where an existing QA plan itself needs changes. for example:

* when an existing feature/functionality has changed, and/or when the info has
become obsolete (eg, old UI specs)
* when the documentation itself is unclear, needs
rephrasing/rewriting/reorganization (i've had to do this recently ;), etc
* when there are mistakes: typos, incorrect links, etc.

Comment 6

16 years ago
We could simply have "Test Case" and "Test Plan" and use the Severity field to
determine whether it is a new test case/plan (Severity: Enhancement) or a
modification (Severity: how serious the change is).

This would be in line with how the rest of bugzilla products are done (you don't
have "Style System - New Properties" and "Style System - Bugs").

Comment 7

16 years ago
That sounds reasonable, Ian.  Critical could mean that the test plan/ test case
is really needed or is very out-of-date.
ian: agreed. simpler, too. :)

Comment 9

16 years ago
By Hick's Law, the time taken to choose from a list of products or components 
in a list is proportional to the log of the number of items in the list. For
that reason, I think should have slightly more products,
and fewer components in each product. However, a QA product would not reduce the
number of components in any other product, so it would make things worse in that

So, what are other good reasons for creating a new product or component?

1.  The bugs are currently in a product/component which is very large, so the
    new product/component would make bugs easier to find. True in this case?
    No. The bugs proposed for this product are currently in the `Documentation' 
    > `Mozilla Developer' component, which has only 82 open bugs in it already.

2.  The creation of the new product/component schema would make people less
    likely to file bugs in the wrong component. True in this case? No. Of the
    22 open bugs I can find about new or existing test plans or test cases,
    only 2 are filed outside of the `Documentation' > `Mozilla Developer'
    component, and my sampling of the other 20 doesn't show that any were
    originally filed in a different component. This strongly suggests that 
    people are comfortable filing such bugs in the component where they
    currently belong.

3.  The very existence of the product or component will inspire people to file
    many more bugs about it than they would have otherwise. True in this case?
    Arguable. The only precedent I know of is the `' > `Bugzilla:
    Keyword & Component Changes' component, where Blake and I filed lots of
    bugs for getting rid of useless keywords after the component was created.
    However, I think that was more to do with knowing that the default assignee
    would directly benefit from fixing the bugs (asa), which wasn't the case
    for the default assignee where such bugs previously belonged (endico).

As for the `Investigation' and `Test Execution' components, these would
basically be redundant with the `qawanted' keyword. In the only investigation
bug I know of which has actually been acted on (small sample, I know), the
investigation resulted in a fix which was checked in *in the same bug*. If that
bug had started in a `QA' > `Investigation' component, it would have been moved
to the `Mail/News' > `Composition' component halfway through its life anyway,
causing nothing more than extra bugspam.

So, are there any other reasons to create this product?

Comment 10

16 years ago
It has been explained to me that this product will be mostly about test cases
and plans which are thorough for particular areas, rather than the usual case in
Bugzilla where test cases and plans just cover individual bugs. That explains
the `Test Cases' and `Test Plans' components, though differentiating between
test cases/plans and other forms of `Documentation' seems quibbling. But it is
still hard to imagine `Investigation' bugs being filed in a dedicated component
rather than the module-specific component, when (1) other bugs dependent on the
investigation will be in the module-specific component, (2) the investigation
will probably be carried out by the default QA contact for that component, and
(3) the bug will often be transferred to the specific component anyway if the
investigation shows the need for code changes.

I'm not violently opposed to a `QA' product (and nor should it make much
difference if I was). I just would feel a lot better if I knew that the new
product was going to have many more than 22 open bugs in it in the near future,
and that most of those bugs wouldn't have been filed under that product in
error. (`Oh, I'm doing some QA, and I have a testcase for a bug, so it belongs
in "QA" > "Test Cases" ...')

Comment 11

16 years ago
"Investigation" was just a brainstorm idea.  What prompted me for that component
is bug which turned out to
something for QA to investigate and report the results back to development.  The
idea is to track that task so it does not get lost.

If we can have some good components and a good process on how they are to be
used (and communicated as such), I think this will be the next step to be more
structured in our QA approach in working with Mozilla and with improving the
quality to ensure that areas do not fall through the cracks.

Comment 12

16 years ago
This will happen soon. I want to think about it a little bit more and I don't
have the time today but it's one of my P1 bugs. 
Priority: -- → P1

Comment 13

16 years ago
Would this section also include defect reports on the "QA" and "Debug" commands
available from within Mozilla itself? (for example, I want to report a problem
with Debug-->Form Manager Samples)

Basically, the QA tools themselves need mecanisms for feedback.

I'm not clear whether the "test cases" below are referring to the QA cases built
into the browser or the set of test cases maintained elsewhere.

Comment 14

16 years ago
Assignee: asa → ian

Comment 15

16 years ago
lchiang: Are you still interested in this? It sounds like a good idea, if QA are
going to use it. If you are still interested, provide a list of components,
descriptions, owners and QA contacts (probably, as well as a
product description, milestones, and versions, and I'll set it up.

Comment 16

16 years ago
Yes, I would still like to explore this.  Other priorities prevented me from
thinking on this further.  However, I was waiting for Asa's input to this.   You
can lower the priority of this if you want and keep this enhancement open.


Comment 17

16 years ago
Righto. Whenever you have a first draft of the
components/descriptions/owners/qa, let me know, and we can get feedback from the
rest of the community.
Priority: P1 → --

Comment 18

16 years ago
I've thought about this further.  I don't think that this will get wide usage
unless someone wants to promote its use.   Unfortunately, at this time, I don't
have the bandwidth to promote its usage in the Mozilla community if the "QA"
product were to be created.

Pls feel free to close this if you want.

Comment 19

16 years ago
righto. Should you change your mind, please feel free to reopen it.
Last Resolved: 16 years ago
Resolution: --- → WONTFIX
Component: Bugzilla: Keywords & Components → Administration
Product: →
You need to log in before you can comment on or make changes to this bug.