Add a QA product to Bugzilla Asa, per our conversation, here is the bug to add a QA product to Bugzilla for QA activities. Please don't implement yet until I've posted a list of components and descriptions for your review. I'm logging this bug to track this creation.
Here is what I'm thinking: ---- Product: QA Description: Tasks relating to the testing of any projects contributed to mozilla.org. ---- Component: Test Plan - New Description: A test plan describes the approach/methodology to test a particular new major feature or group of features. File a bug against this component when there is a new feature being implemented which need a test plan. The bug report is closed when the test plan has been posted to mozilla.org's Quality section. ---- Component: Test Cases - New Description: A test case contains the steps for verifying an outcome of a feature. Normally, there is one test case per outcome. File a bug against this component where there is a new feature being implemented which need test cases (or there are existing features which don't currently have test cases available). All new features should have corresponding test cases. The bug report is closed when all test cases for a feature have been posted to mozilla.org's Quality section. ---- Component: Test Cases - Modify Description: File a bug against this component when any existing test cases need to be modified or to be built upon. The bug report is closed when all test cases needing modification under the bug report have been completed. ---- Component: Investigation Description: There are times when QA gets asked to investigate and recommend default values or other items for various features. File a bug against this component to track and record the results of the investigations. For example, investigate the recommended default cache size. ---- I was debating whether to break this down by functional area or by QA tasks. I decided to do the latter because areas are always changing. I think that the traffic here will be low enough that I can be the default QA contact / assignee. I assume that whomever files the bug will be able to reassign (usually to him/herself) right after filing the bug. Any other suggestions?
Severity: normal → enhancement
Some more thoughts: Component: Test Execution (Help Needed) Description: The URL listed in this bug report needs an owner (outside of the default QA contact) to run it. The bug report will list the frequency and duration. If an owner is found, then this bug report can be closed.
additional data point. bug http://bugzilla.mozilla.org/show_bug.cgi?id=9412 is getting close to fixed and the demo installation at http://bugzilla.mathweb.org:8000/show_bug.cgi?id=1 shows the "Type:" field. Bug types might cover some of the same ground. I still think a Product is a good idea. Will do more thinking.
A "Type" field is useful, but may be hard to query on and still too broad. For example, if a "type" field is used, I assume most of the items I list above will become "tasks". It may be hard then to distinguish between development tasks and QA tasks for a particular feature unless a sort is also done by owner. The types field (defects, feature, inquiry, task, patch) can all apply to tests as well. Will continue thinking....
looking at comment 2, shouldn't there also be a "Test Plan - Modify" component? there are times where an existing QA plan itself needs changes. for example: * when an existing feature/functionality has changed, and/or when the info has become obsolete (eg, old UI specs) * when the documentation itself is unclear, needs rephrasing/rewriting/reorganization (i've had to do this recently ;), etc * when there are mistakes: typos, incorrect links, etc.
We could simply have "Test Case" and "Test Plan" and use the Severity field to determine whether it is a new test case/plan (Severity: Enhancement) or a modification (Severity: how serious the change is). This would be in line with how the rest of bugzilla products are done (you don't have "Style System - New Properties" and "Style System - Bugs").
That sounds reasonable, Ian. Critical could mean that the test plan/ test case is really needed or is very out-of-date.
ian: agreed. simpler, too. :)
By Hick's Law, the time taken to choose from a list of products or components in a list is proportional to the log of the number of items in the list. For that reason, I think bugzilla.mozilla.org should have slightly more products, and fewer components in each product. However, a QA product would not reduce the number of components in any other product, so it would make things worse in that respect. So, what are other good reasons for creating a new product or component? 1. The bugs are currently in a product/component which is very large, so the new product/component would make bugs easier to find. True in this case? No. The bugs proposed for this product are currently in the `Documentation' > `Mozilla Developer' component, which has only 82 open bugs in it already. 2. The creation of the new product/component schema would make people less likely to file bugs in the wrong component. True in this case? No. Of the 22 open bugs I can find about new or existing test plans or test cases, only 2 are filed outside of the `Documentation' > `Mozilla Developer' component, and my sampling of the other 20 doesn't show that any were originally filed in a different component. This strongly suggests that people are comfortable filing such bugs in the component where they currently belong. 3. The very existence of the product or component will inspire people to file many more bugs about it than they would have otherwise. True in this case? Arguable. The only precedent I know of is the `mozilla.org' > `Bugzilla: Keyword & Component Changes' component, where Blake and I filed lots of bugs for getting rid of useless keywords after the component was created. However, I think that was more to do with knowing that the default assignee would directly benefit from fixing the bugs (asa), which wasn't the case for the default assignee where such bugs previously belonged (endico). As for the `Investigation' and `Test Execution' components, these would basically be redundant with the `qawanted' keyword. In the only investigation bug I know of which has actually been acted on (small sample, I know), the investigation resulted in a fix which was checked in *in the same bug*. If that bug had started in a `QA' > `Investigation' component, it would have been moved to the `Mail/News' > `Composition' component halfway through its life anyway, causing nothing more than extra bugspam. So, are there any other reasons to create this product?
It has been explained to me that this product will be mostly about test cases and plans which are thorough for particular areas, rather than the usual case in Bugzilla where test cases and plans just cover individual bugs. That explains the `Test Cases' and `Test Plans' components, though differentiating between test cases/plans and other forms of `Documentation' seems quibbling. But it is still hard to imagine `Investigation' bugs being filed in a dedicated component rather than the module-specific component, when (1) other bugs dependent on the investigation will be in the module-specific component, (2) the investigation will probably be carried out by the default QA contact for that component, and (3) the bug will often be transferred to the specific component anyway if the investigation shows the need for code changes. I'm not violently opposed to a `QA' product (and nor should it make much difference if I was). I just would feel a lot better if I knew that the new product was going to have many more than 22 open bugs in it in the near future, and that most of those bugs wouldn't have been filed under that product in error. (`Oh, I'm doing some QA, and I have a testcase for a bug, so it belongs in "QA" > "Test Cases" ...')
"Investigation" was just a brainstorm idea. What prompted me for that component is bug http://bugzilla.mozilla.org/show_bug.cgi?id=109415 which turned out to something for QA to investigate and report the results back to development. The idea is to track that task so it does not get lost. If we can have some good components and a good process on how they are to be used (and communicated as such), I think this will be the next step to be more structured in our QA approach in working with Mozilla and with improving the quality to ensure that areas do not fall through the cracks.
This will happen soon. I want to think about it a little bit more and I don't have the time today but it's one of my P1 bugs.
Status: NEW → ASSIGNED
Priority: -- → P1
Would this section also include defect reports on the "QA" and "Debug" commands available from within Mozilla itself? (for example, I want to report a problem with Debug-->Form Manager Samples) Basically, the QA tools themselves need mecanisms for feedback. I'm not clear whether the "test cases" below are referring to the QA cases built into the browser or the set of test cases maintained elsewhere.
Assignee: asa → ian
Status: ASSIGNED → NEW
lchiang: Are you still interested in this? It sounds like a good idea, if QA are going to use it. If you are still interested, provide a list of components, descriptions, owners and QA contacts (probably firstname.lastname@example.org), as well as a product description, milestones, and versions, and I'll set it up.
Yes, I would still like to explore this. Other priorities prevented me from thinking on this further. However, I was waiting for Asa's input to this. You can lower the priority of this if you want and keep this enhancement open. Thanks
Righto. Whenever you have a first draft of the components/descriptions/owners/qa, let me know, and we can get feedback from the rest of the community.
Priority: P1 → --
I've thought about this further. I don't think that this will get wide usage unless someone wants to promote its use. Unfortunately, at this time, I don't have the bandwidth to promote its usage in the Mozilla community if the "QA" product were to be created. Pls feel free to close this if you want. Thanks.
righto. Should you change your mind, please feel free to reopen it.
Status: NEW → RESOLVED
Last Resolved: 16 years ago
Resolution: --- → WONTFIX
Component: Bugzilla: Keywords & Components → Administration
Product: mozilla.org → bugzilla.mozilla.org
You need to log in before you can comment on or make changes to this bug.