Closed Bug 369544 Opened 18 years ago Closed 18 years ago

Add resolution "Incomplete" (or something similar)

Categories

(bugzilla.mozilla.org :: Administration, task)

Product:
bugzilla.mozilla.org
Component:
Administration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: Kensie, Assigned: marcia)

Details

mconnor has noted that bugs without an informative comment #0 are most of the time not very useful, even if eventually the problem is pursued and steps to reproduce are later included in the bug. It's also my understanding that support requests, bugs that don't point explicitly to a specific problem are supposed to be considered invalid, rather than trying to dupe them against the bug we *think* the user is hitting. I think we avoid marking these bugs as invalid for 2 reasons, 1) The users are having genuine problems and we don't want to hit them with an "invalid" without confirming that it is infact an invalid issue. Especially since if someone comes along with a proper description of the problem, it's easier to look back to see other users who had the same problem if their bugs aren't marked invalid (as happened with the firefox 2 installer issue) and 2) general unease with essentially telling people looking for support that their problem doesn't matter. I propose we add the resolution "Incomplete" for the cases of support requests and indescript filings. This is a more accurate description of what's wrong with the bug filing and gives some indication to the user that their filing is welcome if it had more information. Ideally setting the resolution as Incomplete would also give the reporter a canned response about seeking support to get more information about the issue and then filing again, but we can say something along those lines ourselves when we set the resolution.
Severity: normal → enhancement
resolved needinfo?
Or maybe a status rather than a resolution?
re: #1 I think something like that still promotes the idea that we'll sit in a bug asking the reporter questions, and eventually come to steps to reproduce 15 comments down, when the idea is that people should be getting help from the support venues rather than hashing it out in a bug. #2 On the same note as above, I believe the idea is that bugs like that shouldn't get any further than comment 0 and it is more desirable that the reporter file another bug with a complete comment 0 after working out the details. Otherwise, I'm fine with it being a status.
A few thoughts: We are changing reporters, sometimes novice reporters, from "visit the page, type in the box, hit submit" to "visit the page, type in the box, work out exactly which radio button to click on, hit submit". The knob isn't the easiest bit of Bugzilla to understand. Is someone going to monitor the entire corpus of INCOMPLETE bugs to see if any have had more information added by reporters who couldn't work the knob? Keeping the bug open keeps it on the radar. A long time ago, we used to have two resolutions, REMIND and LATER, which were "temporary" resolutions - i.e. it was expected that bugs would come out of that resolution at some later point. They got deprecated pretty quickly, as it was realised that it's very good to have a bright line where you resolve bugs only when the issue is actually thought to be resolved. This proposal seems to backtrack on that a bit. Majken talks about two types of bugs: support requests and incomplete reports. I think they need different handling. A support request should be closed, full stop. There is no "please give us more information" about a support request. INVALID seems fine for a support request. We can use boilerplate "here's where you find support" text to be more useful. (Although anyone who's got this far with a support request has probably ignored three such notices already...) An incomplete bug report does indeed require more information. The question is: how best to indicate that this is the status both to the reporter and to anyone else reading the bug later? My assertion is that it is to post a comment which unambiguously asks a question. Then, a month later, someone can say "Closed due to lack of information from reporter". We did this all the time when I did QA. Jesse: The problem with a status is that statuses are not currently customisable, and resolutions are. But yes, a "need info" status would at least sidestep the problem above about resolving things which aren't resolved. Gerv
(In reply to comment #4) > A few thoughts: > > We are changing reporters, sometimes novice reporters, from "visit the page, > type in the box, hit submit" to "visit the page, type in the box, work out > exactly which radio button to click on, hit submit". The knob isn't the easiest > bit of Bugzilla to understand. Is someone going to monitor the entire corpus of > INCOMPLETE bugs to see if any have had more information added by reporters who > couldn't work the knob? Keeping the bug open keeps it on the radar. This isn't a change. We've defaulted to resolve on first pass in bugdays etc for a long time. Asa was pretty clear on the numbers that 90% of bugs never got updated, and a small fraction of those remaining were ever reproducible, so not resolving an at-first-pass worthless bug was a bad tradeoff. That said, what I really want here is to encourage filing a _new_ bug anyway, with more needed details. > A long time ago, we used to have two resolutions, REMIND and LATER, which were > "temporary" resolutions - i.e. it was expected that bugs would come out of that > resolution at some later point. They got deprecated pretty quickly, as it was > realised that it's very good to have a bright line where you resolve bugs only > when the issue is actually thought to be resolved. This proposal seems to > backtrack on that a bit. This is not a temporary status. this is an alternative to INVALID that is much more appropriate to the large numbers of bugs lacking required data/STR/etc. I fully expect that if we add it tomorrow, 95% of bugs marked as INCOMPLETE will never be reopened, or proceed to a useful state. > Majken talks about two types of bugs: support requests and incomplete reports. > I think they need different handling. > > A support request should be closed, full stop. There is no "please give us more > information" about a support request. INVALID seems fine for a support request. > We can use boilerplate "here's where you find support" text to be more useful. > (Although anyone who's got this far with a support request has probably ignored > three such notices already...) INVALID sends the wrong message, because its a valid problem, it just might not be a valid bug yet because we don't know whether its just some extension being wacky, etc. I don't know if INCOMPLETE is right for that, but I think its less bad to the reporter... > An incomplete bug report does indeed require more information. The question is: > how best to indicate that this is the status both to the reporter and to anyone > else reading the bug later? My assertion is that it is to post a comment which > unambiguously asks a question. Then, a month later, someone can say "Closed due > to lack of information from reporter". We did this all the time when I did QA. Yes, and as noted above, we moved away from this because that became a ton of overhead, and the bugs were not as useful as refiling with all needed data.
(In reply to comment #4) I think you may have misunderstood Majken's proposal. > A support request should be closed, full stop. There is no "please give us > more information" about a support request. INVALID seems fine for a support > request. As I understand it, RESOLVED INCOMPLETE is just a "nicer" and perhaps less inflammatory way to close the support request, full stop. I don't think she's proposing the bug be used to actually gather information. Often bugs are filed that do represent actual problems with Firefox, but the problem just isn't described accurately. INVALID can make the reporter feel like their problem is being dismissed. > An incomplete bug report does indeed require more information. The question > is: how best to indicate that this is the status both to the reporter and to > anyone else reading the bug later? My assertion is that it is to post a > comment which unambiguously asks a question. Then, a month later, someone can > say "Closed due to lack of information from reporter". We did this all the > time when I did QA. I think Majken's suggesting that when a bug report lacks essential information when filed, that instead of being left open to gather that information, it should be RESOLVED INCOMPLETE immediately, with a comment describing why it is incomplete, and encouraging the reporter to use the support venues available to them (IRC, forums, etc) to further look into the issue and then possibly file a new, more complete bug later on. The "make a comment and look into it 3 months later" part of triaging leads to people forgetting about the old, open, incomplete bugs.
OK... so the proposal is: "Let's have a new resolution called INCOMPLETE, because INVALID is rude" ? OK. That's reasonable. I remember having to suppress the occasional worry about being rude when closing bugs. Have we considered renaming INVALID instead? Does the name "INCOMPLETE" cover all the potential requirements? Or do we want something even more generic? Gerv
re: comment #6 Yes, gavin has it right. The other thing I was trying to convey that I didn't do properly, was in the case of the installer bug, we didn't clue into it on the first reports, but realized that people affected had a messed up user agent, allowing us to look back and see when we started getting reports by affected users and also some symptoms we hadn't caught on to when diagnosing the missing bookmarks (tabbed browsing was also broken). Preserving these bugs in a different category from INVALID would make a similar "audit" easier should someone want to try tracking down when users were first hit should a similar situation arise. Comment #7 - no, because people file bugs against things that aren't bugs. They clearly and concisely describe a problem that isn't a bug in Mozilla products. People seeking support might actually be experiencing a bug, they just haven't conveyed it clearly in the report. I'd personally be fine with renaming INVALID to something else, but there still is a use in differentiating between "not our problem" and "we don't have enough info to proceed"
OK, last question then: does anyone know of any other requests to add resolutions? While we are thinking about it, it's good to take everything into account. Then we don't end up adding them piecemeal and get several overlapping ones and confusion. I've just noticed that GNOME also use INCOMPLETE. (Perhaps everyone else knew that already.) Gerv
I did not know that. Interesting. I came up with INCOMPLETE independently, so it must be a good idea. ;)
To summarize neatly: INCOMPLETE = Missing info, generally not useful (i.e. Firefox crashes sometimes, I don't know why) INVALID = Clearly not a bug (i.e. Firefox doesn't render my tag soup properly) I don't know of any other resolution buckets, offhand.
I think someone should post this idea in the newsgroups and gather feedback from there as well.
Status: NEW → ASSIGNED
Has that happened yet? Also, do we want to? Feedback on whether this is a good idea, or feedback on what it should be called? If it's the former, I don't think we really want to. The idea is to optimize workflow, and if mconnor is right in comment #5 then feedback isn't going to make a difference. Although I think there's definitely value in getting feedback and setting a standard on how we want to deal with users when we're marking their bug as incomplete. Anyway, I've seen 5 bugs today that I wanted to mark as INCOMPLETE so I figured I'd better poke.
I've posted in mozilla.dev.general. Gerv
Discussion reposted to mozilla.dev.planning, m.d.general is being deprecated...
(In reply to comment #8) > [snip] because people file bugs against things that aren't bugs. They > clearly and concisely describe a problem that isn't a bug in Mozilla products. There are two separate cases there: - not a bug at all, working as intended in all respects, and - a bug in some other product or service (e.g. a remote server, or page content) > People seeking support might actually be experiencing a bug, they just haven't > conveyed it clearly in the report. This is a third case. > I'd personally be fine with renaming > INVALID to something else, but there still is a use in differentiating between > "not our problem" and "we don't have enough info to proceed" ... and "not anyone's bug, correct behavior". Gerv's posting about this bug on 3/26 said this change would happen "in a few days". That was over a week ago. So, what remains to be done or decided about this bug?
There has been discussion in the newsgroup about use and name. It looks like everyone is on board with use, so a name needs to be decided on, and a use policy written up.
I still want NOTOURBUG/NOTOURFAULT. INVALID is very ambiguous if you split it out into the different classes, maybe you can get rid of it and have better classifications to boot.
OK, let's finish this. The three proposals in the newsgroup were INCOMPLETE, INSUFFICIENT_DATA and LACKSINFO. As far as I'm concerned, they are all pretty much the same, with the first two having the advantage that they match what some other projects are using, the last two having a possible advantage of being a bit less ambiguous, and the outside two having the advantage of being short. Let's just go with INCOMPLETE. Unless someone comes up with a tie-breaking fact, I'll create it in a couple of days time. Gerv
INCOMPLETE added to b.m.o configuration. Gerv
Status: ASSIGNED → RESOLVED
Closed: 18 years ago
Resolution: --- → FIXED
We need to add this to https://bugzilla.mozilla.org/page.cgi?id=fields.html#status My proposed text: "The problem is vaguely described with no steps to reproduce, or is a support request. The reporter should be directed to http://mozilla.com/support for help diagnosing the issue. If there are only a few comments in the bug, it may be reopened only if the original reporter provides more info, or confirms someone else's steps to reproduce. If the bug is long when enough info is provided a new bug should be filed and the original bug duped forward."
If you actually want triager's to start using this new resolution you need to actually tell them about it. I would post to the forums, newsgroups and a blog (Mozilla Developer News maybe). Or of course you have all the triager's email addresses.
majken: can you or another member of the QA community look at how to inform people about the new resolution? I agree it should be added to fields.html; that would require a custom copy of that template. Please open a new bug for this work. Gerv
So I was looking a bug someone resolved INCOMPLETE and came up with another name that I think is a good alternative to INCOMPLETE. I'm not proposing we change this, I just want to log it in the bug in case there is a reason to/call to change it in the future. --> CANTFIX
CANTFIX (or NOTOURBUG or similar) was suggested in the newsgroups. I don't think it's an alternative to INCOMPLETE, but rather an augment. But it would be a separate bug :-) Gerv
When looking at: http://developer.mozilla.org/en/docs/What_to_do_and_what_not_to_do_in_Bugzilla#Resolving_bugs_as_INCOMPLETE https://bugzilla.mozilla.org/page.cgi?id=fields.html#status I can't find anything about the "INCOMPLETE" resolution. Regarding comment 21, I don't think support requests should be resolved INCOMPLETE. They seem more INVALID to me, in most cases.
Martijn: I believe there's a bug about at least one of those URLs. If there isn't, file one :-) Re: support requests, sometimes it's a question of tact :-) Gerv
Ok, thanks, it was filed as bug 379033.
Martijn - the idea with using INCOMPLETE is that it's entirely possible they're looking for support because they're hitting a bug. IMO it's useful to leave those not marked as invalid for things like figuring out when users started to hit a bug, and for catching differences between symptoms. If they come back and confirm that support took them through something that fixed it that isn't a bug, then it'd be appropriate to change the resolution to INVALID. There's a lot of valuable information that can be gained from seeing what people are filing, I think it's a win to leave them in their own space so we can ignore INVALID bugs. Also, if someone is asking for support because they're hitting a real bug, I think the odds are lower that they'd come back and update us on their issue if we right away tell them it's "INVALID." INVALID should mean "we know what's going on and that's not our problem" INCOMPLETE means "we don't have enough info to know what's going on." (to oversimplify it at least a little bit)
Component: Bugzilla: Keywords & Components → Administration
Product: mozilla.org → bugzilla.mozilla.org
You need to log in before you can comment on or make changes to this bug.