Closed Bug 42946 Opened 24 years ago Closed 18 years ago

Bugs filed at same time causing one to be duplicated.

Categories

(Bugzilla :: Creating/Changing Bugs, defect, P2)

Product:
Bugzilla
Component:
Creating/Changing Bugs
Version:
2.10
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Bugzilla 3.0

People

(Reporter: dobbinsj, Assigned: bugzilla-mozilla)

References

Details

(Keywords: selenium)

Attachments

(2 files, 5 obsolete files)

Patch v1.4: Really fix nits
7.50 KB, patch
bugzilla-mozilla
: review+
Details | Diff | Splinter Review
selenium test, v1
1.02 KB, patch
LpSolit
: review+
Details | Diff | Splinter Review 
Lately I've noticed a lot of bugs being duplicated. There will be 2 (or more) 
bug reports that are exact duplicate execpt for the assigned number. I've looked 
at several and noticed that the lowest numbered bug of the pair has the same 
filing time as diferent bug filed by another reporter.
Sounds to me like the table was locked while the update action from the first 

person was being performed, and the person submitting the second bug got 

impatient and hit the refresh button to repost it.  Except that it had already 

been posted, it was just waiting for a database lock.  So now it gets posted 

twice.


There is some discussion of this problem in bug 22353.
I might have a patch against this. It checks the last bug submitted by the user
and compares the "subjects".  This however adds an additional mysql call per bug
submission. I'll attach the patch once I test the sql.
*** Bug 46843 has been marked as a duplicate of this bug. ***
Please also compare the Description.  I know that subjects should be globaly 
unique --blah blah blah--, but perhaps someone is writing 3 different cases 
that deserve the same subject. [Possibly give a warning, hold the data and 
allow submission.]
I think this could happen if you hit the Submit button twice in succession
accidentally.  I've done that sort of thing before on show_bug.cgi and midaired
myself.
Target Milestone: --- → Bugzilla 2.16
Priority: P3 → P2
moving
Assignee: tara → myk
Component: Bugzilla → Creating/Changing Bugs
Product: Webtools → Bugzilla
Version: other → 2.10
*** Bug 99812 has been marked as a duplicate of this bug. ***
doron: do you have the patch for this? If so, attach it and put the milestone
back :-)

Gerv
Target Milestone: Bugzilla 2.16 → Bugzilla 2.18
This got lost in the great harddisk crash of march 2001...
I think this feature is nice. I see almost once a day one of those 'double
entry' duplicates. It would be really nice to detect them. 
If this feature can be (re-)implemented in a relatively short time, it would
save a lot of time to the whole community.
It would even allow to diminish the bug rate that is so much taken into account
by the in-famous MozillaQuest site ;)
Someone elsewhere talking about something completely different mentioned
something that gave me a good idea for this...

Embed a unique identifier of some sort in the enter_bug.cgi html as a form
element...  probably the current timestamp would be sufficient as it's very
unlikely that the same user would be able to load two different forms in
different windows within a second of each other.

stash the IDs in a separate table (perhaps the tokens table?) and expire them
after 24 hours.  When the bug is posted, delete the token.  If the token doesn't
exist, bounce the submission as a duplicate.

On the other hand, that may cause problems with automated bug submissions... it
might work better to stash the submitted tokens for a short period of time, and
bounce the submissions as a duplicate if the token already exists in the table
(thus has already been used recently).
just to reiterate points from Comment 5:
msie makes it very easy to open the same exact page in multiple windows.
and I use session history (in mozilla2-5).

If i submit 5 bugs based on a single template which i'm only using once, then 
i'll quickly run into your thing, and i'll complain.

I don't want to make a template, i just want to use fields that i've already 
filled out.



OK, so the obvious thing to do is to store the ticket number that was created
with the token when we file it as having been used, and compare the ticket with
the new one if the token gets reused, and only flag an error if they actually
match...
*** Bug 120724 has been marked as a duplicate of this bug. ***
*** Bug 148223 has been marked as a duplicate of this bug. ***
*** Bug 171205 has been marked as a duplicate of this bug. ***
*** Bug 121848 has been marked as a duplicate of this bug. ***
ok, on irc i basically suggested comment 12 + a variant which would satisfy
comment 13: instead of simply bouncing bug submissions when we think it's an
errant accidental filing, we should provide a link to the successful report and
provide the filer with the filled bug submission form and a new submission id.
if the user wants to file the new bug, the user simply clicks commit again.

the other thing we could do is provide an editable field which would factor into
the token. that way filers who behave as described in comment 13 could simply
change their portion of the token to avoid the extra roundtrip. I don't think
it'd be very easy to explain such a field to the end user, so it's just an idea...

I wonder what we should do if one user sends a bug submission with a token owned
by another user.  normally i'd say this is an invasion of privacy, but if
someone actually posted a bug submission link complete with a token then perhaps
it would be best to complain...  I think what we should do is say that the bug
report used a token assigned to another user and simply reproduce the bug entry
form with a new token without providing a link to the filed bug. this should
solve both the privacy factor and the filing duplicate factor.
*** Bug 158549 has been marked as a duplicate of this bug. ***
Coming back to comment #12: instead of only applying the timestamp, reduce the
risk by adding a random number (or the time fraction), anyhow a small risk
remains ...

On comment #19: I did a hack for a customer some time ago implementing somwthing
like this for a order tracking system. Maybe I can help introducing this here
also...
Attached patch first draft to show the idea (obsolete) — — Splinter Review 
Hi all,

the applied patch plus the other two files in the next attachment will give an
overview my proposal for a solution. Please review and give comments. 

Be aware that this solution is using the UUID library. In case you don't like
this we can change this to rand() (I was unsure if this is really rand()).
>Embed a unique identifier of some sort in the enter_bug.cgi html as a form
>element...  probably the current timestamp would be sufficient as it's very
>unlikely that the same user would be able to load two different forms in
>different windows within a second of each other.

Note that the advent of bookmark tab groups makes this more likely (although
it's still unlikely to happen), as it's now possible to hit one bookmark and
simultaneously open several tabs, each of which could be pointed at the
enter_bug.cgi.  So it's worth adding a random number.

Nonetheless, wouldn't comparing the description of the most recently posted
couple of bugs be a much easier solution technically and handle 99% of the cases?
I basically agree. But there're some more points here: 
 
1. In case two users are generating an issue with the same summary, the summary 
check would fail or we have to include the creator in the query (fine for me), 
because this is handled by standard duplicate check functionality. 
 
2. The short_desc field is handled as a fulltext search field (at least in mysql). In 
case of installations like bugzilla.mozilla.org or other installations the lookup for 
recent summaries will take more time. 
 
3. The unique/radom id prevents to check for same issue entries only to one sender. 
In case of anonymous bug entries this would prevent topic 1.
Attached patch Second try (obsolete) — — Splinter Review 
Attached there's another patch that is handling this issue. I used this in an
installation for some weeks (with a lot on IE users) and the number of
duplicated bugs due to pressing the refresh button or other issues has been
reduced 'dramatically' (about 2-3 bugs per day before -> now ...zarro...)
Attachment #140247 - Attachment is obsolete: true
Pushing out bugs that aren't blockers.  If someone's working on one of these, we
can move it back.
Target Milestone: Bugzilla 2.18 → Bugzilla 2.20
here's a solution which may be better:

Instead of messing around with tokens (creating the situation in comment 13), 
why not just store a checksum of all the form data posted to post_bug.cgi in 
the db?   Then, if someone posts again, just check for any recent entry in the 
db with the same checksum.  

This solution has the advantages that you don't need to worry about token 
uniqueness or randomness, or which user is posting; and you don't have to store 
arbitrary data (like short_desc) in db tables

(In reply to comment #28)
> here's a solution which may be better:
> 
> Instead of messing around with tokens (creating the situation in comment 13), 
> why not just store a checksum of all the form data posted to post_bug.cgi in 
> the db?   Then, if someone posts again, just check for any recent entry in the 
> db with the same checksum.  
> 
... but not any date values. In addition, what means recent? An option is to
provide a new field (e.g. creation_hash) that contains a hash of most of the
fields and is unique. 

In case we find a hit:

- there might be the same issue twice
- another issue with the same hash (very unlikely, but ... ): should be checked
by comparing descriptions
That would not solve the most common duplication problem that I see:  The person
submits the bug.  She realizes that some details or left out or not enough
people were on the CC list or something trivial, so browser back button is used
to go and edit the original report, and hit submit again.  Slightly different
bug, but the same thing.

She should have just submitted changes to the bug that was just opened, but
teaching users is difficult.  I've seen this behavior in multiple people (note
if people do the "back button" mistake on process_bug.cgi, they'll get a midair
collision against themself so this protection already exists for open bugs).  It
would be nice to be able to protect the database from stupid users and force
them to visit the new bug instead.
This bug has not been touched by its owner in over six months, even though it is
targeted to 2.20, for which the freeze is 10 days away. Unsetting the target
milestone, on the assumption that nobody is actually working on it or has any
plans to soon.

If you are the owner, and you plan to work on the bug, please give it a real
target milestone. If you are the owner, and you do *not* plan to work on it,
please reassign it to nobody@bugzilla.org or a .bugs component owner. If you are
*anybody*, and you get this comment, and *you* plan to work on the bug, please
reassign it to yourself if you have the ability.
Target Milestone: Bugzilla 2.20 → ---
*** Bug 292084 has been marked as a duplicate of this bug. ***
(In reply to comment #12)
> stash the IDs in a separate table (perhaps the tokens table?) and expire them
> after 24 hours.  When the bug is posted, delete the token.  If the token doesn't
> exist, bounce the submission as a duplicate.
> 
> On the other hand, that may cause problems with automated bug submissions... it
> might work better to stash the submitted tokens for a short period of time, and
> bounce the submissions as a duplicate if the token already exists in the table
> (thus has already been used recently).

What I had in mind was something like:
- enter_bug.cgi creates an entry in the 'tokens' table;
- post_bug.cgi removes the token when the new bug has been created;
- if the user reloads the page, post_bug.cgi detects that the token is missing
and refuses to create a new bug.

Does this work?

*** Bug 286647 has been marked as a duplicate of this bug. ***
But what about the case where, rather than explicitly creating a bookmarkable
template for a series of similar bugs, you just press the back button adjust the
relevant details and post again?

Could something fancy be done client-side (e.g. javascript) so that on pressing
the "Submit" button you get something like:
 "You have already submitted this bug form. Do you really want to create an
additional bug report?"
*** Bug 297468 has been marked as a duplicate of this bug. ***
I want to implement what LpSolit said in comment #33. With the addition that if the cgi->params doesn't have the token param, the token checking is not done (do allow automated scripts to work.. even if they might have a webservices interface in future)

Still thinking about instead of removing the token, storing the new bug id there. Then we can make the message like: "Visit the bug". Perhaps also another button to force submission of the bug? (another param that updates the original token). This would work when the user really did submit a totally new bug and is far easier than figuring out how to detect such cases.
Assignee: myk → bugzilla-mozilla
QA Contact: mattyt-bugzilla → default-qa
Target Milestone: --- → Bugzilla 2.24
This adds a hidden token to create.html.tmpl and create-guided.html.tmpl. If the token field is NOT present in the form submission, post_bug.cgi will always allow the bug to be created. This means that custom create templates will need to be changed (relnote). Allowing the token field not to be there is a feature to ensure any existing bug submission scripts are not broken.

When the bug has been created, the token data is updated to include the bugnr that was just created.

When post_bug.cgi detects that the token has been used before, it retrieves the old bugnr and passes that on to an 'are-you-really-sure?' template. This template encourages the user to visit the existing bug.

The template also allows the user to file the bug anyway. For this purpose it includes another hidden field called 'ignore_token' which has the older bugnr.

The ignore_token is essentially another token, because otherwise the following situation could be possible:
1. User posts a bug
2. User refreshes page to see latest comments
3. User sees 'are-you-sure-page'.
4. User doesn't read the text and clicks 'File bug again'
5. User refreshes bug again

In step #5, the ignore_token is already set. Without that being in some way unique, Bugzilla will in step #5 silently create new bugs without warning the user  (because of the ignore_token).

I could also have created a new token in the 'are-you-sure' page instead of adding the ignore_token parameter. The ignore_token param however allows me to check if the user is constantly refreshing the bug and clicking 'File bug again'...
Because ignore_token is only added by the 'are-you-sure-template', the ignore_token verification can only fail if the user is refreshing the page / submitting the same bug[1]. If the user had used the back button and the existing enter_bug.cgi form, there wouldn't be an ignore_token field which could fail (a new ignore_token would be generated). Meaning: if ignore_token fails, the user had no way of changing any of the fields, so I can be certain 'File bug anyway' should NOT be allowed. To enforce that, the 'are-you-sure?' template will NOT allow the user to submit the bug again when ignore_token fails to verify.

[1] Well.. except in one theoretical case that will not happen in practice.
Attachment #140248 - Attachment is obsolete: true
Attachment #141627 - Attachment is obsolete: true
Attachment #222746 - Flags: review?(LpSolit)
Status: NEW → ASSIGNED
Comment on attachment 222746 [details] [diff] [review]
Patch v1: Bugzilla HEAD 2006-05-20

>Index: post_bug.cgi

>+    trick_taint($token);

Nit: you don't use $token in a SQL query here. So you should detaint it later.


>+    my $prev_bugnr = $event;

Nit: if you used $prev_bugpr directly when calling GetTokenData(), you would save this line.
Nit: I highly prefer $old_bug_id than $prev_bugnr. Please fix that before committing your patch.


>+if ($token) {
>+    $dbh->do('UPDATE tokens SET eventdata = ? WHERE token = ?', undef, 
>+             ("createbug:$id", $token));
>+}

Nit: You should detaint $token here as you now use it.



>Index: template/en/default/bug/create/confirm-create-dupe.html.tmpl

>+[% PROCESS global/variables.none.tmpl %]
>+[% PROCESS "global/field-descs.none.tmpl" %]

field-descs.none.tmpl already processes variables.none.tmpl, so calling variables.none.tmpl here is useless.


>+  <form name="Create" id="Create" method="post" action="post_bug.cgi">

Nit: 'create' instead of 'Create'.


>+  <input type="hidden" name="ignore_token" value="[% bugid FILTER html %]">
>+  <input type="submit" value="File bug again">
>+  </form>

Nit: the indentation is incorrect. Write:
<form>
  <input>
  <input>
</form>


Please fix these nits in a new patch and carry my r+ forward. r=LpSolit
Attachment #222746 - Flags: review?(LpSolit) → review+
Flags: approval?
Attached patch Patch v1.3: Fix nits, terms.bug (obsolete) — — Splinter Review 
Fixes nits. Carrying forward r+ as per comment 39.

Also add forgotten terms.bug:
<LpSolit> not ok 163 - template/en/default/bug/create/confirm-create-dupe.html.tmpl contains invalid bare words (e.g. 'bug') --WARNING
<LpSolit>   <input type="submit" value="File bug again">
<LpSolit> bkor: File $terms.bug again
<LpSolit> bkor: there are several places in this template where you use bug alone
<LpSolit>   <p>If you are sure you used the same form to submit a new bug, click 'File bug
<LpSolit>   again'.<p>
<LpSolit> bkor: should be value="File [% terms.bug %] again" btw, not what I wrote above
<bkor> LpSolit: already changed that, running tests again
<bkor> ok, passes all tests
<LpSolit> bkor: is the UI still correct? ;)
<LpSolit> no missing whitespaces? ;)
<bkor> yeah, unexpected missing whitespace is annoying, but looks fine
Attachment #222746 - Attachment is obsolete: true
Attachment #224383 - Flags: review+
Seems I made the patch before I saved all the files, because the terms.bug template changes where missing. :-(

Another fixed nit:
<LpSolit> bkor: another nit I missed in my previous review: $cgi->param('ignore_token') ne $old_bug_id
<LpSolit> bkor: both are integers, so use !=
Attachment #224383 - Attachment is obsolete: true
Attachment #224385 - Flags: review+
Flags: approval? → approval+
Checking in enter_bug.cgi;
/cvsroot/mozilla/webtools/bugzilla/enter_bug.cgi,v  <--  enter_bug.cgi
new revision: 1.134; previous revision: 1.133
done
Checking in post_bug.cgi;
/cvsroot/mozilla/webtools/bugzilla/post_bug.cgi,v  <--  post_bug.cgi
new revision: 1.146; previous revision: 1.145
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/template/en/default/bug/create/confirm-create-dupe.html.tmpl,v
done
Checking in template/en/default/bug/create/confirm-create-dupe.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/bug/create/confirm-create-dupe.html.tmpl,v  <--  confirm-create-dupe.html.tmpl
initial revision: 1.1
done
Checking in template/en/default/bug/create/create-guided.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/bug/create/create-guided.html.tmpl,v  <--  create-guided.html.tmpl
new revision: 1.30; previous revision: 1.29
done
Checking in template/en/default/bug/create/create.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/bug/create/create.html.tmpl,v  <--  create.html.tmpl
new revision: 1.58; previous revision: 1.57
done
Status: ASSIGNED → RESOLVED
Closed: 18 years ago
Keywords: relnote
Resolution: --- → FIXED
*** Bug 340837 has been marked as a duplicate of this bug. ***
*** Bug 340838 has been marked as a duplicate of this bug. ***
*** Bug 46843 has been marked as a duplicate of this bug. ***
Added to relnotes in bug 349423. Please let me know if I missed any critical information about this bug in the release notes.
Keywords: relnote
Flags: testcase?
Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/qa/4.0/
modified t/test_bug_edit.t
Committed revision 173.

Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/qa/3.6/
modified t/test_bug_edit.t
Committed revision 144.
Attachment #510043 - Flags: review+
Flags: testcase? → testcase+
Keywords: selenium
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: