Closed Bug 562014 Opened 14 years ago Closed 14 years ago

Advanced searching using "keyword" "contains none" or "does not match regex" field returns incorrect results

Categories

(Bugzilla :: Query/Bug List, defect)

defect
Not set
major

Tracking

()

RESOLVED FIXED
Bugzilla 3.6

People

(Reporter: graeme, Assigned: mkanat)

References

Details

(Keywords: regression)

Attachments

(3 files, 1 obsolete file)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 GTB6 (.NET CLR 3.5.30729)
Build Identifier: Bugzilla 3.6

Searching using the keyword field with "contains none of the words" and "does not match regular expression" do not filter the results as expected and return results that are incorrect where a bug has more than one keyword. 

For example, I search using the condition:
"Keywords" "Contains none of the words": testing

A bug has keywords: live, testing  - this bug is returned within the list for that query. 
A bug has keywords: testing, live  - this bug is returned within the list for that query. 
A bug has keyword: testing - this bug is not returned for that query.

Using regexp searching is no better either and you can even produce some logically impossible results:

eg. advanced search: 

"Keywords" "contains any/all of the words" : testing
with advanced boolean chart search at the same time:
"Keywords" "contains none of the words" : testing

- logically this will return nothing at all times, and yet will return results here!

This is easily reproducible on landfill using either 3.6 or CVS - this is a **massive** inconvenience when you use keywords to categorise your bugs.


Reproducible: Always

Steps to Reproduce:
1. Goto advanced search 
2. Search for bugs with "keyword" "none of the words" set to a given keyword where bugs exist with that keyword in combination with another.
3. Results wrongly include results with that keyword listed (change the columns to view them)
Actual Results:  
Bugs with keywords that should be excluded are included.

Expected Results:  
If you search for a bug with "none of the words" set for the keyword field, then a bug with that word in the keywords field shouldn't be included.
Summary: Advanced searching using "keyword" field returns incorrect results when >1 keyword → Advanced searching using "keyword" field returns incorrect results when >1 keyword present in bugs
Further investigation - the query constructed for the above example has a series of left joins, including one to the keywords table, joining on the bug_id. For bugs with >1 keyword, you have >1 row for the bug in the keywords table. Further along in the query, the following WHERE clause excludes all the entries with the specified keyword name as follows:

WHERE ((NOT (keyworddefs_.name REGEXP '(^|[^a-z0-9])testing($|[^a-z0-9])')) AND ...

Since this acts on keyworddefs_.name (joined to keywords on the keyword_id), it fails to filter out all the rows for a bug that has multiple keywords (since you'd have multiple rows for that one bug) - thus (after the subsequent "group by bug_id" at the end of the query) you haven't got rid of all the bugs you wanted to and the bugs are wrongly included. 

This looks to be a regression due to the implementation of Bug 58731, and it looks like this was predicted by comment 42, Bug 58731 (!!) - though not only does the "regexp" option not work as in the comment, but now the "none of the words" search option for keywords is broken (where it previously worked perfectly in 3.4.6).
Severity: major → normal
Depends on: 58731
Keywords: regression
Hardware: x86_64 → All
Version: unspecified → 3.6
In bugzilla 3.6 advanced search "Keywords" "contains none of the words" returns no bugs without any keywords. I think it's wrong. In bugzilla 3.4 search with the same criterias returns all bugs without keywords.

https://landfill.bugzilla.org/bugzilla-3.6-branch/buglist.cgi?keywords=funny&query_format=advanced&keywords_type=nowords

returns 755 bugs

https://landfill.bugzilla.org/bugzilla-3.4-branch/buglist.cgi?keywords=funny&query_format=advanced&keywords_type=nowords

returns 8638 bugs
(In reply to comment #2)
> In bugzilla 3.6 advanced search "Keywords" "contains none of the words" returns
> no bugs without any keywords. I think it's wrong. 

Agreed - it's because there is no test for NULL keywords when running "none of the words" (the LEFT JOIN to both keywords and keyworddefs results in rows with null entries in keyworddefs_.name).
Is anyone working on a fix for this?  The issue is killing my workflow as I use various keywords to manage bugs, but I could spend a lot of time trying to patch it as my knowledge of neither Perl nor the Bugzilla codebase is recent.
Confirmed by number of reports.
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Advanced searching using "keyword" field returns incorrect results when >1 keyword present in bugs → Advanced searching using "keyword" "contains none" or "does not match regex" field returns incorrect results
Target Milestone: --- → Bugzilla 3.6
Flags: blocking3.6.2+
Attached patch v1 (obsolete) — Splinter Review
Okay, this fixes keywords by making them use multiselect_negative for the negative search types.
Assignee: query-and-buglist → mkanat
Status: NEW → ASSIGNED
Attachment #453610 - Flags: review?(bugzilla)
Attached patch v2Splinter Review
Oh, a tiny change--fixing the arguments wasn't necessary.
Attachment #453610 - Attachment is obsolete: true
Attachment #453611 - Flags: review?(bugzilla)
Attachment #453610 - Flags: review?(bugzilla)
Comment on attachment 453611 [details] [diff] [review]
v2

Works fine and looks good. It now also catches bugs with no keywords, which is fine. r=LpSolit
Attachment #453611 - Flags: review?(bugzilla) → review+
Flags: approval3.6+
Flags: approval+
Note that you have a tiny bitrot to fix before committing this patch.
Attached patch v2 (3.6)Splinter Review
3.6 requires a slightly different patch, because it doesn't have do_search_function.
Attachment #456104 - Flags: review?(LpSolit)
Comment on attachment 456104 [details] [diff] [review]
v2 (3.6)

r=LpSolit
Attachment #456104 - Flags: review?(LpSolit) → review+
Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/trunk/
modified Bugzilla/Search.pm
Committed revision 7267.

Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/3.6/
modified Bugzilla/Search.pm
Committed revision 7120.
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
This fixes xt/search.t to account for the fact that certain tests are now passing that were previously marked "known broken".
Attachment #456361 - Flags: review+
Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/trunk/
modified xt/lib/Bugzilla/Test/Search/Constants.pm
Committed revision 7290.
You need to log in before you can comment on or make changes to this bug.