Advanced searching using "keyword" "contains none" or "does not match regex" field returns incorrect results

RESOLVED FIXED in Bugzilla 3.6

Status

()

Bugzilla
Query/Bug List
--
major
RESOLVED FIXED
7 years ago
7 years ago

People

(Reporter: Graeme Coates, Assigned: Max Kanat-Alexander)

Tracking

({regression})

Bugzilla 3.6
regression
Bug Flags:
approval +
approval3.6 +
blocking3.6.2 +

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

7 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 GTB6 (.NET CLR 3.5.30729)
Build Identifier: Bugzilla 3.6

Searching using the keyword field with "contains none of the words" and "does not match regular expression" do not filter the results as expected and return results that are incorrect where a bug has more than one keyword. 

For example, I search using the condition:
"Keywords" "Contains none of the words": testing

A bug has keywords: live, testing  - this bug is returned within the list for that query. 
A bug has keywords: testing, live  - this bug is returned within the list for that query. 
A bug has keyword: testing - this bug is not returned for that query.

Using regexp searching is no better either and you can even produce some logically impossible results:

eg. advanced search: 

"Keywords" "contains any/all of the words" : testing
with advanced boolean chart search at the same time:
"Keywords" "contains none of the words" : testing

- logically this will return nothing at all times, and yet will return results here!

This is easily reproducible on landfill using either 3.6 or CVS - this is a **massive** inconvenience when you use keywords to categorise your bugs.


Reproducible: Always

Steps to Reproduce:
1. Goto advanced search 
2. Search for bugs with "keyword" "none of the words" set to a given keyword where bugs exist with that keyword in combination with another.
3. Results wrongly include results with that keyword listed (change the columns to view them)
Actual Results:  
Bugs with keywords that should be excluded are included.

Expected Results:  
If you search for a bug with "none of the words" set for the keyword field, then a bug with that word in the keywords field shouldn't be included.
(Reporter)

Updated

7 years ago
Summary: Advanced searching using "keyword" field returns incorrect results when >1 keyword → Advanced searching using "keyword" field returns incorrect results when >1 keyword present in bugs
(Reporter)

Comment 1

7 years ago
Further investigation - the query constructed for the above example has a series of left joins, including one to the keywords table, joining on the bug_id. For bugs with >1 keyword, you have >1 row for the bug in the keywords table. Further along in the query, the following WHERE clause excludes all the entries with the specified keyword name as follows:

WHERE ((NOT (keyworddefs_.name REGEXP '(^|[^a-z0-9])testing($|[^a-z0-9])')) AND ...

Since this acts on keyworddefs_.name (joined to keywords on the keyword_id), it fails to filter out all the rows for a bug that has multiple keywords (since you'd have multiple rows for that one bug) - thus (after the subsequent "group by bug_id" at the end of the query) you haven't got rid of all the bugs you wanted to and the bugs are wrongly included. 

This looks to be a regression due to the implementation of Bug 58731, and it looks like this was predicted by comment 42, Bug 58731 (!!) - though not only does the "regexp" option not work as in the comment, but now the "none of the words" search option for keywords is broken (where it previously worked perfectly in 3.4.6).
(Assignee)

Updated

7 years ago
Severity: major → normal
Depends on: 58731
Keywords: regression
Hardware: x86_64 → All

Updated

7 years ago
Version: unspecified → 3.6

Comment 2

7 years ago
In bugzilla 3.6 advanced search "Keywords" "contains none of the words" returns no bugs without any keywords. I think it's wrong. In bugzilla 3.4 search with the same criterias returns all bugs without keywords.

https://landfill.bugzilla.org/bugzilla-3.6-branch/buglist.cgi?keywords=funny&query_format=advanced&keywords_type=nowords

returns 755 bugs

https://landfill.bugzilla.org/bugzilla-3.4-branch/buglist.cgi?keywords=funny&query_format=advanced&keywords_type=nowords

returns 8638 bugs
(Reporter)

Comment 3

7 years ago
(In reply to comment #2)
> In bugzilla 3.6 advanced search "Keywords" "contains none of the words" returns
> no bugs without any keywords. I think it's wrong. 

Agreed - it's because there is no test for NULL keywords when running "none of the words" (the LEFT JOIN to both keywords and keyworddefs results in rows with null entries in keyworddefs_.name).

Comment 4

7 years ago
Is anyone working on a fix for this?  The issue is killing my workflow as I use various keywords to manage bugs, but I could spend a lot of time trying to patch it as my knowledge of neither Perl nor the Bugzilla codebase is recent.
(Assignee)

Comment 5

7 years ago
Confirmed by number of reports.
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Advanced searching using "keyword" field returns incorrect results when >1 keyword present in bugs → Advanced searching using "keyword" "contains none" or "does not match regex" field returns incorrect results
Target Milestone: --- → Bugzilla 3.6
(Assignee)

Updated

7 years ago
Duplicate of this bug: 563144
(Assignee)

Updated

7 years ago
Flags: blocking3.6.2+
(Assignee)

Comment 7

7 years ago
Created attachment 453610 [details] [diff] [review]
v1

Okay, this fixes keywords by making them use multiselect_negative for the negative search types.
Assignee: query-and-buglist → mkanat
Status: NEW → ASSIGNED
Attachment #453610 - Flags: review?(bugzilla)
(Assignee)

Comment 8

7 years ago
Created attachment 453611 [details] [diff] [review]
v2

Oh, a tiny change--fixing the arguments wasn't necessary.
Attachment #453610 - Attachment is obsolete: true
Attachment #453611 - Flags: review?(bugzilla)
Attachment #453610 - Flags: review?(bugzilla)

Comment 9

7 years ago
Comment on attachment 453611 [details] [diff] [review]
v2

Works fine and looks good. It now also catches bugs with no keywords, which is fine. r=LpSolit
Attachment #453611 - Flags: review?(bugzilla) → review+

Updated

7 years ago
Flags: approval3.6+
Flags: approval+

Comment 10

7 years ago
Note that you have a tiny bitrot to fix before committing this patch.
(Assignee)

Comment 11

7 years ago
Created attachment 456104 [details] [diff] [review]
v2 (3.6)

3.6 requires a slightly different patch, because it doesn't have do_search_function.
Attachment #456104 - Flags: review?(LpSolit)

Comment 12

7 years ago
Comment on attachment 456104 [details] [diff] [review]
v2 (3.6)

r=LpSolit
Attachment #456104 - Flags: review?(LpSolit) → review+
(Assignee)

Comment 13

7 years ago
Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/trunk/
modified Bugzilla/Search.pm
Committed revision 7267.

Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/3.6/
modified Bugzilla/Search.pm
Committed revision 7120.
Status: ASSIGNED → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
(Assignee)

Comment 14

7 years ago
Created attachment 456361 [details] [diff] [review]
Fix search.t for now-passing tests

This fixes xt/search.t to account for the fact that certain tests are now passing that were previously marked "known broken".
Attachment #456361 - Flags: review+
(Assignee)

Comment 15

7 years ago
Committing to: bzr+ssh://bzr.mozilla.org/bugzilla/trunk/
modified xt/lib/Bugzilla/Test/Search/Constants.pm
Committed revision 7290.
You need to log in before you can comment on or make changes to this bug.