[tracker] Improve quality of search results

RESOLVED WORKSFORME

Status

RESOLVED WORKSFORME
8 years ago
7 years ago

People

(Reporter: verdi, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

8 years ago
This one is hard to explain. Here's a video illustrating the problem (2min):
http://sumo.graymattergravy.com/videos/nonsense.ogv
The issue is that the user searched for "can't find downloaded files" and we don't have an article that addresses that specific problem. The search results though, instead of turning up forum posts like https://support.mozilla.com/rm/questions/754176 or KB articles like https://support.mozilla.com/en-US/kb/Downloads+window turn up results that are irrelevant with our least trafficked article listed as the #1 result: https://support.mozilla.com/en-US/kb/AUS+Update+XML+File+Malformed+200
The Question you linked isn't marked as solved, so is excluded from the default search. Maybe it's worth reconsidering that? We could search "questions with answers marked helpful" by default, instead, for example.

The article, "Downloads window" doesn't have the search term "find", so is unlikely to match, since I think we're in "ALL" mode now (see bug 607306). (If "can't" is treated as a unit, and I'd have to check that, it doesn't appear in the article, either.)

This situation is the reason for the keywords field: try sticking "can't find downloaded files" into the keywords of "Downloads window".

Comment 3

8 years ago
(In reply to comment #2)
> The Question you linked isn't marked as solved, so is excluded from the default
> search. Maybe it's worth reconsidering that? We could search "questions with
> answers marked helpful" by default, instead, for example.

Oh yeah, we should definitely change that. A question with answers marked helpful is as close to "solved" as you can get without hearing back from the original poster.

Cheng, any additional thoughts?

Comment 4

8 years ago
At some point we had the feature of emailing the user back if they haven't voted in X days.  What happened to that?

Also, having a helpful answer should be included in search results.  In fact, I think every thread should be included in search results since we have the whole me-too system.
(In reply to comment #4)
> In fact, I
> think every thread should be included in search results since we have the whole
> me-too system.

Just clarifying: all threads are indexed and searchable via advanced search, we're just talking about the default, basic search behavior here.
It's probably a matter of prioritization. People will already see those unanswered threads when they want to ask a question. But in the search results they would need to be the lowest priority.

Comment 7

8 years ago
In reply to comment #4)
> At some point we had the feature of emailing the user back if they haven't
> voted in X days.  What happened to that?

Not related to this bug, but yeah, we want that. It was a P2 I think, so it wasn't included in the release? (I'm guessing)

> Also, having a helpful answer should be included in search results.  In fact, I
> think every thread should be included in search results since we have the whole
> me-too system.

Agreed -- showing all results actually makes more sense and now that I recall, that's what we discussed when designing the new forum. I guess this change slipped the radar. Of course, unsolved/unanswered-helpful posts should have lower priority if all other weights in the search match are the same.

As James said elsewhere, this is likely an iterative process to get it exactly right.
(Reporter)

Comment 8

8 years ago
I guess I'm hoping that we can find some way to make the search smarter.

Unfortunately, for the search "can't find downloaded files" the article "AUS Update XML File Malformed 200" has a term in it's title and versions of all of them in the actual article. The problem is this article has absolutely nothing to do with the question. In fact all of the results are useless. But if I type that same search into Google, it knows what I'm talking about. Can we make our search smarter without being Google? 

From the other example in the video:
When you do a search for "find recently downloaded" on kitsune https://master.support.mozilla.com/en-US/search?q=find+recently+downloaded&x=12&y=18&page=1 you get "Quick find bar opens when typing in text fields" as the top article. It has one term - "find" in the tile and part of another - "download" in the article itself. It has no keywords. The "Downloads window" article doesn't show up at all in the results yet it has part of one term in it's title - "Downloads"and the actual term "downloaded" in the article itself plus it has the keyword "Downloads". It doesn't make sense that Downloads Window doesn't show up at all. 

I'll try adding new keywords to the Downloads Window article but with a different search I ran into the problem in bug 607306:
I added the keywords "Flash flashplayer player plugin crash adobe" to "The Adobe Flash plugin has crashed" but a search for "What to do when flashplayer crash" still gets zero results. Limiting the search to just "flashplayer crash" does give you "The Adobe Flash plugin has crashed" listed number one.
(Reporter)

Comment 9

8 years ago
(In reply to comment #2)
> 
> This situation is the reason for the keywords field: try sticking "can't find
> downloaded files" into the keywords of "Downloads window".

I did this and the result is that Downloads Window is now #1 for "can't find downloaded files" but AUS Update XML File Malformed 200 is #2 and the rest of the results are meaningless. Also Downloads Window again doesn't show up at all in a search for "find recently downloaded" - the users first attempt at this search.

So I think keywords certainly have their place and will help, we still need some way to make the search smarter.
I think we can do a lot better just by fixing two things:

* We're using the default match mode, ALL.
* We have almost no stop words.

That means (A) if you search for "what to do when flashplayer crash", the terms used are "what" "to" "do" "when" "flashplay" "crash", and (B) if the article is written using declarative language, "what" is probably not in it. Combining A and B excludes the article from the results.

We're fixing the stop word issue in bug 586034. Since this is clearly causing you headaches, I'm willing to up the priority (it's not currently a 2.3 blocker).

As I said in bug 607306, I think we should be using the EXTENDED match mode, not ALL. Unfortunately that switch isn't trivial, since it means we need to add some input handling to stop users from triggering Server Errors. So unless we get 2.3 buttoned up sooner than I expect, I don't think we'll be able to get to this.

Those will both make things better.

But before, and even with these changes, remember the first rule of SEO: "Content is king." The single best way to affect the results for a query is to change the content. If you don't want the AUS article to show up in searches for "download," reword it to remove the word "download." If you want more articles to come up, add content about finding downloaded files to them.

(Note that I'm talking about what ends up in the result set. When we talk about ordering the result set there are different things to consider.)
Summary: Need better search results when there isn't a great match → [tracker] Improve quality of search results
Depends on: 607306, 586034
(In reply to comment #10)
> I think we can do a lot better just by fixing two things:
> 
> * We're using the default match mode, ALL.
> * We have almost no stop words.

Yeah, things will especially improve once bug 607306 is fixed, as we will still show matches even if not all words are in the results.

To put things into perspective though, even without these two fixes, the Kitsune search demonstrably works much better than the current one according to tests Michael already did, so 2.3 will be an improvement regardless. Yay!
Depends on: 616569
Depends on: 623365
Depends on: 633338
I think we can close this, since we have fixed numerous things in individual bugs, and there is no way to actually call this bug done.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.