765768 - [research] analyze elastic search unified search results

Status: RESOLVED FIXED

(support.mozilla.org :: Search, defect, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Once elastic search unified search results lands, we need to:

1. analyze the search results compared to the elastic search bucketed search results

2. come to a concensus of: a) they look good enough, or b) they need more work

3. in the case of b) we need to write up new bugs to implement necessary changes to make the search results better


The outcome of this bug is analysis from number 1 and either thumbs-up or more bugs.

Putting this in the 2012.12 sprint as a P1 and with 2 points, though that's kind of a guess and this is not really an implementation bug but rather an analysis-and-talk-with-other-people bug. Also, it might make sense for Matt to do the analysis.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

cc:ing Matt on this since I'm pretty sure he's going to be driving the work here.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Fixing the summary so it doesn't look like the other "elastic search unified search results" bugs.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Assigning this one to Matt.

Matt: If you need help or anything, let me know.

Comment 4

[Matt Grimes [:Matt_G]]

Fantastic Will. I am meeting with Michael Verdi later today. We'll put together a testing plan and get moving.

Comment 5

[Matt Grimes [:Matt_G]]

Hey Will. We finished our initial testing. I am still hoping you can explain what happened in that example search. That issue aside, we were able to identify a few issues. I'll open new tickets for them.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Just to clarify, you're adding blockers to this bug which I infer to mean that this bug requires those things to be complete before we can close this. And that means that we're staying with bucketed search until all of these things are implemented. Is that correct?

The reason I ask is that unified search is almost assuredly better than bucketed search already. Like I explained in our discussions on Vidyo, I was hoping we could do some minor tweaking with the existing infrastructure, then switch to unified search for everything and ditch all the bucketed search code. Then we could continue honing unified search going forward.

If it's really the case that the unified search results are bad enough that we need to do things we can't currently do (like bug #768240), then so be it. But it means we're not switching to unified search results for a while.

Also, has any of the analysis done for this bug so far been documented anywhere?

Comment 7

[Matt Grimes [:Matt_G]]

Hey Will. I think that with the removal of contributor posts it will be as good as bucketed search or better. We are documenting our testing in a spreadsheet right now and we'll take another run at it later this week to confirm. I was adding those other bugs to get it really crisp and accurate based on what we've seen so far in our testing. I don't think they'll be blockers before pushing it live. They can probably come further down the road, but I'll need confirmation from a few other folks before I can make that call.

Comment 8

[Kadir Topal [:atopal]]

I'd suggest that we push this to 50% of our users as soon as possible, take a look at the CTR for both and then make a decision to move to unified search or not. What do you guys think?

Comment 9

[Matt Grimes [:Matt_G]]

That sounds good to me Kadir. I'll continue to test on my end as well to look for future improvements and content tuning opportunities.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

Given that, I'm going to nix the uncompleted bugs from the blocker list for this bug, change the waffle to 50% and then mark this as FIXED.