819755 - Add a Google Analytics tracking where we have zero search results

Status: VERIFIED FIXED

(support.mozilla.org :: Search, defect, P2)

Description

[Kadir Topal [:atopal]]

Currently we have no idea how often people are served with "no results" on our search results page. Please add url parameters, so we can count those cases and please add different ones for search results on the main search and the AAQ.

main: zeroresults=main
AAQ: zeroresults=aaq

If there is a simpler way to identify cases where we serve zero results, please feel free to suggest that.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Is it better to track this sort of thing (and possibly counts-sorts of things) with statsd and graphite? For example, we already track ES timeouts and errors in graphite.

Comment 2

[Ricky Rosario [:rrosario, :r1cky]]

Adding a URL parameter would force us to redirect to the new url with the parameter.

I think this is a good case for using custom variables in Google Analytics.

I started that conversation with Ibai, and we are planning on starting to add some custom variables in Q1.

Comment 3

[Ibai Garcia [:ibai]]

Agree with Ricky. No point on adding more parameters.

Comment 4

[Ibai Garcia [:ibai]]

Question: What's the learning that we are looking to get from this?

I'm thinking that instead of a page level custom variable who may want to use an Event whenever this happen. But it all depends of the list of questions we are trying to answer with this effort.

I.e. if we only want to know the amount of searches with no results, events are cleaner to implement.

Comment 5

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to ibai from comment #4)
> Question: What's the learning that we are looking to get from this?
> 
> I'm thinking that instead of a page level custom variable who may want to
> use an Event whenever this happen. But it all depends of the list of
> questions we are trying to answer with this effort.
> 
> I.e. if we only want to know the amount of searches with no results, events
> are cleaner to implement.

Events seems like a good idea to me!

Comment 6

[Kadir Topal [:atopal]]

Yeah, the question is how often we are displaying zero results and for whom?

Comment 7

[Ricky Rosario [:rrosario, :r1cky]]

Making this a 1pter because it is similar to other bugs in this sprint.

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Kadir, Ibai:

Do we only want to track the fact that zero results were returned? As in:

    _gaq.push(['_trackEvent', 'Zero Search Results']);


Or am I missing something?

Comment 9

[Ibai Garcia [:ibai]]

We may want to add the query?

If so it will be something like:

    _gaq.push(['_trackEvent', 'Zero Search Results', $query]);

Thanks for putting this together.

Ibai

Comment 10

[Ricky Rosario [:rrosario, :r1cky]]

PR:
https://github.com/mozilla/kitsune/pull/1080

Landed on master:
https://github.com/mozilla/kitsune/commit/4d1988327b0c68ff0dabf2b7c0ed8c40bb2e7764

Comment 11

[Ricky Rosario [:rrosario, :r1cky]]

Deployed to prod now.

Comment 12

[Ibai Garcia [:ibai]]

Data is being collected in GA.

Thanks for this!

Comment 13

[Kadir Topal [:atopal]]

Data is in and the answer is that about 5% of searches have zero results. In other words: We might be able to improve our CTR by up to 5 percentage points if we can offer those users something more helpful than an empty page.

Comment 14

[Will Kahn-Greene [:willkg] ET needinfo? me]

Do we know which searches users are doing that come up with zero results?

Comment 15

[Kadir Topal [:atopal]]

Yeah, we do, but it's a very, very long tail. From 15,000 searches the top search is "deutsch" and has been performed 82 times. The next one on the list has 55 searches (xnxx), and it tails off from there. 

The whole top 10 is skewed towards German terms, which makes (some) sense since German is the biggest source of visitors after English and whereas we have loads of forum data to feed to English speaking users we have way less content for German speaking users.

As far as I can see there are some spelling mistakes, but also a lot of searches that are obviously not suited for SUMO. I guess there are several things we can do:

1. Correct spelling mistakes in different languages
2. Make sure we cover synonyms in different languages
3. Offer a generic "what you might be looking for" page with top articles
4. Offer people to use Google search.

Comment 16

[Kadir Topal [:atopal]]

1. and 2. are on our roadmap, and I filed bug 839041 for 3.