860024 - [Meta] Are users satisfied with the search feature?

Status: RESOLVED INVALID

(developer.mozilla.org Graveyard :: General, defect)

Description

[David Bruant]

What problems would this solve?
===============================
The MDN team would have a better understanding of what people search. This knowledge can help dramatically improve the quality of search results.

Who would use this?
===================
MDN team

What would users see?
=====================
Gradually better search results eventually

What would users do? What would happen as a result?
===================================================
Users don't need to do anything more than usual. We just need to gather what they're searching and do statistics of that.

Is there anything else we should know?
======================================

Comment 1

[David Bruant]

This bug is for search stats at first. Based on this data, we'll see if our search results are bad or 
This had been discussed previously (though I haven't found an existing bug on that). Came back to mind while reading http://andrewspittle.net/2013/04/09/search-first-docs/

Comment 2

[James Socol [:jsocol, :james]]

What is meant by "stats" here? top queries? (should be able to get that from GA now.) click-through rate? failed searches? This needs a lot more specification.

Comment 3

[David Bruant]

(In reply to James Socol [:jsocol, :james] from comment #2)
> This needs a lot more specification.
I admit, my description is very draftly. I currently don't have a full list of what I'd need as I feel it will change while discovering the data. I'm also unclear on what kind of data can be easily pulled out and what kind will need small or big work.
This bug is here to better understand the need, not necessarily implement something yet, especially if some data is already available.

The main question I would love to get a grasp on is: are users satisfied with the search feature?
[changing bug summary to reflect that]

To begin with, we could say:
* number of searches per month
* top queries indeed.
* All queries can be interesting too to get stats on, for instance, how often people use quotes in the search input (to know if we need to implement that too, etc.)
* Number of people who come from Google Search. The evolution of this number is interesting. If it increases, it may indicate that our search results aren't as good as Google's (currently it's impossible, but that'll be interesting)

(In reply to James Socol [:jsocol, :james] from comment #2)
> click-through rate?
I don't know what that means exactly. If that means whether people click on the presented search result, then yes.

> failed searches?
If we define "failed search" by "search results were presented, but none was clicked and the user bounced", that's interesting too.

> (should be able to get that from GA now.)
What is all the data currently available from GA?
Are there statistics specific to the use of Google Custom Search Engine?
Will the GA data be available and as accurate after moving to Elastic Search?

Comment 4

[Jean-Yves Perrier [:teoli]]

I have some info.

Last month, we got 107,273 internal searches done on the MDN; search results pages represented 1.67% of the total page views. Note that I've restricted the search to /en-US/, but that's not a big deal.

The top-100 searches of last months are (with the # of occurrence, showing they are very very diverse and no single search shows up, as the #1 represents less than 1% of all searches):
array,1,040
date,621
string,577
input,481
apply,441
window,425
replace,423
slice,411
splice,382
split,331
map,330
addEventListener,322
call,320
event,316
foreach,315
canvas,299
match,289
prototype,284
background,281
Date,274
document,270
setTimeout,268
regexp,261
settimeout,254
regex,253
transition,252
select,251
object,249
typeof,248
indexof,245
substring,245
overflow,240
element,238
border,233
Array,232
box-shadow,232
substr,224
onload,222
for,219
form,216
switch,216
bind,215
location,208
position,207
button,206
transform,206
json,202
forEach,194
indexOf,192
iframe,190
display,188
xmlhttprequest,183
addeventlistener,182
setInterval,182
sort,182
table,179
push,175
onclick,174
parseInt,172
XMLHttpRequest,172
,166 (yes, an empty search)
events,166
image,166
scroll,164
filter,161
String,159
function,153
float,149
trim,147
getElementById,146
setinterval,145
arguments,144
style,144
window.open,142
textarea,141
img,140
join,140
appendChild,138
concat,137
font,137
reduce,136
console,133
cursor,133
parseint,131
node,130
alert,127
getelementbyid,127
instanceof,127
localstorage,124
margin,124
border-radius,123
css,123
innerHTML,121
createElement,120
delete,120
background-size,117
checkbox,114
JSON,112
width,112
cookie,111
 
And 58.96% of our traffic comes from search engines (mainly Google).

Comment 5

[John Karahalis [:openjck]]

The user experience team is currently looking at search. Your best bet might be to start an email thread that includes :habber and dev-mdn@.

I am going to mark this as invalid. I do not mean to close the door on the discussion -- I think it's an important one -- but we would like to keep Bugzilla for actionable development work only.