877679 - UX, Search: Implement related docs navigator within article level pages

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: General, defect)

Description

[Holly Habstritt Gaal [:Habber]]

What problems would this solve?
===============================
Within the docs, themselves, this UI allows the user to skip through search results and related docs and horizontally navigate through content. The user then does not have to return to the separate search results page (or leave and go back to google right away before trying more MDN results.

First implement within article level pages.

Who would use this?
===================
User searching for docs within MDN, users who land at an article level page from another search tool like google, users entering MDN from Mozilla.org who would like an easy way to access related docs.

What would users see?
=====================
Users would see a bar below the main navigation and above the docs page. This bar contains a dropdown menu or < > toggle that allows the user to skip through to other related content. 

What would users do? What would happen as a result?
===================================================
Users would have easy access to related docs, but also be able to scan through search results in a quick way without going back to the MDN search results page (or google search results page - keeping them within MDN content).

Is there anything else we should know?
======================================

Comment 1

[Holly Habstritt Gaal [:Habber]]

See pages 24-26 of this doc: http://cl.ly/0C0o2V1G3O14

Comment 2

[Jannis Leidel [:jezdez]]

Attachment: search navigator.mov

Just a quick demo of the stuff I'm working on in https://github.com/mozilla/kuma/pull/1655

Comment 3

[Jannis Leidel [:jezdez]]

Hi all, I've worked on an implementation for this based on the recent changes of the search. See https://github.com/mozilla/kuma/pull/1655 for the code. It needs polish on the UI side, e.g the popup doesn't hide itself right now after switching the page. I fear I need some help from a more front-end affine person (:davidwalsh?).

The main concept behind this implementation is storing the title and URL of the documents shown on a search result page under a unique identifier ("ref") in the localStorage of the browser. The unique identifier is created by taking the search query, the locale of the user, the selected topic filters and the page number of the sarch result into account.

Once a user clicks on a search result a JavaScript embedded in the document page checks if there are stored search results matching the unique identifier of the search results. The main search result page doesn't attach the unique identifier to the URL of the search results to not circumvent the caching of our HTTP load balancer (which takes query parameters into account). On the first selected document page the embedded JavaScript inspects the referrer URL of the document page to calculate the unique identifier from. It then checks if there is a stored local search and populates both the next/previous links as well as the table of contents in the popover. 

An advantage of using localStorage for this problem is that it works for both logged-in and anonymous users as well as working around the HTTP cache of our frontend load balancer (Zeus) which varies its cache based on URL and Cookie.

The localStorage is currently set to expire each stored search result set after a day to prevent filling up the local browser storage of our users. If a user copy/pastes a URL of a search around and another user opens that link without having previously populated the local search store the JavaScript currently removes the unique identifier from the URL again. In other words, these stored search are purely per-user/per-browser/per-day right now.

On the long run this should probably auto-populate the local storage from the server if a user is passed a URL with a unique identifier included. That would allow sharing specific search result sessions between users and could be an incredible useful tools for editors as well as users.

What I'd like to know now is how to proceed?

Comment 4

[Jannis Leidel [:jezdez]]

Ah, also, please take a look at the attached screencast of the current state to get a better picture, apologies for the many technical details above :)

Comment 5

[ali spivak]

Adding Jean-Yves and Sheppy for their input

Comment 6

[Jean-Yves Perrier [:teoli]]

I like very much the idea. 

Two questions:
1. Is the two arrows understood by the users? The first time I saw them (on a wireframe) I had to ask habber about them. So I'm a bit concerned about this.
2. Did this appear only after a search or all the time?

I think we should add a text like "Related search results" at the top of the pop-up window. To make it clearer what this is.

Comment 7

[Holly Habstritt Gaal [:Habber]]

For now it would only appear after search since that is the only path accounted for at this time. The additional paths in that we need to consider (but are harder to implement) are when the user enters via direct link or from non-MDN search. 

I also agree that a label at the top of the list of links would help describe what this is. 

Another addition to consider is being able to display list and allowing the user to select from the list via key command.



(In reply to Jean-Yves Perrier [:teoli] from comment #6)
> I like very much the idea. 
> 
> Two questions:
> 1. Is the two arrows understood by the users? The first time I saw them (on
> a wireframe) I had to ask habber about them. So I'm a bit concerned about
> this.
> 2. Did this appear only after a search or all the time?
> 
> I think we should add a text like "Related search results" at the top of the
> pop-up window. To make it clearer what this is.

Comment 8

[Eric Shepherd [:sheppy]]

(In reply to Jean-Yves Perrier [:teoli] from comment #6)
> I like very much the idea. 
> 
> Two questions:
> 1. Is the two arrows understood by the users? The first time I saw them (on
> a wireframe) I had to ask habber about them. So I'm a bit concerned about
> this.
> 2. Did this appear only after a search or all the time?
> 
> I think we should add a text like "Related search results" at the top of the
> pop-up window. To make it clearer what this is.

I agree that the arrows are not super clear. We need to be sure the UX is as obvious as possible. I agree with Jean-Yves that a popup menu or something might be better, although I kind of personally love the spinner/wheel-of-fortune vibe of the arrows. :)

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/2339350ad0ad6cd506566ccae9d52fb8e5cb27e3
fix bug 877679 - added a document navigator stored in localStorage for MDN search traffic

https://github.com/mozilla/kuma/commit/5b6455e0b88557d7ae3889f3b81df66c0e6d6f06
Merge pull request #1655 from jezdez/search-navigator-877679

fix bug 877679 - added a document navigator stored in localStorage for MDN search traffic

Comment 10

[Luke Crouch [:groovecoder]]

At long last, this is merged and pushed. It's available to everyone on the stage server:

https://developer.allizom.org/en-US/search?q=geolocation

But it's still behind a waffle flag on the production site. I'm comfortable with the tracking elements we have in place to open the feature to everyone - i.e., we don't need to waffle 50/50% of visitors for any kind of A/B test. We can easily segment the docs nav viewers/users from others by the custom var or even just /docs/*?search= in the URL.

Hmm .. we may get a bug filed about that URL being too messy :/

Comment 11

[Luke Crouch [:groovecoder]]

:habber and :jezdez - you both good with opening the feature to everyone?

Comment 12

[Jannis Leidel [:jezdez]]

FTR, the ?search= parameter is removed from the URL in the location bar as soon someone goes to a page and doesn't have the given search id stored in his local browser storage. I know this isn't perfect but it at least prevents cargo culting URLs with wrong IDs around.

:groovecoder: I'm okay with launching it to beta testers, if :habber is fine with it.

Comment 13

[Luke Crouch [:groovecoder]]

I enabled the docs navigator for site super-users and beta testers. Hopefully we'll start to see the GA events come thru soon so we can measure the usage.

Comment 14

[Luke Crouch [:groovecoder]]

The GA events are coming thru, and I created a custom dashboard of reports showing the Docs Navigator usage:

https://www.google.com/analytics/web/?hl=en#dashboard/NbSVcZbuSTWaYveIrhTI8A/a36116321w64965862p66726481/%3F_u.date00%3D20140212%26_u.date01%3D20140214/

With this we can learn:

* How many visitors are seeing the docs nav
* If/How visitors use it (hover, click links, next, previous)
* Usage by page

My idea behind the latter report is to tell us which pages may need to be pushed lower in the search rank algorithm. assumption is if visitors are using the docs navigator on those pages then they didn't find what they were looking for on that page. :( But it will take some tuning and refining - probably by original search term.

Comment 15

[Luke Crouch [:groovecoder]]

I released this feature to 50% of visitors and created GA segments for each. We can let it run for a week or so and re-visit to learn how visitors are using it and if we need to modify it before full release.

While we wait, :habber - what are the measurable goals we want to achieve here? Fewer page-views on results pages? Fewer repeat searches?

Comment 16

[Holly Habstritt Gaal [:Habber]]

Hi Luke. Could you share some GA results from this test? 

I'm interested in the following:
- Was it discoverable? (we may need to alter some things in the visual design, point it out to users, etc)
- Was there a rise in navigating between docs on MDN? 
- Were users navigating back to search results? (that would help us determine if this was discoverable)

Did you notice anything specific that we could improve? I can meet this week to discuss next steps. 


(In reply to Luke Crouch [:groovecoder] from comment #15)
> I released this feature to 50% of visitors and created GA segments for each.
> We can let it run for a week or so and re-visit to learn how visitors are
> using it and if we need to modify it before full release.
> 
> While we wait, :habber - what are the measurable goals we want to achieve
> here? Fewer page-views on results pages? Fewer repeat searches?

Comment 17

[Luke Crouch [:groovecoder]]

Sorry, I had disabled the feature until bug 973961 was fixed. Looks like it's fixed and pushed live so I re-enabled the feature for 50% of visitors. I'll leave a needinfo? for me on this to remind me to get back into it when we've collected some more data.

Comment 18

[Luke Crouch [:groovecoder]]

First week of data is attached.

11.81% of all visits made some interaction with the docs navigator! (73% of those events are "Close on blur" though, so there might be an issue with the way we're tracking. :/)

But, 0.2% clicked next or previous, and .18% clicked a result in the dialog. :(

The next iteration should focus on:

* Verify the event tracking is correct
* Make the feature more discoverable

But, IMO the docs navigator is a low-priority feature right now.

Comment 19

[Luke Crouch [:groovecoder]]

Attachment: Analytics developer.mozilla.org (production) Top Events 20140321-20140327.pdf

Comment 20

[Holly Habstritt Gaal [:Habber]]

11.81% interacting with the docs navigator is great! I expected greater issues with discoverability. Even though they are discovering it, I'd like to see if users understand that the items in the dropdown menu are related and/or from their search results.

As for discovering the "next" and "previous" buttons, I'll give it some thought. 



(In reply to Luke Crouch [:groovecoder] from comment #18)
> First week of data is attached.
> 
> 11.81% of all visits made some interaction with the docs navigator! (73% of
> those events are "Close on blur" though, so there might be an issue with the
> way we're tracking. :/)
> 
> But, 0.2% clicked next or previous, and .18% clicked a result in the dialog.
> :(
> 
> The next iteration should focus on:
> 
> * Verify the event tracking is correct
> * Make the feature more discoverable
> 
> But, IMO the docs navigator is a low-priority feature right now.

Comment 21

[Luke Crouch [:groovecoder]]

Attachment: Analytics developer.mozilla.org (production) Top Events 20140311-20140410.pdf

For some reason there was a big drop off in events after the last report, so here's the latest.

Comment 22

[Holly Habstritt Gaal [:Habber]]

I think that the docs navigator could use a bit better guidance for users, but it can launch to all users before these are complete. However, could #1 be added quickly? 

1. a tooltip when the user hovers over an arrow to say "next/previous search result" (this may raise interaction rate with this element)

2. in the list itself, a title to explain what this list is. "Your search results" (too explicit? open for copy ideas)

3. Sean's visual design skills to better connect the back/forward arrows, picker icon, and list view with a similar style. 


The above items would make this feature a lot more polished, but are not necessary for making it live.

Comment 23

[Luke Crouch [:groovecoder]]

:davidwalsh - how quick is #1 there?

Comment 24

[David Walsh :davidwalsh]

I can make this priority for tomorrow!

Comment 25

[Luke Crouch [:groovecoder]]

Can we wrap up #1 and #2 quickly so we can call the first docs nav iteration done?

Comment 26

[David Walsh :davidwalsh]

https://github.com/mozilla/kuma/pull/2348

Comment 27

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/217716064926a5a411877ece3ddd8b5dfd9a8d26
fix bug 877679 - Final touches to launch 0.1 of search navigator

https://github.com/mozilla/kuma/commit/7bbe211385416cb11a1601c2462c82f0ac16d142
Merge pull request #2348 from darkwing/877679-killer

fix bug 877679 - Final touches to launch 0.1 of search navigator

Comment 28

[Holly Habstritt Gaal [:Habber]]

Hi Sean, would you have time to take a look at the visual design for the search navigator? Currently, it is only implemented for users coming from search results. 

Check out the following and see if it needs any style updates: 
- next and previous buttons
- button/icon to open list
- list style

screenshots:
list closed - http://cl.ly/image/0Z1J0G2I0M0U
list open - http://cl.ly/image/262m2h1h3D2r

Comment 29

[Sean Martell]

It's close! 

I'd prefer it to be as close to http://mozilla.seanmartell.com/mdn-preview/index.php?directory=.&currentPic=7 as possible.

I know it's not going to be attached to the window edge as in that shot, but the vertical aligning of the arrows would be nice.

Also, not a fan of that default heavy dropshadow. We need to standardize the dropshadow for overlay modules to be as close to what you see in my PSD example above as possible.

I'd also lose the outline on the grey overlay box too. Not needed if we're using that grey + the shallow dropshadow from my design, which acts as a border. 

In an ideal world, we'd do something as close as possible to my mockup with regards to all overlays. <3

Thanks!