677804 - Start tracking "reads" on KB articles

Status: VERIFIED FIXED

(support.mozilla.org :: Knowledge Base Software, task)

Description

[[:Cww]]

The key metric (according to Daniel) is how many people actually read a KB article rather than click a link and then go back or close tab. For now, we just want to send information about how long users have been on a page for KB articles.

So the behavior we want is:

For a KB article: 

When you leave the page, fire off a request to the server for a gif with two parameters: duration=x&pageid=y

Where x is a count in seconds from when the page was loaded and pageid is the document/revision id of the page.

Only do this if x > 10 (we can change the number) to have it not fire for the immediate-back-button case and reduce the load on metrics.

If possible pause the timer if the page is loaded in a background tab or loses focus.

We can retrieve the actual data from server logs in a future bug.

Comment 1

["Saptarshi Guha[:joy]"]

If the reader switches to another tab and switches back

1. coalesce the page view into one
2. However if the time to switch back exceeds some number, keep these as separate page views. I could have read it twice. That measures intensity per user. 

BTW, what is a KB? Knowledge Base articles on MDN?

Comment 2

[James Socol [:jsocol, :james]]

(In reply to Saptarshi Guha from comment #1)
> If the reader switches to another tab and switches back

Unless something has changed dramatically, JS within the page has no way of knowing that this happened.

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

I think for this first version, we shouldn't worry about more complex monitoring of page switching or activity.

The point I made to Cheng which was mentioned in comment #0 is that we should investigate whether we can pause the duration timer when the JS blur event fires and resume it when the onfocus event fires.  That way having the page opened in a background tab won't result in a very high duration unless the user is at least potentially viewing the page.

Comment 4

[James Socol [:jsocol, :james]]

window.onfocus/onblur works, with document.onfocusin/onfocusout for old IE (window.onfocus/onblur fire too often in IE 6/7).

Comment 5

[Daniel Einspanjer [:dre] [:deinspanjer]]

cool, that sounds very doable.

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

Do we need the duration? I am worried that making this request as the user leaves the page will make it unreliable (I assume browsers will cancel pending requests when the user leaves the page, closes the tab, shuts down computer, etc).

If the duration isn't important, we can just fire off a request once the timer hits 10 seconds. We could also fire off a second request after 90 (or X) seconds. 

We should also add a query param with a random value so that browsers (and zeus?) don't cache the image.

Comment 7

[[:Cww]]

I'd like to have duration for the first go around so that we can see what the best "fire after" time is.  That said, we should test to make sure that it does fire reliably on close tab and close window and browse away otherwise we should go with your idea (but perhaps at 10, 30 and 90)

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

OK, so lets try to do the request on 'beforeunload'. How will we determine how well that is working? Can you compare the #'s to what is reported by the analytics system? Or should we start out with a request at 10 seconds and then on 'beforeunload'? That way we can see what % of time this works/fails.

Comment 9

[Daniel Einspanjer [:dre] [:deinspanjer]]

It might be interesting to set up a preliminary version of this to signal both on a timer and on 'beforeunload'.  Ideally, to be able to study the effectiveness of the 'beforeunload' method, we need a strong link to the timer based signals.  I'd recommend generating an ID just for the duration of the page and have that ID sent in for both the timer and the 'beforeunload' signals.

Once we have figured out how reliable the 'beforeunload' signal is, we can remove both the timer and the ID.


The alternative to this would be to go the traditional browser compatibility testing route.  Configure the signal submission URL to go somewhere that the tester can tail the logs and have them hit test pages with a variety of browsers and different conditions such as:
* clicking a link
* hitting back
* closing the tab
* closing the window
* closing the app
* force closing the app
* shutting down the OS

Comment 10

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Daniel Einspanjer :dre [:deinspanjer] from comment #9)
> The alternative to this would be to go the traditional browser compatibility
> testing route.
Went down this route and unfortunately (or fortunately) didn't get too far. I only had to test one browser, Firefox 6, to prove it wouldn't work. I tried both options below:

```
$(window).bind('beforeunload', function() {
    // option 1
    $.ajax({
        url: '/media/img/pixel.gif'
    });
    // option 2
    $('body').append('<img src="/media/img/pixel.gif"/>');
});
```

Neither of these ended up in requests being made to my dev server. Of course, if I add an `alert()` before exiting the function then the request happens. We could make the ajax call synchronous (sjax?), but that's not a cool thing to do to users.

Comment 11

[Daniel Einspanjer [:dre] [:deinspanjer]]

Adding the synchronous attribute would be one thing.. I think we could probably make it non-annoying just by putting a reasonable timeout on the request so there isn't a noticeable lag.

I'll look back through my previous corpus of work and see if I can come up with any other nifty ideas. :)

Comment 12

[Daniel Einspanjer [:dre] [:deinspanjer]]

Came across this stackoverflow question.  Doesn't bring much new to what you've already researched though..

http://stackoverflow.com/questions/3584288/can-the-unload-event-be-used-to-reliably-fire-ajax-request

Comment 13

[Daniel Einspanjer [:dre] [:deinspanjer]]

I think we should consider starting with a quick implementation that does the following:

1. pings at 10/30/90 seconds elapsed time
2. hooks the unload event and tries to ping with a 300ms timeout synchronous ajax call

3. Measure the results and see what the effectiveness of the unload event is.  If it is roughly as good as the timed count then we can shift to using it, otherwise, we remove that part and stick with the intervals.

Comment 14

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Daniel Einspanjer :dre [:deinspanjer] from comment #13)
> 1. pings at 10/30/90 seconds elapsed time
> 2. hooks the unload event and tries to ping with a 300ms timeout synchronous
> ajax call

I have this mostly working (still need to test some other browsers). A few questions:
* Do we still want a minimum duration to send back on unload or should we always record the duration for now?
* Anything else we want to track while we are at it? Maybe knowing if the user scrolled and how far is useful (in combination with knowing the window height)?

Comment 15

[Ricky Rosario [:rrosario, :r1cky]]

https://github.com/jsocol/kitsune/commit/02cf65de5e895b0437676998364f0126668bccdd

Notes:
* Everything is behind a waffle switch 'track-article-reads'
* Requests will look like: http://<domain>/media/img/pixel.gif?duration=10.01&documentid=703&trackid=1314054182564&referrer=search&query=cookies
** The trackid is for tracking pings from the same page view
** referrer will be one of: 'search', 'inproduct', actual document.referrer url, or empty
** query is the keywords used in the case the user came from sumo search or a search engine

Comment 16

[Tobias (:Tobbi) Markus]

Verified pixel.gif being loaded with correct parameters for search and document.referrer

Comment 17

[Daniel Einspanjer [:dre] [:deinspanjer]]

Great.  could someone please file a metrics bug a week after this goes live so we have some metrics to count out of the logs?