Last Comment Bug 973612 - Import page visit stats from Google Analytics and make it available to the API
: Import page visit stats from Google Analytics and make it available to the API
Status: NEW
:
Product: Mozilla Developer Network
Classification: Other
Component: API (show other bugs)
: unspecified
: x86_64 Linux
-- normal with 1 vote (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
:
Mentors:
Depends on:
Blocks: 839730 1133071 968736
  Show dependency treegraph
 
Reported: 2014-02-17 07:06 PST by Florian Scholz [:fscholz] (MDN)
Modified: 2015-04-11 12:14 PDT (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
dashboard available to the L10N contributors (46.89 KB, image/jpeg)
2014-10-28 07:31 PDT, Julien G
no flags Details
Dashboard available for English speaking contributors (49.55 KB, image/jpeg)
2014-10-28 07:31 PDT, Julien G
no flags Details

Description User image Florian Scholz [:fscholz] (MDN) 2014-02-17 07:06:41 PST
Especially for translation status pages like [1] it would be really helpful to have visit counts to prioritize the l10n work based on visitors.

For each document a "visits" field could be added and a (monthly) script import from Google Analytics could populate this field with data. We don't really need live data here. A "visits" field could then also be added to the $json of a document, so that we can use KumaScript to work with this data.

Les, Luke? Is there a way with Google Analytics to get us this data like described above?

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Doc_status/ja
Comment 1 User image Florian Scholz [:fscholz] (MDN) 2014-02-17 07:07:42 PST
Needinfo'ing Les and Luke.
Comment 2 User image Jean-Yves Perrier [:teoli] 2014-02-17 07:25:25 PST
Note that we don't need live values. Values updated monthly or even quarterly are perfectly fine.
Comment 3 User image Luke Crouch [:groovecoder] 2014-02-18 19:50:01 PST
Yes, there's a way and we should be able to re-use code like SUMO has for this:

https://github.com/mozilla/kitsune/blob/master/kitsune/sumo/googleanalytics.py
Comment 4 User image Ricky Rosario [:rrosario, :r1cky] 2014-03-08 09:39:55 PST
You will need to file an WebOps bug to get PyOpenSSL==0.13 installed on the servers. The fun part is that RHEL has 0.10 installed and WebOps will need to jump through some hoops for you. See all the gory details we went through in bug 830496
Comment 5 User image Julien G 2014-10-28 07:29:23 PDT
After (quickly) digging into kitsune's code,  found the basic bricks for GA/metrics there:

1. getting GA data in [1] (see #3) : This uses OAuth2client [2] (see #4 from :rrosario on this). Fecthing is done on different periods : 7 days / 30 days / 90 days / since the beginning. Only the en-US views are counted (this may be enough based on the hypothesis that en-US pages are the most visited pages and "give" the trends) (see also [3] and [4])
2. storing this data in the database in the table 'dashboards_wikidocumentvisits' ([5])
3. querying/creating a view for the content (the query uses info like tags, removes non-relevant pages ) ([6] and [7])

This allows to 
- give a ratio dashboard ala "top 100/50/20" gauge to the final user for l10n
- give a freshness dashboard to English speaking contributors 
(see picture attachments after)

needinfo'ing Luke for the following:
- is GA data available in a self-made kuma environment (i.e. one that I could make out of github and so on)?
- are we collecting GA data about things like "someone went to the [locale] version of the page but was redirected to the en-US version" or simply views per locale for a page?

[1] https://github.com/mozilla/kitsune/blob/master/kitsune/sumo/googleanalytics.py
[2] https://pypi.python.org/pypi/oauth2client
[3] https://github.com/mozilla/kitsune/blob/master/kitsune/dashboards/cron.py
[4] https://github.com/mozilla/kitsune/blob/master/kitsune/dashboards/models.py
[5] https://github.com/mozilla/kitsune/blob/master/scripts/schema.sql (goto dashboards_wikidocumentvisits definition)
[6] https://github.com/mozilla/kitsune/blob/master/kitsune/dashboards/readouts.py
[7] https://github.com/mozilla/kitsune/blob/master/kitsune/dashboards/views.py
Comment 6 User image Julien G 2014-10-28 07:31:14 PDT
Created attachment 8512699 [details]
dashboard available to the L10N contributors

We can see the "Top 50/20..." on this screenshot. I am not saying this is something we need in doc status pages.
Comment 7 User image Julien G 2014-10-28 07:31:51 PDT
Created attachment 8512700 [details]
Dashboard available for English speaking contributors
Comment 8 User image Julien G 2014-10-28 07:35:45 PDT
(In reply to Jean-Yves Perrier [:teoli] from comment #2)
> Note that we don't need live values. Values updated monthly or even
> quarterly are perfectly fine.

Do we want absolute values like 'nb of visits since forever' and/or also 'fresh visits' (like last 30 days) to react on "hot" pages? I don't know if this is relevant...
Comment 9 User image Les Orchard [:lorchard] 2014-10-28 07:40:58 PDT
> needinfo'ing Luke for the following:

lorchard is not luke :)
Comment 10 User image Julien G 2014-10-28 07:45:02 PDT
(In reply to Les Orchard [:lorchard] from comment #9)
> > needinfo'ing Luke for the following:
> 
> lorchard is not luke :)
mea culpa maxima :$ ^^
Comment 11 User image Jean-Yves Perrier [:teoli] 2014-10-29 00:20:11 PDT
(In reply to Julien G from comment #8)
> (In reply to Jean-Yves Perrier [:teoli] from comment #2)
> > Note that we don't need live values. Values updated monthly or even
> > quarterly are perfectly fine.
> 
> Do we want absolute values like 'nb of visits since forever' and/or also
> 'fresh visits' (like last 30 days) to react on "hot" pages? I don't know if
> this is relevant...

Visits over the last month (either 1st - 1st or sliding month, whatever is the most easy for the devs) is enough here (last = last at the point of the latest update, as we don't need live values)
Comment 12 User image Luke Crouch [:groovecoder] 2014-11-05 05:26:31 PST
+1 :teoli

Note You need to log in before you can comment on or make changes to this bug.