Snippet impressions tracking for in-page interactions

RESOLVED FIXED

Status

defect
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: adam, Assigned: giorgos)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

We recently ran an interactive snippet experience called 'Hack the Snippet' (See Bug 1027377). As part of this process we would have liked to know more about how the in-page interactions were used and how popular this activity was (See Bug 1045106). As this was an educational experience for the user, we would also like to know at which point in the process users get stuck or lose interest so that we can improve future versions of the interactive experience.

It was not feasible to implement additional metrics in the time-frame of the campaign, but we decided to follow this up as a potential development for similar future snippets.

# The request in this bug is to develop/enable snippet impressions/hit tracking for in-page interactions in interactive snippet experiences. #

This would allow us to understand how 'interactive snippets' are being used so that we can make better decisions. For example, deciding how long to promote a particular snippet for; activities with a low engagement rate can be taken down quickly, and those with a high engagement rate can be run for longer. Over time we can better learn what content is and isn't suitable for use on the snippet so that we can present a higher quality experience that users engage with. 

This would of course need to comply with the existing privacy requirements of snippet data collection. 

(Quoting Chris More [:cmore] from Bug 1045106)
> Adam: Can you file a snippets:service bug as the JS developed in bug 690881
> that collect snippet impressions will have to be extended to be able to send
> request on in-page interactions instead of just purely impressions. The
> current infrastructure and schema for snippets impressions may not be
> suitable for the use case described in comment 0. We'll need to work with
> Jean to see where this falls into the Snippets roadmap. This isn't a small
> amount of work given that the infrastructure would need to be changed to
> handle these type of metrics.

There is not an immediate use case for this data with a deadline, but the Foundation are planning to build an interactive experience of some kind for the end of year fundraising appeal. This ticket would not be a blocker for that, but would allow us to learn more about what works/doesn't in that design.

Further reading about 'Hack the Snippet': 
* http://jessicaklein.blogspot.co.uk/2014/08/remix-hack-firefox-home-page-no-really.html
* http://adamlofting.com/1203/something-special-within-hack-the-snippet/
To get basic impression data on about:home snippets required legal, privacy, and governance to get involved and was tracked here: https://wiki.mozilla.org/Websites/Snippets/Impressions

The scope and abilities of snippets have increased quite a bit over the past few years and basic impression is no longer suffice to understand interactive snippets. We should kick-off a new discussion with privacy/legal and they can decide if we need to go to governance. 

Should we official kick off this with filling the main project review bug?

https://bugzilla.mozilla.org/form.moz-project-review
Project review bug 1084677 has been filed.
Duplicate of this bug: 1046321
In terms of technical implementation, I think the easiest path that seems reasonable would be to add an extra field to hits going to snippets-stats.mozilla.org, called `type`, that specifies the type of metric we're measuring. then, we can add a function to the default snippet JavaScript called something like `sendMetric(metricName)` that can be called by custom snippets to log their metrics.

This would require some input from the metrics team on how to add a new field for querying on in Tableau.

giorgos: What's your input on this idea?
Flags: needinfo?(giorgos)
An additional field called type seems to make sense. The default type would be view or impression and anything else we can override with a specific use-case.
+1 makes sense and it's fairly easy to implement from our side.
Flags: needinfo?(giorgos)
Assignee: nobody → giorgos
Status: NEW → ASSIGNED
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Re-opening this to wait for Tableau changes (bug 1087524)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
New snippets worksheet is here: https://dataviz.mozilla.org/views/SnippetMetrics/

Updated metrics function pushed to prod today as well.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.