Closed Bug 1045106 Opened 5 years ago Closed 5 years ago

Decide if tracking is possible on the Interactive Snippet

Categories

(Snippets Graveyard :: General, defect)

x86
macOS
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: adam, Assigned: jcollings)

References

Details

Re Bug 1027377 - Ship interactive snippet for Webmaker Campaign

Is is possible to implement any metrics tracking for the on-page interactions?

* Number of clicks on 'Click here to change the background'
* Number of times the CTA is displayed (displayed after user types a color) 

As this is such a new process for all of us, this could be really informative data.

I don't know if there's an existing framework in place that could be used to collect this data.

If not, and the time-frame is too short to implement something new, then let's close this bug.

But if some data collection is possible, I'm happy to help in any way I can.
Let me check this out and get back to you.
Flags: needinfo?(chrismore.bugzilla)
I don't think there are any privacy concerns with collecting these metrics, but copying Mika just in case.
(In reply to Adam Lofting (:adamlofting) from comment #0)
> Re Bug 1027377 - Ship interactive snippet for Webmaker Campaign
> 
> Is is possible to implement any metrics tracking for the on-page
> interactions?
> 
> * Number of clicks on 'Click here to change the background'
> * Number of times the CTA is displayed (displayed after user types a color) 
> 
> As this is such a new process for all of us, this could be really
> informative data.
> 
> I don't know if there's an existing framework in place that could be used to
> collect this data.
> 
> If not, and the time-frame is too short to implement something new, then
> let's close this bug.
> 
> But if some data collection is possible, I'm happy to help in any way I can.

We currently don't have a mechanism to do this type of metrics collection. We have snippet impression data that is collected in-house and we have CTA click data from the target website's Google Analytics UTM tags. We had to go through governance before for the generic snippet impressions and thus I would imagine in-page analytics would require governance. Since about:home is in-product and not a web page, it may require a change to the Firefox privacy policy. It doesn't mean that we cannot do this type of metrics in a privacy-aware method, but since about:home is in-product, we would have to jump through different hoops. Though different hoops take a while and we have to show that there is user benefit by having this data.
Flags: needinfo?(chrismore.bugzilla)
Thanks Chris. That makes sense.

@Andrea, @Jean: I suggest we start the 'hoop-jumping' as a long term strategy but don't make this a blocker for going live with the interactive snippet for Maker Party. Do you agree?
Flags: needinfo?(jcollings)
Flags: needinfo?(andrea)
I do think this data would be hugely beneficial to users. What it allows us to do is create more personalized, interactive experiences on about:home (not just the for foundation, but Firefox and possibly sponsors as well). I don't think it should be a blocker today for this Webmaker snippet per-se, but I think it will be very difficult to determine if this is a delightful or annoying experience for users unless we understand how they're interacting with this more engaging page design.
Flags: needinfo?(andrea)
+1 to the reason as it is the very same reasons why we have GA on our websites. It is just because about:home is in-product and we would have to bake Google Analytics code into the Firefox product, which would set off alarm bells in some people. We tried to add Google Analytics to the firefox accounts sign-in page and that was decided against in bug 971963. We also tried to add Google Analytics to the Firefox Android product so that we have conversion rates on campaign->install, but that was also decided against (with passion) in 2013.
(In reply to Chris More [:cmore] from comment #6)
> +1 to the reason as it is the very same reasons why we have GA on our
> websites. It is just because about:home is in-product and we would have to
> bake Google Analytics code into the Firefox product, which would set off
> alarm bells in some people. We tried to add Google Analytics to the firefox
> accounts sign-in page and that was decided against in bug 971963. We also
> tried to add Google Analytics to the Firefox Android product so that we have
> conversion rates on campaign->install, but that was also decided against
> (with passion) in 2013.

Thanks Chris this is valuable insight / context / history to have as we're focused on what our data can/can't tell us. :)
Agreed. I think the story for why this is valuable to users is simple enough to explain. 

Finding a technical solution that answers our metrics questions and does so in an appropriate way is going to be the bulk of the work, but isn't insurmountable (this doesn't have to be GA for example).

I'm happy to take this ticket and move it along, but wanted to separate it from the timings for the maker party promotion, as I suspect this will (and should) take longer to work through.

I think that's agreed now, so I'll remove the blocker and rename this ticket.
Actually, I'll mark this ticket resolved, as we've decided tracking is possible, but not in the timeframe for launch. I'll file a new ticket for finding a metrics solution for the snippet.
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(jcollings)
Resolution: --- → FIXED
Blocks: 1046321
I believe your use case is already covered under our Websites Privacy Policy (the second section title on "We use cookies, clear GIFs, third party web analytics, and IP addresses for functionality and to improve our products, services, and communications and the "learn more" expander.  It's here: https://www.mozilla.org/en-US/privacy/websites/

Can you review the text and LMK if you think your use case is covered, or not?

Thanks,
Mika
No longer blocks: 1046321
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Mika from comment #10)
> I believe your use case is already covered under our Websites Privacy Policy
> (the second section title on "We use cookies, clear GIFs, third party web
> analytics, and IP addresses for functionality and to improve our products,
> services, and communications and the "learn more" expander.  It's here:
> https://www.mozilla.org/en-US/privacy/websites/
> 
> Can you review the text and LMK if you think your use case is covered, or
> not?
> 
> Thanks,
> Mika

Mike: Snippets are not displayed on a website as they are in-product. The websites privacy policy does not cover the Firefox product and specifically the about:home page. I've cc'd you on the previous legal/privacy review of about:home analytics in bug 723589.

Adam: Agree that it doesn't have to be GA. We have impression data on snippets via an internal service and can be reviewed in bug 719090 and https://wiki.mozilla.org/Websites/Snippets/Impressions.
Mika: I meant Mika, not Mike, above. opps. :)
Flags: needinfo?(udevi)
Closing this bug - there's no privacy concern.   The snippet data collection is covered in the Firefox Browser Privacy Notice.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Flags: needinfo?(udevi)
Resolution: --- → FIXED
Adam: Can you file a snippets:service bug as the JS developed in bug 690881 that collect snippet impressions will have to be extended to be able to send request on in-page interactions instead of just purely impressions. The current infrastructure and schema for snippets impressions may not be suitable for the use case described in comment 0. We'll need to work with Jean to see where this falls into the Snippets roadmap. This isn't a small amount of work given that the infrastructure would need to be changed to handle these type of metrics.
See Also: → 1067324
Product: Snippets → Snippets Graveyard
You need to log in before you can comment on or make changes to this bug.