Closed Bug 1352821 Opened 7 years ago Closed 7 years ago

Add additional analytics tool to www.mozilla.org pages for a one-off experiment

Categories

(www.mozilla.org :: Analytics, enhancement)

Product:
www.mozilla.org
Component:
Analytics
Version:
Production
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: cmore, Unassigned)

References

Details

Attachments

(1 file)

GitHub pull request
44 bytes, text/x-github-pull-request
Details | Review 
We are going to be performing a short 2-week experiment to help validate some core assumptions tied to our 2017 organization objectives and key results. With that, we are going to be adding an additional analytics tool to specific www.mozilla.org pages.

The analytics JS snippet should be wrapped in our normal DNT condition that we have throughout the website.

Test recipe and experiment details coming soon.
For the sake of keeping this updated, I'll add my progress as I go.

Two things:
1. I am getting a Content-Security-Policy console error "The page’s settings blocked the loading of a resource at ..." Not sure what this is about quite yet.
2. The <noscript> html elements of this code block need to be added after the JS. There seems to be a global template block, "task_completion_survey"[1] which is placed after the global "js" [2] block. Rather than creating a new block in the base templates, should I use the currently existing block?

[1] https://github.com/mozilla/bedrock/blob/433619292bf1449c7d9b7c000429951cbb05c247/bedrock/mozorg/templates/mozorg/plugincheck.html#L235

[2] https://github.com/mozilla/bedrock/blob/433619292bf1449c7d9b7c000429951cbb05c247/bedrock/mozorg/templates/mozorg/plugincheck.html#L231

Exploring my options. Will follow up.
(In reply to Francesco Polizzi [:fpolizzi] from comment #2)
> 2. The <noscript> html elements of this code block need to be added after
> the JS. There seems to be a global template block,
> "task_completion_survey"[1] which is placed after the global "js" [2] block.
> Rather than creating a new block in the base templates, should I use the
> currently existing block?

Added a new block. Didn't realize there was a *global* global base template group. 

> 1. I am getting a Content-Security-Policy console error "The page’s settings blocked the loading of > a resource at ..." Not sure what this is about quite yet.

Added our the analytics tool to it. No longer an issue.
Ready to roll when you are :cmore
Flags: needinfo?(chrismore.bugzilla)
:fpolizzi: just confirming that the JS is only added to the items described in the experiment recipe and not the entire www.mozilla.org website.

If it is just the pages in the recipe, you are unblocked to submit the pull request.
Flags: needinfo?(chrismore.bugzilla)
Got confirmation from Francesco it is only on the pages described in the recipe.
PR submitted(In reply to Chris More [:cmore] from comment #5)
> If it is just the pages in the recipe, you are unblocked to submit the pull
> request.

Confirmed. PR submitted!

:jpetto, :agibson any chance either of you have bandwidth to review?
Flags: needinfo?(jon)
Flags: needinfo?(agibson)
Before we get into code review, can I just check this has had a privacy review? I'm assuming there's a good reason why the tools name isn't mentioned anywhere in the bug itself.
Flags: needinfo?(agibson) → needinfo?(francescosapolizzi)
(In reply to Chris More [:cmore] from comment #0)
> The analytics JS snippet should be wrapped in our normal DNT condition that
> we have throughout the website.

I'd also like to point out that adding this snippet to /firstrun pages gives the user little opportunity to opt-our via DNT?
*opt-out via DNT
(In reply to Alex Gibson [:agibson] from comment #8)
> Before we get into code review, can I just check this has had a privacy
> review? I'm assuming there's a good reason why the tools name isn't
> mentioned anywhere in the bug itself.

(In reply to Alex Gibson [:agibson] from comment #8)
> Before we get into code review, can I just check this has had a privacy
> review? I'm assuming there's a good reason why the tools name isn't
> mentioned anywhere in the bug itself.

The depends on field up top has a bug (bug 1353178) for the legal review and approval that happened last week. I will add you to it, but Elvin/Jishnu signed off on the two week experiment and the privacy policy is being updated as noted in the bug.
(In reply to Chris More [:cmore] from comment #11)
> (In reply to Alex Gibson [:agibson] from comment #8)
> > Before we get into code review, can I just check this has had a privacy
> > review? I'm assuming there's a good reason why the tools name isn't
> > mentioned anywhere in the bug itself.
> 
> (In reply to Alex Gibson [:agibson] from comment #8)
> > Before we get into code review, can I just check this has had a privacy
> > review? I'm assuming there's a good reason why the tools name isn't
> > mentioned anywhere in the bug itself.
> 
> The depends on field up top has a bug (bug 1353178) for the legal review and
> approval that happened last week. I will add you to it, but Elvin/Jishnu
> signed off on the two week experiment and the privacy policy is being
> updated as noted in the bug.

Great, thanks :cmore.
(In reply to Alex Gibson [:agibson] from comment #9)
> (In reply to Chris More [:cmore] from comment #0)
> > The analytics JS snippet should be wrapped in our normal DNT condition that
> > we have throughout the website.
> 
> I'd also like to point out that adding this snippet to /firstrun pages gives
> the user little opportunity to opt-our via DNT?

Yes, we are aware and legal is fine with these pages and asked we wrap it in our DNT condition for consistency perspectives, but understanding that /firstrun/ condition is almost pointless given the flag wouldn't have been set yet. This would be the same case for GA. Thanks :agibson! :)
Just a quick note on timing. We need to get this experiment live sometime this week given the 2 week timeframe and giving enough post-experiment time to have the product team decide next steps before Q2 is too far gone. Thanks!
Flags: needinfo?(francescosapolizzi)
Flags: needinfo?(jon)
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/737771da5a23840df94064eb95fea6f6fb594975
[Fix bug 1352821] Add additional analytics tool

https://github.com/mozilla/bedrock/commit/94336030d3acc0206be09f19a1e4885152676cee
Merge pull request #4755 from FrancescoSTL/bug-1352821-Add-aditional-analytics-tools

[Fix bug 1352821] Add additional analytics tool
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/bae5d8263ea86e82e0cd1e1a40bc328149561e5b
[bug 1352821] Fix StatCounter CSP script error

https://github.com/mozilla/bedrock/commit/ffc5a263707ba5b76221072f27644b8ccad50f4e
Merge pull request #4772 from alexgibson/bug-1352821-fix-csp-error-statcounter

[bug 1352821] Fix StatCounter CSP script error
Given this was a "one off experiment" and we saw the results of this test published in an email yesterday, I'm going to assume we're now OK to remove this code. Chris can you confirm please?
Status: RESOLVED → REOPENED
Flags: needinfo?(chrismore.bugzilla)
Resolution: FIXED → ---
(In reply to Alex Gibson [:agibson] from comment #17)
> Given this was a "one off experiment" and we saw the results of this test
> published in an email yesterday, I'm going to assume we're now OK to remove
> this code. Chris can you confirm please?

Please don't remove the code. There is an anomaly in the data and StatCounter is looking into it now with their developers and we don't want to stop it until they figure out what is going on. Once we have an answer, we will ping this bug. Thanks
Flags: needinfo?(chrismore.bugzilla)
:agibson: for the pages that will launch next week with the Firefox home website, the only page that is part of this experiment that will be impacted by the launch is /firefox/features/. Is StatCounter going to be still on /firefox/features/ for the launch of the new website? Thanks!
Flags: needinfo?(agibson)
(In reply to Chris More [:cmore] from comment #19)
> :agibson: for the pages that will launch next week with the Firefox home
> website, the only page that is part of this experiment that will be impacted
> by the launch is /firefox/features/. Is StatCounter going to be still on
> /firefox/features/ for the launch of the new website? Thanks!

Yep still there in the new template, thanks.
Flags: needinfo?(agibson)
(In reply to Alex Gibson [:agibson] from comment #20)
> (In reply to Chris More [:cmore] from comment #19)
> > :agibson: for the pages that will launch next week with the Firefox home
> > website, the only page that is part of this experiment that will be impacted
> > by the launch is /firefox/features/. Is StatCounter going to be still on
> > /firefox/features/ for the launch of the new website? Thanks!
> 
> Yep still there in the new template, thanks.

Cool. Thanks!
Blocks: 1368402
Update: please remove StatCounter from the pages defined in the pages defined on the mana page from comment 1. Thanks!
Flags: needinfo?(agibson)
(In reply to Chris More [:cmore] from comment #22)
> Update: please remove StatCounter from the pages defined in the pages
> defined on the mana page from comment 1. Thanks!

Will do, thanks cmore
Flags: needinfo?(agibson)
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/05ce6a0f2c2b47d9b25ab45a6e4e9a859235330a
[fix bug 1352821] Remove statcounter from www.mozilla.org pages

https://github.com/mozilla/bedrock/commit/fc1d68dda738536646c2646e7dc74a1ace499626
Merge pull request #4869 from alexgibson/bug-1352821-remove-statcounter

[fix bug 1352821] Remove analytics tool from www.mozilla.org pages
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: