1275570 - Add Telemetry probe to see how often users enter print preview, and how many simplify the view

Status: RESOLVED FIXED

(Toolkit :: Printing, defect)

Description

[Benjamin Smedberg]

Sounds like a good thing to collect: should be a temporary (6-12 month) collection unless somebody is signed up for permanent dashboarding/monitoring, which seems unlikely in this case. I'll want to do a data-review on the final Histograms.json (or other collection mechanism) when that's ready.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Bug 962433 adds a mode to print preview that allows the user to "simplify" the printed document markup using the Reader Mode backend.

carnold asked whether or not we could gather data on how often it will be used in the wild, so that's what this bug is all about.

Comment 2

[Mike Conley (:mconley) (:⚙️)]

bsmedberg - any problems gathering this data?

Comment 3

[Mike Conley (:mconley) (:⚙️)]

mlongaray reports that thaua can take this bug and has a working patch.

Comment 4

[Thauã Silveira]

Attachment: WIP1: Add telemetry probe

This patch adds telemetry probe to keep track of how many users enter on print preview, and how many simplify the view.

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Comment on attachment 8760763 [details] [diff] [review]
WIP1: Add telemetry probe

Review of attachment 8760763 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/components/printing/content/printUtils.js
@@ +533,5 @@
>            URL: this._listener.getSourceBrowser().currentURI.spec,
>            windowID: this._listener.getSourceBrowser().outerWindowID,
>          });
> +
> +        // Here we log telemetry data for when the user clicks on simplify page

Not strictly true - we're logging when the user is entering simplification mode. We're not counting clicks on the node at all (which would also count exits from simplify mode).

::: toolkit/components/telemetry/Histograms.json
@@ +5870,5 @@
>      "n_buckets": 50,
>      "description": "(Bug 1207089) Time in ms between displaying a popup notification and dismissing it without an action the first time, keyed by ID"
>    },
> +  "PRINT_PREVIEW_OPENED_COUNT": {
> +    "alert_emails": ["[e-mail]"],

I guess we'd better make sure carnold knows we're hooking him up here. :)

@@ +5874,5 @@
> +    "alert_emails": ["[e-mail]"],
> +    "bug_numbers": [1275570],
> +    "expires_in_version": "56",
> +    "kind": "count",
> +    "releaseChannelCollection": "opt-out",

We're going to need to be sure that bsmedberg or some other data collection peer is okay with this. Personally, I think we can probably make some decent decisions based on our beta population, and opt-ins from Release.

@@ +5883,5 @@
> +    "bug_numbers": [1275570],
> +    "expires_in_version": "56",
> +    "kind": "count",
> +    "releaseChannelCollection": "opt-out",
> +    "description": "(Bug 1275570) Telemetry probe to see how often users click on simplify page checkbox when using print preview UI"

See above - this isn't about clicks, but about how often the user enters simplify mode.

Comment 6

[Mike Conley (:mconley) (:⚙️)]

Also, forgot to say, thanks for the patch! Looking good!

Comment 7

[Thauã Silveira]

(In reply to Mike Conley (:mconley) - (needinfo me!) from comment #5)
> Comment on attachment 8760763 [details] [diff] [review]
> WIP1: Add telemetry probe
> 
> Review of attachment 8760763 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: toolkit/components/printing/content/printUtils.js
> @@ +533,5 @@
> >            URL: this._listener.getSourceBrowser().currentURI.spec,
> >            windowID: this._listener.getSourceBrowser().outerWindowID,
> >          });
> > +
> > +        // Here we log telemetry data for when the user clicks on simplify page
> 
> Not strictly true - we're logging when the user is entering simplification
> mode. We're not counting clicks on the node at all (which would also count
> exits from simplify mode).
> 

Yes, you're right. I mistakenly used the word "click" in both cases. We should use "enter" instead.

> ::: toolkit/components/telemetry/Histograms.json
> @@ +5870,5 @@
> >      "n_buckets": 50,
> >      "description": "(Bug 1207089) Time in ms between displaying a popup notification and dismissing it without an action the first time, keyed by ID"
> >    },
> > +  "PRINT_PREVIEW_OPENED_COUNT": {
> > +    "alert_emails": ["[e-mail]"],
> 
> I guess we'd better make sure carnold knows we're hooking him up here. :)
> 
> @@ +5874,5 @@
> > +    "alert_emails": ["[e-mail]"],
> > +    "bug_numbers": [1275570],
> > +    "expires_in_version": "56",
> > +    "kind": "count",
> > +    "releaseChannelCollection": "opt-out",
> 
> We're going to need to be sure that bsmedberg or some other data collection
> peer is okay with this. Personally, I think we can probably make some decent
> decisions based on our beta population, and opt-ins from Release.
> 
> @@ +5883,5 @@
> > +    "bug_numbers": [1275570],
> > +    "expires_in_version": "56",
> > +    "kind": "count",
> > +    "releaseChannelCollection": "opt-out",
> > +    "description": "(Bug 1275570) Telemetry probe to see how often users click on simplify page checkbox when using print preview UI"
> 
> See above - this isn't about clicks, but about how often the user enters
> simplify mode.

I asked for more info from Christopher and Benjamin.

Comment 8

[Benjamin Smedberg]

Comment on attachment 8760763 [details] [diff] [review]
WIP1: Add telemetry probe

As a nit, I think the description can be improved: "(Bug N)" is unneeded now that we have the bug_numbers field. "Telemetry probe" is also unnecessary. But most importantly I encourage people to write the description in terms of what they are actually recording: "A counter incremented every time the browser enters print preview."

data-review=me

I highly recommend keeping this opt-out, as long as you are going to take the extra time to analyze this on the release population. Features like printing and print preview are likely very skewed between beta and release because beta is a more technical population.

Comment 9

[Thauã Silveira]

Attachment: WIP2: Nits fixed

This patch adds telemetry probe to keep track of how many users enter print preview and/or simplified mode.

Comment 10

[Mike Conley (:mconley) (:⚙️)]

Comment on attachment 8761359 [details] [diff] [review]
WIP2: Nits fixed

Review of attachment 8761359 [details] [diff] [review]:
-----------------------------------------------------------------

This looks fine - but based on bsmedberg's comment, we should go with the original plan of doing opt-out collection on release.

Also, bsmedberg gave his data-review+, so no need to re-request, I don't think.

Comment 11

[Christopher Arnold]

Thanks Mike, that sounds good.

Comment 12

[Thauã Silveira]

Comment on attachment 8761359 [details] [diff] [review]
WIP2: Nits fixed

>+  "PRINT_PREVIEW_OPENED_COUNT": {
>+    "alert_emails": ["[e-mail]"],
>+    "bug_numbers": [1275570],
>+    "expires_in_version": "56",
>+    "kind": "count",
>+    "releaseChannelCollection": "opt-out",
>+    "description": "A counter incremented every time the browser enters print preview."
>+  },
>+  "PRINT_PREVIEW_SIMPLIFY_PAGE_OPENED_COUNT": {
>+    "alert_emails": ["[e-mail]"],
>+    "bug_numbers": [1275570],
>+    "expires_in_version": "56",
>+    "kind": "count",
>+    "releaseChannelCollection": "opt-out",

I think I didn't understand, should we use opt-in instead?

>+    "description": "A counter incremented every time the browser enters simplified mode on print preview."
>+  },

Comment 13

[Mike Conley (:mconley) (:⚙️)]

Comment on attachment 8761359 [details] [diff] [review]
WIP2: Nits fixed

Review of attachment 8761359 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Thauã Silveira from comment #12)
> Comment on attachment 8761359 [details] [diff] [review]
> WIP2: Nits fixed
> 
> >+  "PRINT_PREVIEW_OPENED_COUNT": {
> >+    "alert_emails": ["[e-mail]"],
> >+    "bug_numbers": [1275570],
> >+    "expires_in_version": "56",
> >+    "kind": "count",
> >+    "releaseChannelCollection": "opt-out",
> >+    "description": "A counter incremented every time the browser enters print preview."
> >+  },
> >+  "PRINT_PREVIEW_SIMPLIFY_PAGE_OPENED_COUNT": {
> >+    "alert_emails": ["[e-mail]"],
> >+    "bug_numbers": [1275570],
> >+    "expires_in_version": "56",
> >+    "kind": "count",
> >+    "releaseChannelCollection": "opt-out",
> 
> I think I didn't understand, should we use opt-in instead?
> 
> >+    "description": "A counter incremented every time the browser enters simplified mode on print preview."
> >+  },

I'm sorry - I misread your patch! Yes, this looks like it's doing the right thing.

Comment 14

[Mike Conley (:mconley) (:⚙️)]

So r=me, data-review=bsmedberg.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/fx-team/rev/696e8df2724e
"Add Telemetry probe to see how often users enter print preview, and how many simplify the view". r=mconley

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/696e8df2724e