Closed Bug 1569672 Opened 2 months ago Closed 2 months ago

Collect denominators for use counters in release

Categories

(Toolkit :: Telemetry, task)

task
Not set

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox68 --- fixed
firefox69 --- fixed
firefox70 --- fixed

People

(Reporter: tdsmith, Assigned: tdsmith)

References

Details

Attachments

(2 files)

Use counters encode the count of documents that have used a given platform feature. The CONTENT_DOCUMENTS_DESTROYED and TOP_LEVEL_CONTENT_DOCUMENTS_DESTROYED probes, as implemented in bug 1204994, are the denominators for the use counters.

Bug 1477433 began collecting use counters in release, but the denominators are still only recorded in prerelease. We need to flag the denominator histograms for collection in release.

Use counters are collected in release as of bug 1477433. Per bug 1204994, use counters need to be interpreted relative to CONTENT_DOCUMENTS_DESTROYED and TOP_LEVEL_CONTENT_DOCUMENTS_DESTROYED, so these need to be collected in release, as well.

Also cleans up probe definitions and removes from whitelists.

Attached file Data collection review
Attachment #9081364 - Flags: data-review?(bmiroglio)
Comment on attachment 9081364 [details]
Data collection review


1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way? 
Data will be documented in the [probe dictionary](https://telemetry.mozilla.org/probe-dictionary/). 

2. Is there a control mechanism that allows the user to turn the data collection on and off? 
Yes, can be turned off in preferences. 

3. If the request is for permanent data collection, is there someone who will monitor the data over time?
Yes, tdsmith + the client telemetry team

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2 - Interaction Data

5. Is the data collection request for default-on or default-off?
Default on. 

6. Does the instrumentation include the addition of any new identifiers?
No new identifiers. 

7. Is the data collection covered by the existing Firefox privacy notice?
Yes.

8. Does there need to be a check-in in the future to determine whether to renew the data? 
No. 

9. Does the data collection use a third-party collection tool? 
No. 
—
Data review +
Attachment #9081364 - Flags: data-review?(bmiroglio) → data-review+
Pushed by tismith@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/254df321d0df
Collect use counter denominators in release r=janerik
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70

Comment on attachment 9081336 [details]
Bug 1569672 - Collect use counter denominators in release

Beta/Release Uplift Approval Request

  • User impact if declined: These probes make it possible to interpret the use counters that have shipped in release. Having these probes available will help us make better webcompat decisions sooner.
  • Is this code covered by automated tests?: Unknown
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Not risky; change limited to histogram definitions.
  • String changes made/needed: none
Attachment #9081336 - Flags: approval-mozilla-beta?

Comment on attachment 9081336 [details]
Bug 1569672 - Collect use counter denominators in release

If possible, getting this out in the next dot release would be super helpful for current webcompat metrics efforts.

Attachment #9081336 - Flags: approval-mozilla-release?

Comment on attachment 9081336 [details]
Bug 1569672 - Collect use counter denominators in release

Improvements to use counters so we can better interpret the data coming from them. Approved for 69.0b10.

I'm less enthusiastic about uplifting to release for 68.0.2, however. This doesn't appear to be fixing any user-facing issue or a big regression shipped. What webcompat decisions are going to be made easier by getting newer data from this probe a month before we ship Fx69?

Flags: needinfo?(miket)
Attachment #9081336 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

I'm less enthusiastic about uplifting to release for 68.0.2, however. This doesn't appear to be fixing any user-facing issue or a big regression shipped. What webcompat decisions are going to be made easier by getting newer data from this probe a month before we ship Fx69?

I'm working with Mike (who's on PTO for the next few days) to produce metrics for Firefox webcompat in support of the Gecko codevelopment goal. The specific things I want to do soon are a) to compare numbers coming out of our use counter infrastructure with Chrome's use counter data to understand whether they're comparable, and b) to compare use counter numbers with crawls that we're going to start piloting this month to understand whether they're comparable. Ultimately, we want to count attempts by web pages to use APIs that we don't support, and we think that a use-counter-like mechanism will be the way to go. If our use counters indicate something radically different from the other datasets, we'll want to figure out why. Waiting for beta (for which: thanks!) won't kill us, but riding along would help remove some risk from our schedule, which is targeting late September for our key deliverables.

Flags: needinfo?(miket)

How do bugs like 1568950 affect that data and its analysis?

Comment on attachment 9081336 [details]
Bug 1569672 - Collect use counter denominators in release

anyway.. I guess we can take this as a ride along, assuming there are no perf implications to enabling that collection on release. approved for 68.0.2.

Attachment #9081336 - Flags: approval-mozilla-release? → approval-mozilla-release+
You need to log in before you can comment on or make changes to this bug.