1315663 - Add URI Count and Domain Count to main summary

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[Mark Reid [:mreid]]

Per bug 1271313, we have a couple of new measures landing on the release channel. We should add fields to the main_summary dataset for two scalar measures:

browser.engagement.unique_domains_count
browser.engagement.total_uri_count

Comment 1

[:Harald Kirschner :digitarald]

:mreid, could we get this prioritized as it blocks us from comparing our crash rate against the numbers that Chrome shared with us; which is crucial for baselining our stability efforts.

Comment 2

[Mark Reid [:mreid]]

I'm planning to add all the "browser.engagement" metrics defined in Scalars.yaml, sound OK? Specifically:
max_concurrent_tab_count
tab_open_event_count
max_concurrent_window_count
window_open_event_count
total_uri_count
unfiltered_uri_count
unique_domains_count


Do we need to add the browser.engagement.navigation ones at the same time?

Comment 3

[GitHub Autolander Bot]

Attachment: [telemetry-batch-view] mreid-moz:bug1315663 > mozilla:master

Comment 4

[:Harald Kirschner :digitarald]

:joy was blocking on this for bug 1315406, so ni?

Comment 5

["Saptarshi Guha[:joy]"]

From bug  1315406, we need

"Chrome measures crashes per million pageloads (CPM), where pageloads include subframes and crashes include any unclean shutdown."


This bug is supposed to collection information to answer "pageloads include subframes". Which measures  in Comment 2 answer this?

Comment 6

[Mark Reid [:mreid]]

Looks like total_uri_count should give you the denominator you need. More info in the scalar definitions at
https://dxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/Scalars.yaml

Comment 7

["Saptarshi Guha[:joy]"]

then that is all that is required (i think)

Comment 8

["Saptarshi Guha[:joy]"]

would i see this in main_summary in redash as of today? 
how far back would it go?

Comment 9

[Mark Reid [:mreid]]

It's not there yet - the new columns will appear the day after the attached PR lands.

It will need to be backfilled separately as far back as you need - is 1 month of backfill a good amount?

Comment 10

["Saptarshi Guha[:joy]"]

1. Do you have an idea when the PR lands? Thanks
2. Harald, how far back do you want backfilled?

Comment 11

[Mark Reid [:mreid]]

(In reply to "Saptarshi Guha[:joy]" from comment #10)
> 1. Do you have an idea when the PR lands? Thanks
Hoping it'll land tomorrow. Ran into a detour with my initial approach. The current method should be OK.

Comment 12

[:Harald Kirschner :digitarald]

> 2. Harald, how far back do you want backfilled?

1 month should be enough

Comment 13

[Mark Reid [:mreid]]

Comment on attachment 8814253 [details] [review]
[telemetry-batch-view] mreid-moz:bug1315663 > mozilla:master

r=roberto per the attached PR.

See Bug 1321316 for follow-up on backfill.

Comment 14

["Saptarshi Guha[:joy]"]

I also recommend adding this count to crash_aggregates table