1762281 - Improve first-instrumentation experience with docs, about:glean

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P2)

Description

[Chris H-C :chutten]

k88hudson was adding some Glean metrics to Firefox Desktop today and brought up some pain points.... what's devastating isn't that there were pain points, but that I'd heard them all before.

Let's fix them.

Pain points:

1. I've just instrumented something, how do I "gut check" that it's actually running?
2. Maybe I've found the Book of Glean docs on Debugging things but how do I actually do this? It's a little abstract.
3. Maybe I've found the section of the FOG docs on manual testing, but it just routes me to the Book of Glean without really helping me understand what to do.

Proposed fixes:

1. In the FOG New Metrics and Ping Docs link to the section of the FOG docs on manual testing. Augment that section to include information on how to use test APIs to "gut check" your new collection.
   • Provide examples, for goodness' sake!
2. Add "Debugging Firefox Desktop using the Glean SDK" subpage. Either link out to FOG docs or put the content in there.
   • Provide examples, for goodness' sake!
3. See #1

Comment 1

[Chris H-C :chutten]

Another pain point:

4. I've found myself on about:glean... now what?

Proposed fix:

4. Yes, yes, I know chutten fully intends to fully rework about:glean to be A Great Place, but that's a much bigger lift and is far in the future. For now we could at least update the strings and links to point at the FOG Instrumentation Testing docs properly, supply an example of how to use it, and fix the button label (bug 1743368)

Comment 2

[Ted Campbell [:tcampbell]]

Pain point:
5. During early development, I like to use about:telemetry to see distributions in a graph form and make sure I'm not way off-base in what I am measuring. In the glean workflow, I can use Glean Debug Pings Viewer to get raw data, but then then need to manually graph it (matplotlib, google sheets, etc). A more direct pathway would be pretty useful for me to prototype new metrics.

Comment 3

[Chris H-C :chutten]

Attachment: divDistDisplay.png

I know it's not much, but here's a dirt-simple distribution display for local values. I've tested it by opening about:glean, popping open devtools, and using Glean.wr.framebuildTime.testGetValue() as my aDist (see screenshot attachment).

let aDist = Glean.myCat.myName.testGetValue();
let sortedValues = Object.entries(aDist.values).sort((a, b) => -a[0] < -b[0]); // belt and suspenders
let maxBucketCount = sortedValues.reduce((prev, cur) => prev < cur[1] ? cur[1] : prev, 0);
let distDiv = document.createElement('div');
distDiv.style.display = 'flex';
distDiv.style.flexDirection = 'column';
distDiv.style.width = '50%';
for (let [bucket, count] of sortedValues) {
    let barDiv = document.createElement('div');
    barDiv.style.width = 100 * count / maxBucketCount + '%';
    barDiv.style.height = '1em';
    barDiv.style.backgroundColor = 'blue';
    barDiv.textContent = bucket;
    barDiv.title = `bucket: ${bucket}, count: ${count}`;
    distDiv.appendChild(barDiv);
}
document.body.appendChild(distDiv);

Comment 4

[Chris H-C :chutten]

Another improvement that'd help out: When submitting a tagged ping using about:glean's UI, we could and should then supply a link that'll take you directly to your tag's ping list (like https://debug-ping-preview.firebaseapp.com/pings/chutten-dev for the tag chutten-dev).

• Gotta double-check that the Debug Viewer works if the tag has no pings yet (ie, will the list automatically update if it's empty to begin with?)
• Won't necessarily play well with tags set by envvar, or across refreshes of the about:glean page (oh well. That's a corner of a corner case)

Comment 5

[Chris H-C :chutten]

Another improvement:

about:telemetry tells you whether upload is enabled/disabled and whether you're collecting release or pre-release data. about:glean could/should/might benefit from having a similar display of prefs, envvars, and config that affect FOG or Glean behaviour. "Upload is disabled because !MOZILLA_OFFICIAL, but you have set debug tag chutten-dev so pings will be uploaded to the Debug Ping Viewer." or similar...

Comment 6

[Travis Long [:travis_]]

When just setting the Debug View Tag in about:glean, leaving the ping-you-wish-to-send input blank, the console spits out a message indicating it tried to send out an unknown ping "", it should probably short-circuit and not try to send anything if the box is empty, or perhaps have its own button to trigger sending of pings.

Comment 7

[Chris H-C :chutten]

Attachment: Bug 1762281 - Improve testing docs r?perry.mcmanis!

Comment 8

[Chris H-C :chutten]

Attachment: Bug 1762281 - Clarify FOG prefs and defines docs r?perry.mcmanis!

Depends on D168158

Comment 9

[Chris H-C :chutten]

Attachment: Bug 1762281 - about:glean redesign r?brosa!

Be more clear about what Glean is and how to debug it.
Reference more documentation.
Be more helpful.

Depends on D168159

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4612eef477f6
Improve testing docs r=perry.mcmanis
https://hg.mozilla.org/integration/autoland/rev/9a6decfe1575
Clarify FOG prefs and defines docs r=perry.mcmanis
https://hg.mozilla.org/integration/autoland/rev/3538720506fb
about:glean redesign r=brosa,bolsson

Comment 11

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/4612eef477f6
https://hg.mozilla.org/mozilla-central/rev/9a6decfe1575
https://hg.mozilla.org/mozilla-central/rev/3538720506fb