Closed Bug 1635253 Opened 5 years ago Closed 5 years ago

Debug Tagging and QA Support for Project FOG

Categories

(Toolkit :: Telemetry, task, P1)

Product:
Toolkit
Component:
Telemetry
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
81 Branch
Tracking Flags:
Tracking Status
firefox81 --- fixed

People

(Reporter: chutten, Assigned: chutten)

References

Details

(Whiteboard: [telemetry:fog:m4])

Attachments

(5 files)

Bug 1635253 - Add new about:glean to control Glean SDK debugging, logging r?janerik!
47 bytes, text/x-phabricator-request
Details | Review
Bug 1635253 - Update glean-core to 31.4.0 r?janerik!
47 bytes, text/x-phabricator-request
Details | Review
Bug 1635253 - Tweak FOG to be able to compile against updated Glean SDK r?janerik!
47 bytes, text/x-phabricator-request
Details | Review
Bug 1635253 - Pass the about:glean commands to the Glean SDK r?janerik!
47 bytes, text/x-phabricator-request
Details | Review
Bug 1635253 - Test that debug tagging works via about:glean r?janerik!
47 bytes, text/x-phabricator-request
Details | Review

How may we best serve QA needs with FOG? At the very least we want to be able to avail ourselves of the Debug Ping Viewer, and to do that we'll need to be able to tag pings and trigger them.

  • How will we do that?
    • Command-line? about:telemetry? about:glean?
  • Will tagging be enough?
    • Will embedding an iframe of the debug ping viewer be helpful? How will we know it's address?
    • Will we need to support pointing the ping server at localhost:NNNN? How will we know the port?
    • What else can we help with?
  • What holes are there in FOG's present test setup?
    • Can we add Gecko symbols to rusttests via an appropriately-worded build.rs? And if so, do we have to keep updating it every time we add more symbols?
    • What does this mean for the parts of FOG that will be generating JS and C++ code? The parts that are JS and C++ code? (if there will be such parts)
  • Don't forget to add all this to the in-tree documentation.

We must consult with QA (:raphael's an excellent choice) about some or most or all of these questions, propose a design, get it approved, implement the design, and document the design.

Maybe this is a metabug.

Depends on: 1643395
Depends on: 1632790
Depends on: 1646706

The present Glean SDK Debugging use cases can best be seen in the Glean SDK book here. In short, there are two pieces of state (the ping tag and whether we log pings) and one command (to send a named ping). All pings being assembled and sent adhere to the two pieces of state that are current at the time of assembly and sending. The pieces of state can be set during runtime or (ping tag only) before launch (via envvar, see bug 1646706).

So to bring us up to parity with other platforms, FOG needs to provide a mechanism to, at runtime, set the two pieces of state (the ping tag and whether we log pings) and to assemble and send a named ping. I think the most ergonomic way for FOG would be through an about:glean that could take these actions from query parameters in the url, a la

  • about:glean?tagPings=my-debug-tag
  • about:glean?logPings=true -- note that this just turns logging on. If the log level (MOZ_LOG or RUST_LOG) for glean_core (glean_core::* for MOZ_LOG) isn't also set to at least DEBUG (3) then the pings won't log.
  • about:glean?sendPing=baseline

In addition we'll support multiple simultaneous commands, running the tagging first, the logging second, and the sending third:

  • about:glean?sendPing=baseline&tagPings=my-debug-tag&logPings=true will tag all pings with my-debug-tag, then turn on logging, then assemble and send the "baseline" ping (which will be tagged and logged).

:raphael, how does this sound in terms of bringing capabilities even with other Glean products? Can you think of any FOG-specific capabilities this might miss? (if not just now, no worries, we have another test checkin already planned for a future milestone)

Flags: needinfo?(rpierzina)
Assignee: nobody → chutten
Status: NEW → ASSIGNED
Priority: P3 → P1

Hi chutten, as we've discussed on Zoom using environment variables for setting ping tags sounds very reasonable to me. I also think it makes sense to use about:glean to change the Glean SDK debugging state at runtime. I discussed this with Krupa and asked her for feedback on the proposed design.

Flags: needinfo?(rpierzina) → needinfo?(kraj)

Included documentation, but no tests, as it doesn't yet actually control the
Glean SDK.

The Glean SDK was recently updated to centralize debug tagging support.
We're gonna need that.

Depends on D83977

Input from the #security:mozilla.org channel on Element suggests that UI is better than query strings for chrome-privileged pages. The attack surface that allows folks to trick the browser or the user to navigate to about:glean?speciallyCraftedQueryStringThatGivesAttackersControlSomehow is broader than the attack surface provided by some form controls in HTML.

I propose to change the design to have three inputs: a checkmark for logPings, a text input for tagPings, and a text input for submitPing. And a button. This should serve the needs of manual verification of data collection. For automated verification I assume that automation has access to execute chrome-privileged script, in which case they can call nsIAboutGlean methods directly.

ni?raphael, would this design still work for your use cases?

Flags: needinfo?(rpierzina)

(In reply to Chris H-C :chutten from comment #8)

I propose to change the design to have three inputs: a checkmark for logPings, a text input for tagPings, and a text input for submitPing. And a button. This should serve the needs of manual verification of data collection. For automated verification I assume that automation has access to execute chrome-privileged script, in which case they can call nsIAboutGlean methods directly.

Fly-by note: Glean also supports environment variables :-)

(In reply to Alessio Placitelli [:Dexter] from comment #9)

(In reply to Chris H-C :chutten from comment #8)

I propose to change the design to have three inputs: a checkmark for logPings, a text input for tagPings, and a text input for submitPing. And a button. This should serve the needs of manual verification of data collection. For automated verification I assume that automation has access to execute chrome-privileged script, in which case they can call nsIAboutGlean methods directly.

Fly-by note: Glean also supports environment variables :-)

Yes, but the environment variables don't let you submit pings : )

(In reply to Chris H-C :chutten from comment #8)

ni?raphael, would this design still work for your use cases?

Sounds good to me!

Flags: needinfo?(rpierzina)
Pushed by jrediger@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/173120b52295 Add new about:glean to control Glean SDK debugging, logging r=janerik,flod https://hg.mozilla.org/integration/autoland/rev/f80c57cc4abc Update glean-core to 31.4.0 r=janerik https://hg.mozilla.org/integration/autoland/rev/7a669b03b0ec Tweak FOG to be able to compile against updated Glean SDK r=janerik https://hg.mozilla.org/integration/autoland/rev/4cd9c888d01b Pass the about:glean commands to the Glean SDK r=janerik https://hg.mozilla.org/integration/autoland/rev/d3679cfb5b68 Test that debug tagging works via about:glean r=janerik
Regressions: 1656680

Sounds like the Pioneer one we ought to be able to adapt to an about:glean-based (with Debug Ping Viewer) verification mechanism after Pioneer migrates to Glean (in 2021). So we're clear with the implementation, no follow-ups to file (yet).

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: