1600385 - Settle on a Design for the FOGotype Glean ping uploader

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

Though the design proposal has answers to many questions, I neglected to put anything in there about how we were going to get pings to the pipeline.

This might have been in consideration of broader potential architectural changes (like making the uploader a callback), but I honestly can't remember. I might have just omitted it through ignorance.

The important thing to note is that, after the call to sendPing returns true, there is a ping on disk in the configured Glean SDK data directory. It is up to Firefox Desktop to send it.

We have options (TelemetrySend.jsm and Pingsender come to mind), let's evaluate them for suitability and make a decision.

Comment 1

[Chris H-C :chutten]

TelemetrySend.jsm

Pros:

• May provide access to the Archive
  • (which might even include about:telemetry support (Raw JSON only, I'm sure))
• Built-in compliance for data upload pref
• Reliable retry

Cons:

• Unknown level of deep integration with the Common Ping Format.
  • It'll definitely at least need changes to send to Structured Ingestion instead of Telemetry Ingest
• Written in JS (have to get there from Rust somehow... maybe reformat the ping and move to the archive dir?)

Pingsender

Pros:

• Invoking it via nsIProcess shouldn't be too hard from Rust
• Takes the endpoint as a parameter, so we can point it at Structured Ingestion without grief
• Doesn't require modification of a core component of Firefox Telemetry

Cons:

• No retry
• No handling for failure (deleting on bad ping format, archiving on successful send)

---

Neither option supports the Glean SDK Ping Format (with the endpoint on the first line and the payload the second). Both options would satisfy the requirements for the FOGotype (learning that and how a ping can be sent).

As either would suffice, I'm leaning towards the Pingsender option as we might be able to carry it off without modifying in-tree Firefox Telemetry code. We should be able to reformat (strip the endpoint out of) the file easily enough in Rust and pass it along via nsIProcess. We can observe the process complete (successfully or in failure) and then delete the file ourselves if it remains.

Thoughts, Jan-Erik?

Comment 2

[Chris H-C :chutten]

Further detail: To avoid cluttering the profile directory, let's use NS_OS_TEMP_DIR to store our ping files.

Comment 3

[Jan-Erik Rediger [:janerik]]

I think Option 2: Pingsender is the better option.

We need some processing around the files anyway (picking them up, extracting the URL, etc).
Mixing it into about:telemetry/the archive doesn't seem like such a big usage, the fact that we need to deal with the commong ping format seems suboptimal.
Going the pingsender route also means it's easier to rip out again.
Plus we can keep that code nightly-only and don't need to defend against changes to a core component that only end up getting used on nightly anyway.

Comment 4

[Chris H-C :chutten]

Forgot to close this one off. Pingsender it is, then.