Closed Bug 1194238 Opened 9 years ago Closed 8 years ago

send heartbeat data to telemetry instead of Input

Categories

(Toolkit :: Telemetry, defect, P4)

defect

Tracking

()

RESOLVED DUPLICATE of bug 1193535

People

(Reporter: willkg, Unassigned)

References

Details

Currently Heartbeat data is sent to Input using the heartbeat API and stored in a mysql db.

Input isn't set up to be 100% reliable in regards to capturing all incoming data nor is it set up to store increasing amounts of Heartbeat data. The plan was to use Input as a stopgap backend until the data pipeline system was set up.

This bug covers transitioning Heartbeat from sending data to Input to sending data to the data pipeline.
cc:d the people I cc:d because Gregg suggested this is the group to talk with.

The Input heartbeat API docs are here:

    http://fjord.readthedocs.org/en/latest/hb_api.html

Are there other people that should be involved with this work?

What information is missing from this bug?

How can I move this forward?
If we send the Heartbeat scores to Telemetry (bug 1193535), then you can use the Telemetry analysis infrastructure to analyze & aggregate the data. Should we mark this as a dupe of bug 1193535?
Flags: needinfo?(willkg)
Vladen: I don't do any analysis on the heartbeat data. I'm the developer that built and maintains the heartbeat backend on Input. I think you want to ask Gregg.
Flags: needinfo?(willkg)
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
Reopening, as we want to start working towards storing _all_ of the Heartbeat data we currently store in Input in Telemetry, rather than just the scores and flowId. Bug 1193535 is still relevant but separate from this.

We'll probably wait until the SHIELD[1] system addon lands before we start sending to Telemetry instead of Input, which we're hoping to land sometime in 2016q2. Since that system addon will be what's sending data to Telemetry, I don't think we need any client changes to support this, just a place for the data to live.

I'm NEEDINFOing gregglind to provide a schema of what we want to store that will let us drop Input storage from Heartbeat.

Ilana mentioned a few other useful things to include in this request:

- Top-level key: I don't have a preference here, but gregglind might.
- Any derived streams: I dunno, but Ilana says our answer is probably "no".
- Size of the data: The current input MySQL table with this data is about 4 gigabytes.

[1] https://wiki.mozilla.org/Firefox/Recipe_Server
Status: RESOLVED → REOPENED
Flags: needinfo?(glind)
Resolution: DUPLICATE → ---
Depends on: 1193535
What is left to do here? Is this bug still relevant?
Priority: -- → P4
I'm waiting on Gregglind to confirm that the existing telemetry data being sent is good enough that we can get rid of Input's heartbeat API. If it is, then there's nothing else to do here and we can close this.

Actually, we can probably just close this and file a new bug for any additional information needed at this point.
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Flags: needinfo?(glind)
Resolution: --- → DUPLICATE
I *THINK* this is ok.  +1 to shut down input.
You need to log in before you can comment on or make changes to this bug.