Telemetry ingestion: Flash video site ping

RESOLVED FIXED

Status

Webtools
Telemetry Server
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: bsmedberg, Assigned: mreid)

Tracking

Trunk
x86_64
Windows 7

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
For bug 1108668, the experiment will be submitting a ping for Flash video loads. The ping will have at least the following format:

{
  "version": 1,
  "buildid": "2015...",
  "channel": "beta",
  "os": "WINNT",
  "locale": "en-US",
  "videoOrigin": "videos.cdn.facebook.net",
  "swfOrigin": "cdn.facebook.net",
  "pageOrigin": "facebook.com",
  "topOrigin": "facebook.com"
}

I can add other fields if that would help you.

For analysis, I only want these bucketed by day. The data should have normal telemetry data-handling rules except reduce expiration to 30 days instead of 180.

I'd like to have this experiment running in beta by next Thursday (15-Jan).
Flags: needinfo?(mreid)
(Assignee)

Comment 1

3 years ago
If we can make the payload look like a stripped-down Telemetry payload, and live with the 180-day expiry period, we can send these to the current server pipeline with no change.  We'd want payloads like:

{
  "ver": 1,
  "info": {
    "reason": "flash-site",
    "appName": "Firefox",
    "appUpdateChannel": "beta",
    "appVersion": "35.0a1",
    "appBuildID": "20150109030201",
    "OS": "WINNT",
    "locale": "en-US"
  }
  ... other fields that are not in standard telemetry payloads ...
}

If the 30-day expiry is a hard requirement, we can implement a periodic task to clean up expired files. Sound OK?
Flags: needinfo?(mreid) → needinfo?(benjamin)
(Reporter)

Comment 2

3 years ago
the expiry is important, but a cleanup task is a fine way to implement it. I think that schema is fine.

Given that, what's the URL and method I should be using?
Flags: needinfo?(benjamin)
(Assignee)

Comment 3

3 years ago
Submissions should POST the JSON payload to a URL of the form:
https://incoming.telemetry.mozilla.org/submit/telemetry/<uuid>/<reason>/<appName>/<appVersion>/<appUpdateChannel>/<appBuildID>

Using the example from comment 1:
https://incoming.telemetry.mozilla.org/submit/telemetry/2094e844-ef5a-48b7-a468-db4cb0fad287/flash-site/Firefox/35.0a1/beta/20150109030201
(Assignee)

Comment 4

3 years ago
I've added a "flash-video" reason (to match the client code at https://github.com/bsmedberg/plugin-request-logging/blob/master/bootstrap.js#L88) and updated the server config.
(Reporter)

Comment 5

3 years ago
Mark, is there anything left to do here (retention fixup?) or can we mark this FIXED. I know we're receiving data.
Assignee: nobody → mreid
Flags: needinfo?(mreid)
(Assignee)

Comment 6

3 years ago
Yep, data is coming in as expected. I just have to deploy the 30-day expiry task then I'll close this out.
Flags: needinfo?(mreid)
(Assignee)

Comment 7

3 years ago
Expire code added with this PR:
https://github.com/mozilla/telemetry-server/pull/88

And the code has been deployed to run daily.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.