1119291 - Telemetry ingestion: Flash video site ping

Status: RESOLVED FIXED

(Webtools Graveyard :: Telemetry Server, defect)

Description

[Benjamin Smedberg]

For bug 1108668, the experiment will be submitting a ping for Flash video loads. The ping will have at least the following format:

{
  "version": 1,
  "buildid": "2015...",
  "channel": "beta",
  "os": "WINNT",
  "locale": "en-US",
  "videoOrigin": "videos.cdn.facebook.net",
  "swfOrigin": "cdn.facebook.net",
  "pageOrigin": "facebook.com",
  "topOrigin": "facebook.com"
}

I can add other fields if that would help you.

For analysis, I only want these bucketed by day. The data should have normal telemetry data-handling rules except reduce expiration to 30 days instead of 180.

I'd like to have this experiment running in beta by next Thursday (15-Jan).

Comment 1

[Mark Reid [:mreid]]

If we can make the payload look like a stripped-down Telemetry payload, and live with the 180-day expiry period, we can send these to the current server pipeline with no change.  We'd want payloads like:

{
  "ver": 1,
  "info": {
    "reason": "flash-site",
    "appName": "Firefox",
    "appUpdateChannel": "beta",
    "appVersion": "35.0a1",
    "appBuildID": "20150109030201",
    "OS": "WINNT",
    "locale": "en-US"
  }
  ... other fields that are not in standard telemetry payloads ...
}

If the 30-day expiry is a hard requirement, we can implement a periodic task to clean up expired files. Sound OK?

Comment 2

[Benjamin Smedberg]

the expiry is important, but a cleanup task is a fine way to implement it. I think that schema is fine.

Given that, what's the URL and method I should be using?

Comment 3

[Mark Reid [:mreid]]

Submissions should POST the JSON payload to a URL of the form:
https://incoming.telemetry.mozilla.org/submit/telemetry/<uuid>/<reason>/<appName>/<appVersion>/<appUpdateChannel>/<appBuildID>

Using the example from comment 1:
https://incoming.telemetry.mozilla.org/submit/telemetry/2094e844-ef5a-48b7-a468-db4cb0fad287/flash-site/Firefox/35.0a1/beta/20150109030201

Comment 4

[Mark Reid [:mreid]]

I've added a "flash-video" reason (to match the client code at https://github.com/bsmedberg/plugin-request-logging/blob/master/bootstrap.js#L88) and updated the server config.

Comment 5

[Benjamin Smedberg]

Mark, is there anything left to do here (retention fixup?) or can we mark this FIXED. I know we're receiving data.

Comment 6

[Mark Reid [:mreid]]

Yep, data is coming in as expected. I just have to deploy the 30-day expiry task then I'll close this out.

Comment 7

[Mark Reid [:mreid]]

Expire code added with this PR:
https://github.com/mozilla/telemetry-server/pull/88

And the code has been deployed to run daily.