1134503 - Capture FxOS AppUsage pings that come in on invalid URLs

Status: RESOLVED DUPLICATE of bug 1180058

(Webtools Graveyard :: Telemetry Server, defect)

Description

[Dave Zeber [:dzeber]]

Current (v2.1) FxOS AppUsage pings are getting sent in on an invalid URL and subsequently discarded by the server. 

We'd like to capture these pings as they come in and divert them to some storage that can be parsed or queried. This will enable us to recover some amount of activity data on FxOS until the new ping lands in 2.2.

Comment 1

[Mark Reid [:mreid]]

We would need to update the server's validation code to do the following:
- allow a non-standard number of URL Path components
- handle the AppUsage payload format (missing 'ver' field, missing 'info' section)

The relevant section of server-side code is here:
Path check: https://github.com/mozilla/telemetry-server/blob/master/process_incoming/process_incoming_standalone.py#L315
Payload validation/conversion: https://github.com/mozilla/telemetry-server/blob/master/telemetry/convert.py#L149

Comment 2

[Dave Zeber [:dzeber]]

Can we tell the appusage pings apart from other invalid pings? I think the URL contains the string "appusage". 

If can detect appusage pings somewhere in the invalid URL check, maybe we can inject version/info so that they can get processed in the usual way. If possible we could give them their own reason code (eg. "oldappusage") or else send them to reason=="OTHER", but I'd rather keep them separate from the new appusage pings (reason=="appusage") for now.

How much work do you think this would be to deploy?

Comment 3

[Mark Reid [:mreid]]

It would probably be a day or 2 to develop, test, and deploy. I won't have time to look at it this week, but possibly next week.

Comment 4

[Mark Reid [:mreid]]

This work will be addressed as part of the migration to the Unified Telemetry backend over in bug 1180058.