Shield studies submit pings of type "shield-study". Can we please add this to the telemetry schema for docType so that they are broken out of the OTHER bucket.
can oyu take a quick look :whd?
I'll deploy this by end of week.
https://github.com/mozilla-services/puppet-config/pull/2425 This has been deployed, from UTC Dec. 15 forward the data should no longer be in the OTHER bucket.
This change only affects data from the 15th forward. Since the deploy occured on the 15th, data is in both the OTHER and shield-study buckets for that day, so here's how you might get all the pings (python): submissionDate = "20161215" dataset = Dataset.from_source("telemetry").where(submissionDate=submissionDate) pings = dataset.where(docType='OTHER').records(sc, sample=1).filter(lambda p: p.get('type') == 'shield-study') pings2 = dataset.where(docType='shield_study').records(sc, sample=1) (pings + pings2).collect() You should use the former records() call for days preceding and the latter for days succeeding.