One hypothesis for d0/.../d30 being off in bug 1284932 is, that we count clients in these more than once. We need to confirm this with an analysis.
This re:dash query keeps getting killed: https://sql.telemetry.mozilla.org/queries/767 Blake says this is pulling Gigabytes of data and gets killed by the OOM watcher. I'll have to move this to some kind of Spark job.
Ok, i ran an analysis on this and thats definitely not the problem: https://gist.github.com/georgf/0ffe4f915861be180909037a7204d7b9 There are only 4 clients for which "profiledate == submissiondate" was true for more than one day. Furthermore, there are only 6509 clients who ever submitted more than one profiledate value. This was my first time using sqlContext though - Mauro, do you think you could give this notebook a quick review?
Per IRC, Mauro found the analysis ok.