949764 - apps_added_by_* stats stopped recording after the 9th

Status: RESOLVED FIXED

(Marketplace Graveyard :: Statistics, defect)

Description

[Rob Hudson [:robhudson]]

Querying the monolith_record table on prod I see that the stats starting with "apps_added_by_" stopped recording.

You can see this by executing:
select * from monolith_record where recorded >= '2013-12-08' and `key` like 'apps_added_by_%' order by recorded;

I checked logs for any errors but I'm not certain cron logs are collected with the other logs?

If we can run this manually, perhaps with bug 949759 we might see some errors or logs as to what might be wrong?

Comment 1

[Rob Hudson [:robhudson]]

Updates:
* 'apps_added_by_%' were logged on the 8th, 9th, and now 10th.
* We manually ran them on the 11th to test if the code path was working as expected. It worked fine.
* They are running, but just returning a count of zero which we don't insert into the database.
* The logs are in the AMO logs since the cron is in AMO.

If we need to manually update these here's the small script to run:

> import datetime
> from stats.tasks import update_monolith_stats
> 
> day = datetime.date(2013, 12, 11)
> update_monolith_stats('apps_added_by_package_type', day)
> update_monolith_stats('apps_added_by_premium_type', day)

So far I still don't see why these are returning zero counts when run with the cron job but not when run manually. Or why dev doesn't show this problem.

Comment 2

[Rob Hudson [:robhudson]]

With the help of Jason we figured out what's going on...

We moved the cron to 0810 UTC so that it runs just after midnight PST. But now the cron is running, e.g., Dec 15th 00:10 PST and querying for stats for Dec 15th. It should be running Dec 15th 00:10 PST and querying for stats for Dec 14th.

This updates the cron to add the explicit date we want to collect stats for.
https://github.com/mozilla/zamboni/commit/8eb2012 

To verify:
After push and monolith extraction verify that the apps_added graphs have recent data. https://marketplace.firefox.com/statistics/apps-by-type/