888281 - Dashboard updates do no appear to be incremental

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[(dormant account)]

Looks like we might be blowing away old data with every map/reduce job. Daily Submissions graph looks like there is only 1 day of data even though there should be 2 days of data in dashboard by now

Comment 1

[(dormant account)]

Looks like the dashboard just isn't updating...Can we set it on a 6hour update interval to knock all of the bugs out of this?

Comment 2

[Laura Thomson :laura]

Hmm, maybe a cron permission issue?  Moving to webops so they can tweak the crontab for testing purposes.

Webops: can we run the batch cron every 6 hours instead of daily at 0205, and can you check why it didn't run last night?

Details of the cron and where it runs are here:
https://mana.mozilla.org/wiki/display/websites/telemetry-dash.mozilla.org

We should also get a monitor on that cron so we know when it fails.

Comment 3

[Jake Maul [:jakem]]

It appears to have updated just fine:

-rw-r--r--  1 telemetrydash telemetrydash  822 Jul  1 03:02 versions.json

Don't know what the problem is, but the job appears to have ran without issue.

After talking with :laura, we're going to push this back over the fence to Webtools and assign to :lonnen.

Comment 4

[Lonnen :lonnen]

the Makefile has some magic hardcoded files in it that it pulls down.

* validation/nightly/23.0a1/histogram_descriptions.json
* http://hg.mozilla.org/mozilla-central/raw-file/139b6ba547fa/toolkit/components/telemetry/Histograms.json
* http://hg.mozilla.org/mozilla-central/raw-file/139b6ba547fa/toolkit/components/telemetry/histogram_tools.py

which could be the cause of the static data.

Comment 5

[Lonnen :lonnen]

Also, even if we had incremental increases I believe the current build process blows away whatever was there before the build finished.

Comment 6

[Lonnen :lonnen]

Fixed in 1625d4f0e828233403f1e5fe3a4aafe40752c704