Some update counts are recorded on Thursday instead of Wednesday

RESOLVED FIXED

Status

RESOLVED FIXED
11 years ago
3 years ago

People

(Reporter: fligtar, Assigned: oremj)

Tracking

Details

(Reporter)

Description

11 years ago
spinoff from bug 421951

Need to figure out why some counts are recorded on Thursday instead of Wednesday... possibly related to timezones.
(Reporter)

Updated

11 years ago
Duplicate of this bug: 425184
Justin, any estimate on when this might be fixed? 

The statistics feature is really unusable right now (except for the total download count) as all the additional functionality (addon status, application status, OS, addon version) is based on the number of active users.
(Reporter)

Comment 3

11 years ago
Gahh. I can't believe it took me this long to figure it out.

IT has a script that runs 4 different stats commands to parse the update pings. This takes several hours and is started on Thursday night, extending into Friday morning. I've been looking for problems with my script, but the problem is actually in that cron script.

We use `date --date='yesterday'` to parse for Wednesday's stats, but after it becomes Friday morning, any scripts that haven't run yet will actually be counting the logs of the new yesterday, Thursday.

oremj, can you change the script so that we set the yesterday date in a variable at the start of the script so it won't change later?
Assignee: nobody → oremj
(Assignee)

Comment 4

11 years ago
Done.

#!/bin/bash

DATE=`date --date='yesterday' +%Y-%m-%d`
cd /data/stats/amo_stats/bin/parse_logs;
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/versioncheck.addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$DATE/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/versioncheck.addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$DATE
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → FIXED
(Reporter)

Comment 5

11 years ago
Hmm, this line:

/usr/bin/php -f parse_logs.php
logs=/data/stats/logs/nladm01/addons.mozilla.org/ geo=NL temp=/tmp
type=updatepings date=$DATE

used to have a different date format because the logs names are like access_050908. I guess we'll need 2 variables.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 6

11 years ago
Good catch.
(Assignee)

Comment 7

11 years ago
SJ_DATE=`date --date='yesterday' +%Y-%m-%d`
NL_DATE=`date --date='yesterday' +%m%d%y`
cd /data/stats/amo_stats/bin/parse_logs;
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$SJ_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$NL_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/versioncheck.addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$SJ_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/versioncheck.addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$NL_DATE
Status: REOPENED → RESOLVED
Last Resolved: 11 years ago11 years ago
Resolution: --- → FIXED
(Reporter)

Comment 8

11 years ago
versioncheck.addons.mozilla.org NL logs actually use the same format at SJ.

addons.mozilla.org NL is the only one that's different.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 9

11 years ago
SJ_DATE=`date --date='yesterday' +%Y-%m-%d`
NL_DATE=`date --date='yesterday' +%m%d%y`
cd /data/stats/amo_stats/bin/parse_logs;
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$SJ_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$NL_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/im-log01/versioncheck.addons.mozilla.org/ geo=SJ temp=/tmp type=updatepings date=$SJ_DATE
/usr/bin/php -f parse_logs.php logs=/data/stats/logs/nladm01/versioncheck.addons.mozilla.org/ geo=NL temp=/tmp type=updatepings date=$SJ_DATE
Status: REOPENED → RESOLVED
Last Resolved: 11 years ago11 years ago
Resolution: --- → FIXED

Updated

11 years ago
Blocks: 426038
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.