628631 - TSV export has incorrect timestamps

Status: VERIFIED FIXED

(Input :: General, defect)

Description

[Michael Kurze [:michaelk]]

The exported UNIX timestamps, in fact, aren't.

>>> # our Exports are using calendar.timegm
>>> from calendar import timegm
>>> timegm.__doc__
'Unrelated but handy function to calculate Unix timestamp from GMT.'
>>>
>>> # wish I had read that before doing the review
>>> # you'd think timegm would give you, you know, GMT...
>>>
>>> from time import mktime
>>> mktime.__doc__
'mktime(tuple) -> floating point number\n\nConvert a time tuple in local time to seconds since the Epoch.'
>>>
>>> # aaaahaaa


TL;DR: 
we want to use int o time.mktime instead of calendar.gmtime here

https://github.com/michaelku/reporter/commit/a7864db06595898a1a12122c5fbeaefd928e6783




The bug in action:

--- website (OK):
http://input.stage.mozilla.com/en-US/opinion/1163642
<time datetime="2011-01-22T23:01:00Z" title="Jan 22, 2011 3:01:00 PM">2 days ago</time>

-- database (system timezone dependent, but OK):
mysql> -- date adjusted cause on my system the DB runs on UTC+1 while `created` is LA time (currently UTC-8)
mysql> select UNIX_TIMESTAMP(DATE_ADD(`created`, INTERVAL 9 HOUR)) as t from feedback_opinion where id=1163642;
+------------+
| t          |
+------------+
| 1295737260 |
+------------+

--- TSV export (does not make the UNIX time):
1163642 1295708460 praise firefox 4.0b9 winxp en-US facbook.com

mysql> select (1295737260 - 1295708460)/3600;
+--------------------------------+
| (1295737260 - 1295708460)/3600 |
+--------------------------------+
|                         8.0000 |
+--------------------------------+

So, the timestamp in the export lags 8 hours behind ...just as I suspected *rubs hands*.

Comment 1

[Fred Wenzel [:wenzel]]

Oooh, that's no good.

Thanks for writing the patch. I pulled it into master: http://github.com/fwenzel/reporter/commit/01b1659

Marking this qa-, because Michael can verify this himself.

I'll have IT run the cron job once manually for us.

Comment 2

[Fred Wenzel [:wenzel]]

Jabba ran the export script for us by hand. This is fixed: Please take a look at the data now.

Comment 3

[Michael Kurze [:michaelk]]

Now we have

...
1163642 1295737260 praise firefox 4.0b9 winxp en-US facbook.com
...

And that is good --- same as mysql above!