281132 - Improve handling of bad date values in the series_data table

Status: NEW

(Bugzilla :: Reporting/Charting, defect)

Description

[Albert Ting]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041223 Firefox/1.0
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041223 Firefox/1.0

When upgrading from 2.18 to 2.19.2+, I had one entry in my series_data table
with the series_date value of "0".  Seems like the str2time() call in
Bugzilla/Chart.pm does not handle this correctly, and returns an empty string. 
Net result, bugzilla was unable to produce a chart.

I don't know how I got a data row with series_date=0.  I suppose check_setup.pl
could be updated to check/purge these. 

I don't how best to fix this, but it seems Chart.pm should at least output an
error message somewhere.  Had to use the debug flag to track this down.

I ended up doing two things:

   1. purge the series* tables and regenerate (I didn't have any user saved series)
   2. Have Chart.pm ignore all rows with series_date=0:

--- Bugzilla/Chart.pm	2005-02-01 22:44:51.000000000 -0800
+++ Bugzilla/Chart.pm	2005-02-04 15:55:07.000000000 -0800
@@ -212,7 +213,8 @@
     # it's the earliest or latest date in the database as appropriate.
     my $datefrom = $dbh->selectrow_array("SELECT MIN(series_date) " . 
                                          "FROM series_data " .
-                                         "WHERE series_id IN ($series_ids)");
+                                         "WHERE series_id IN ($series_ids)" .
+                                         "AND series_date > 0");
     $datefrom = &::str2time($datefrom);
 
     if ($self->{'datefrom'} && $self->{'datefrom'} > $datefrom) {

   

Reproducible: Didn't try

Steps to Reproduce:

Comment 1

[Gervase Markham [:gerv]]

Albert: the migration code is presumably not getting a valid date from your data
file. Do you know the rest of the data (i.e. the value and the series) for the
duff data point? Can you go back to the original data file in data/mining/ and
find out what the corrupted line looks like?

Gerv

Comment 2

[Albert Ting]

Sorry Gerv, I don't know how I got the bad data to start with.  It wasn't
obvious where the culprit was nor could I easily repeat it.

Maybe a sanity checker might be sufficient?

Comment 3

[Teemu Mannermaa (:wicked)]

I got this when I tried to view one of my bookmarked charts yesterday:

DBD::mysql::st execute failed: You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right syntax to use
near ')), series_value FROM series_data WHERE series_id = '101' AND series_date
>= FRO' at line 1 [for Statement "SELECT TO_DAYS(series_date) -
TO_DAYS(FROM_UNIXTIME()), series_value FROM series_data WHERE series_id = ? AND
series_date >= FROM_UNIXTIME() AND series_date <= FROM_UNIXTIME(1126904400)"] at
Bugzilla/Chart.pm line 259
	Bugzilla::Chart::readData('Bugzilla::Chart=HASH(0x9191a28)') called at
Bugzilla/Chart.pm line 200
	Bugzilla::Chart::data('Bugzilla::Chart=HASH(0x9191a28)') called at
Bugzilla/Chart.pm line 364
	Bugzilla::Chart::dump('Bugzilla::Chart=HASH(0x9191a28)') called at chart.cgi
line 272
	main::plot() called at chart.cgi line 119

Obviously FROM_UNIXTIME() is called with no date so the $datefrom seems to be
empty. I verified my series_data in DB and there are entries with series_date of
zero. So some collectstats run must have generated zero date and used that for
all entries it created. Values of that run seems to be correct and place this
happening sometime from 2005-02-08 to 2005-05-11. Collectstats is running
everyday at midnight and all days are counted for in that time period so really
don't know to which date these entries belong to.