234142 - new charts start from 1995 when date range is not set

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, defect)

Description

[mmc]

User-Agent:       
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8

In the new charting system ( http://bugzilla.mozilla.org/chart.cgi ), if a date
range is not set, the chart obtained start from 1995-11-10 for category Browser
(I queried the component Bugzilla and it starts from 1998-08-27).

Reproducible: Always
Steps to Reproduce:
1. Go to http://bugzilla.mozilla.org/chart.cgi
2. Choose Browser and any fields in Name.
3. Run it!
Actual Results:  
The graph starts from 1995, which means that there is a long flat line from 1995
to 1998.

Expected Results:  
Start charting from 1998.

Comment 1

[Dave Miller [:justdave]]

I'm pretty sure the Browser category is accurate.  Bugzilla existed at Netscape
for a couple years prior to Mozilla being open sourced in 1998.  Granted, very
little happened with it for a while. :)

If anything, the Bugzilla category is wrong...  it shouldn't start until 2002. 
Prior to that, Bugzilla was part of the Webtools category.

Comment 2

[Gervase Markham [:gerv]]

The way to check is to ssh into mecha and see what data is in the original
source text files in data\mining. If Dave doesn't get around to it, I'll do that
this weekend. But both these dates sound fairly plausible to me.

Gerv 

Comment 3

[Gervase Markham [:gerv]]

Both dates are accurate according to the data/mining files. However, it could be
that the --regenerate code has made a bit of a mistake, and/or there's some data
corruption somewhere. Because there appears to be only a single bug in the
database under Browser from 10th Nov 1995 to 28th March 1997. 

We are going to nuke and reimport that data quite soon - so it might be good to
make a note to check the original files for sanity first, and perhaps trim off
any dodgy stuff at the start.

Gerv

[gerv@mecha mining]$ head Bugzilla
# Bugzilla Daily Bug Stats
...
# Product: Bugzilla
...
19980827|1|0|0|0|0|0|0|0|0|0|0|0|0|0|0
[gerv@mecha mining]$ head Browser
# Bugzilla Daily Bug Stats
...
# Product: Browser
...
19951110|1|0|0|0|0|0|0|0|0|0|0|0|0|0|0
[gerv@mecha mining]$

Gerv

Comment 4

[Gervase Markham [:gerv]]

Reassigning to Dave, in case he wants to check the files for sanity when we
re-import them when b.m.o. next updates.

Gerv

Comment 5

[Gervase Markham [:gerv]]

Dave: is there a b.m.o. upgrade bug we can make depend on this?

Gerv

Comment 6

[Dave Miller [:justdave]]

yes.  alias bmo-upgrade conveniently enough ;)

Comment 7

[Dave Miller [:justdave]]

I'm assuming since the chart code re-import ended up not happening that we
likely need to fix this data by hand now.  Do I need to delete some rows from a
series data table as well as the old charts files or something?

Comment 8

[Gervase Markham [:gerv]]

Dave: yes. The procedure is:

- Edit the old charts files and remove all the rows up until 1997-03-28, if 
  present.
- Execute the following SQL:
  DELETE FROM series_data WHERE series_date < "1997-03-28 00:00:00";
- Close this bug :-)

Gerv

Comment 9

[Dave Miller [:justdave]]

mysql> DELETE FROM series_data WHERE series_date < "1997-03-28 00:00:00";
Query OK, 64336 rows affected (2.92 sec)

mysql> select count(*) from series_data;
+----------+
| count(*) |
+----------+
|   859841 |
+----------+
1 row in set (0.00 sec)

Done.