343189 - checksetup.pl HASH reference error with 2.18.5 to 2.20.2/2.22 upgrade

Status: RESOLVED FIXED

(Bugzilla :: Installation & Upgrading, defect)

Description

[bbillings]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4

     I'm attempting to update our old version of Bugzilla 2.16.4 on an ancient Red Hat box to a new system running Gentoo Linux, MySQL being used on both systems.  I'm taking an export of the database on the 2.16.4 system, doing an import on the new system into a database of the same name, but I get an error if I attempt to upgrade Bugzilla to 2.20.2 or 2.22.  I've tried using both MySQL 4.1.20 and 5.0.22 with no change in the outcome.  The only upgrade I can seem to get working is going from 2.16.4 to 2.18.5.  Anything beyond that and the checksetup.pl script dies in the same spot.  I've done some Google digging but I haven't seen anyone else with this problem.

Here's the output of my checksetup.pl script coming from 2.18.5:

Checking perl modules ...r/www/bugzilla/htdocs/bugzilla/localconfig
Checking for       AppConfig (v1.52)   ok: found v1.56
Checking for             CGI (v2.93)   ok: found v3.20
Checking for    Data::Dumper (any)     ok: found v2.121_08
Checking for    Date::Format (v2.21)   ok: found v2.22
Checking for             DBI (v1.38)   ok: found v1.51
Checking for      File::Spec (v0.84)   ok: found v3.18
Checking for      File::Temp (any)     ok: found v0.16
Checking for        Template (v2.08)   ok: found v2.15
Checking for      Text::Wrap (v2001.0131) ok: found v2005.0824
Checking for    Mail::Mailer (v1.65)   ok: found v1.74
Checking for        Storable (any)     ok: found v2.15

The following Perl modules are optional:
Checking for              GD (v1.20)   ok: found v2.34
Checking for     Chart::Base (v1.0)    ok: found v2.3
Checking for     XML::Parser (any)     ok: found v2.34
Checking for       GD::Graph (any)     ok: found v1.4308
Checking for GD::Text::Align (any)     ok: found v1.18
Checking for     PatchReader (v0.9.4)  ok: found v0.9.5

Checking user setup ...
Creating data directory (./data) ...
Creating local attachments directory ...
Creating graphs directory...
Repairing .htaccess...
Creating ./data/attachments/.htaccess...
Creating Bugzilla/.htaccess...
Creating ./data/.htaccess...
Creating ./template/.htaccess...
Creating ./data/webdot/.htaccess...
Removing existing compiled templates ...
Precompiling templates ...
Checking for      DBD::mysql (v2.9003) ok: found v3.0002
Checking for           MySQL (v3.23.41) ok: found v4.1.20-log

Building Schema object from database...
Adding new table bz_schema ...
Initializing the new Schema storage...
Adding new table bug_severity ...
Adding new table bug_status ...
Adding new table classifications ...
Adding new table email_setting ...
Adding new table op_sys ...
Adding new table priority ...
Adding new table profile_setting ...
Adding new table rep_platform ...
Adding new table resolution ...
Adding new table setting ...
Adding new table setting_value ...
Adding new table whine_events ...
Adding new table whine_queries ...
Adding new table whine_schedules ...
Updating column lastused in table logincookies ...
Old: TIMESTAMP DEFAULT 'CURRENT_TIMESTAMP'
New: datetime NOT NULL
Updating column delta_ts in table bugs ...
Old: TIMESTAMP DEFAULT 'CURRENT_TIMESTAMP'
New: datetime NOT NULL
SQL fragment found in the 'fielddefs' table...
Old field name: (to_days(now()) - to_days(bugs.delta_ts))
Fixing the 'fielddefs' table...
New field name: days_elapsed
Inserting value 'P1' in table priority with sortkey 100...
Inserting value 'P2' in table priority with sortkey 200...
Inserting value 'P3' in table priority with sortkey 300...
Inserting value 'P4' in table priority with sortkey 400...
Inserting value 'P5' in table priority with sortkey 500...
Inserting value 'UNCONFIRMED' in table bug_status with sortkey 100...
Inserting value 'NEW' in table bug_status with sortkey 200...
Inserting value 'ASSIGNED' in table bug_status with sortkey 300...
Inserting value 'REOPENED' in table bug_status with sortkey 400...
Inserting value 'RESOLVED' in table bug_status with sortkey 500...
Inserting value 'VERIFIED' in table bug_status with sortkey 600...
Inserting value 'CLOSED' in table bug_status with sortkey 700...
Inserting value 'All' in table rep_platform with sortkey 100...
Inserting value 'DEC' in table rep_platform with sortkey 200...
Inserting value 'HP' in table rep_platform with sortkey 300...
Inserting value 'Macintosh' in table rep_platform with sortkey 400...
Inserting value 'PC' in table rep_platform with sortkey 500...
Inserting value 'SGI' in table rep_platform with sortkey 600...
Inserting value 'Sun' in table rep_platform with sortkey 700...
Inserting value 'Other' in table rep_platform with sortkey 800...
Inserting value '' in table resolution with sortkey 100...
Inserting value 'FIXED' in table resolution with sortkey 200...
Inserting value 'INVALID' in table resolution with sortkey 300...
Inserting value 'WONTFIX' in table resolution with sortkey 400...
Inserting value 'LATER' in table resolution with sortkey 500...
Inserting value 'REMIND' in table resolution with sortkey 600...
Inserting value 'DUPLICATE' in table resolution with sortkey 700...
Inserting value 'WORKSFORME' in table resolution with sortkey 800...
Inserting value 'MOVED' in table resolution with sortkey 900...
Inserting value 'blocker' in table bug_severity with sortkey 100...
Inserting value 'critical' in table bug_severity with sortkey 200...
Inserting value 'major' in table bug_severity with sortkey 300...
Inserting value 'normal' in table bug_severity with sortkey 400...
Inserting value 'minor' in table bug_severity with sortkey 500...
Inserting value 'trivial' in table bug_severity with sortkey 600...
Inserting value 'enhancement' in table bug_severity with sortkey 700...
Inserting value 'All' in table op_sys with sortkey 100...
Inserting value 'Windows 3.1' in table op_sys with sortkey 200...
Inserting value 'Windows 95' in table op_sys with sortkey 300...
Inserting value 'Windows 98' in table op_sys with sortkey 400...
Inserting value 'Windows ME' in table op_sys with sortkey 500...
Inserting value 'Windows 2000' in table op_sys with sortkey 600...
Inserting value 'Windows NT' in table op_sys with sortkey 700...
Inserting value 'Windows XP' in table op_sys with sortkey 800...
Inserting value 'Windows Server 2003' in table op_sys with sortkey 900...
Inserting value 'Mac System 7' in table op_sys with sortkey 1000...
Inserting value 'Mac System 7.5' in table op_sys with sortkey 1100...
Inserting value 'Mac System 7.6.1' in table op_sys with sortkey 1200...
Inserting value 'Mac System 8.0' in table op_sys with sortkey 1300...
Inserting value 'Mac System 8.5' in table op_sys with sortkey 1400...
Inserting value 'Mac System 8.6' in table op_sys with sortkey 1500...
Inserting value 'Mac System 9.x' in table op_sys with sortkey 1600...
Inserting value 'Mac OS X 10.0' in table op_sys with sortkey 1700...
Inserting value 'Mac OS X 10.1' in table op_sys with sortkey 1800...
Inserting value 'Mac OS X 10.2' in table op_sys with sortkey 1900...
Inserting value 'Mac OS X 10.3' in table op_sys with sortkey 2000...
Inserting value 'Linux' in table op_sys with sortkey 2100...
Inserting value 'BSD/OS' in table op_sys with sortkey 2200...
Inserting value 'FreeBSD' in table op_sys with sortkey 2300...
Inserting value 'NetBSD' in table op_sys with sortkey 2400...
Inserting value 'OpenBSD' in table op_sys with sortkey 2500...
Inserting value 'AIX' in table op_sys with sortkey 2600...
Inserting value 'BeOS' in table op_sys with sortkey 2700...
Inserting value 'HP-UX' in table op_sys with sortkey 2800...
Inserting value 'IRIX' in table op_sys with sortkey 2900...
Inserting value 'Neutrino' in table op_sys with sortkey 3000...
Inserting value 'OpenVMS' in table op_sys with sortkey 3100...
Inserting value 'OS/2' in table op_sys with sortkey 3200...
Inserting value 'OSF/1' in table op_sys with sortkey 3300...
Inserting value 'Solaris' in table op_sys with sortkey 3400...
Inserting value 'SunOS' in table op_sys with sortkey 3500...
Inserting value 'other' in table op_sys with sortkey 3600...
Can't use an undefined value as a HASH reference at /var/www/bugzilla/htdocs/bugzilla/checksetup.pl line 2212

     Additional attempts to run the checksetup.pl script give near the same results:

Checking perl modules ...
Checking for       AppConfig (v1.52)   ok: found v1.56
Checking for             CGI (v2.93)   ok: found v3.20
Checking for    Data::Dumper (any)     ok: found v2.121_08
Checking for    Date::Format (v2.21)   ok: found v2.22
Checking for             DBI (v1.38)   ok: found v1.51
Checking for      File::Spec (v0.84)   ok: found v3.18
Checking for      File::Temp (any)     ok: found v0.16
Checking for        Template (v2.08)   ok: found v2.15
Checking for      Text::Wrap (v2001.0131) ok: found v2005.0824
Checking for    Mail::Mailer (v1.67)   ok: found v1.74
Checking for    MIME::Base64 (v3.01)   ok: found v3.07
Checking for    MIME::Parser (v5.406)  ok: found v5.420
Checking for        Storable (any)     ok: found v2.15

The following Perl modules are optional:
Checking for              GD (v1.20)   ok: found v2.34
Checking for     Chart::Base (v1.0)    ok: found v2.3
Checking for       XML::Twig (any)     ok: found v3.25
Checking for       GD::Graph (any)     ok: found v1.4308
Checking for GD::Text::Align (any)     ok: found v1.18
Checking for     PatchReader (v0.9.4)  ok: found v0.9.5
Checking for   Image::Magick (any)     ok: found v6.2.8

Checking user setup ...
Repairing .htaccess...
Removing existing compiled templates ...
Precompiling templates ...
Checking for      DBD::mysql (v2.9003) ok: found v3.0004
Checking for           MySQL (v4.0.14) ok: found v4.1.20-log
Checking for        GraphViz (any)     ok: found

<h1>Software error:</h1>
<pre>Can't use an undefined value as a HASH reference at ./checksetup.pl line 2234.
</pre>
<p>
For help, please send mail to this site's webmaster, giving this error message
and the time and date of the error.

</p>
[Thu Jun 29 13:50:45 2006] checksetup.pl: Can't use an undefined value as a HASH reference at ./checksetup.pl line 2234.


The homepage loads, but you cannot perform searches:

DBD::mysql::st execute failed: Unknown column 'products.classification_id' in 'where clause' [for Statement "SELECT products.name, classifications.name FROM products, classifications WHERE classifications.id = products.classification_id ORDER BY classifications.name"] at Bugzilla/DB.pm line 84
	Bugzilla::DB::SendSQL('SELECT products.name, classifications.name FROM products, cla...') called at globals.pl line 140
	main::GenerateVersionTable() called at globals.pl line 312
	main::GetVersionTable() called at /var/www/bugzilla/htdocs/bugzilla/query.cgi line 218

or log into the database:

DBD::mysql::st execute failed: Unknown column 'grant_type' in 'where clause' [for Statement "SELECT groups.name, groups.id
                            FROM group_group_map
                      INNER JOIN groups
                              ON groups.id = grantor_id
                           WHERE member_id = ? 
                             AND grant_type = 0" with ParamValues: 0='6'] at Bugzilla/User.pm line 290
	Bugzilla::User::groups('Bugzilla::User=HASH(<REMOVED>)') called at Bugzilla/User.pm line 361
	Bugzilla::User::in_group('Bugzilla::User=HASH(<REMOVED>)', 'bz_sudoers') called at Bugzilla.pm line 176
	Bugzilla::login('Bugzilla', 0) called at /var/www/bugzilla/htdocs/bugzilla/index.cgi line 37

...presumably from missing database upgrade steps that would be done by the checksetup.pl script.

Reproducible: Always

Steps to Reproduce:
1. Install 2.22/2.20.2 files over 2.18.5 files
2. Run checksetup.pl against 2.18.5 database, upgraded from 2.16.4
3. Error occurs




Server is a Gentoo Linux 2006.0 install running the 2.6.17 "vanilla" kernel.

Comment 1

[Max Kanat-Alexander]

Yeah, this technically is caused by somebody modifying your database at some point, and removing the index on profiles.name. That should never happen, so checksetup doesn't expect it. But it's easy to patch.

Comment 2

[Max Kanat-Alexander]

Attachment: v1

Here you go, reporter. You should be able to apply this patch and have checksetup work properly.

Remember that you need to start fresh from your 2.18.5 database, not from a broken DB.

Comment 3

[bbillings]

Much appreciated, that did the trick.  I ran into the same error on line 2468 for the next run through, but now that I know the fix, I'm going through the .pl script and making the appropriate changes as the Perl script "finds" lost indexes.  Thanks!

Comment 4

[Max Kanat-Alexander]

Hey. Glad that it fixed it! But we don't mark it as FIXED until we've checked in the patch to our code.

If you run into any other "missing indexes," checksetup should still probably be able to deal with those. Feel free to file bugs on them.

Comment 5

[bbillings]

Whoops..Sorry about that, force of habbit.

Do you want a separate bug list for each instance referencing this one, added to this bug, or one new bug with all of them in it also referencing this bug?

Comment 6

[Max Kanat-Alexander]

Well, if there are more than two more of them, just make one bug. If there are just two more, file a bug for each one. We may not be able to fix them, but I'd at least like to know where Bugzilla dies on that.

Comment 7

[Olav Vitters]

Comment on attachment 227631 [details] [diff] [review]
v1

r=bkor

Obviously correct, but also tested that it fixes the case when someone does:
alter table profiles drop index login_name on a 2.18 installation;

Comment 8

[Max Kanat-Alexander]

Tip:

Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.502; previous revision: 1.501
done

2.22:

Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.469.2.9; previous revision: 1.469.2.8
done