1100989 - Convert current checksetup upgrade test to no longer need database dumps and to auto-generate testing databases

Status: NEW

(Bugzilla :: QA Test Scripts, defect)

Description

[David Lawrence [:dkl]]

In the old way of doing upgrade tests for commits to bugzilla/trunk, database dumps were used from landfill.bugzilla.org for each version and then an upgrade was performed. 

We can no longer use database dumps from testing systems used by the general public and need to systematically generate the test databases for our upgrade tests. 

Also, Travis CI has a limit on how big the databases can be in a single worker process so we need to delete each version being upgraded after the test has completed before creating the new one.

dkl

Comment 1

[Mark Côté [:mcote]]

I don't have access to set the blocking5.0 flag, but it should be plussed. :)

Comment 2

[David Lawrence [:dkl]]

Setting as blocker per mcotes comment.

Comment 3

[Frédéric Buclin]

It shouldn't be too hard to generate data for each major release. We already do it for Selenium scripts.

Comment 4

[Mark Côté [:mcote]]

This is going to take too long for 5.0.  We should just do manual upgrade tests before releasing.

Comment 5

[Mark Côté [:mcote]]

In other words, I am reversing my decision in comment 1. :)

Comment 6

[David Lawrence [:dkl]]

I managed to get MySQL dumps for most stable versions back to 2.14 and here is the results of my upgrade testing:

mysql-community-server-5.6.24-3.el7.x86_64

2.14 -> 5.0 FAIL

Updating column userregexp in table groups ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
DBD::mysql::db do failed: BLOB/TEXT column 'userregexp' can't have a default value [for Statement "ALTER TABLE groups ALTER COLUMN userregexp
							   SET DEFAULT ''"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x68bc5a8)', 'groups', 'userregexp', 'HASH(0x6a52a20)', 'HASH(0x6b273f0)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x68bc5a8)', 'groups', 'userregexp', 'HASH(0x6a52a20)') called at Bugzilla/Install/DB.pm line 432
	Bugzilla::Install::DB::update_table_definitions('HASH(0x2921b80)') called at ./checksetup.pl line 170

2.16.11 -> 5.0 FAIL

Updating column userregexp in table groups ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
DBD::mysql::db do failed: BLOB/TEXT column 'userregexp' can't have a default value [for Statement "ALTER TABLE groups ALTER COLUMN userregexp
							   SET DEFAULT ''"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x66abf88)', 'groups', 'userregexp', 'HASH(0x9c7a978)', 'HASH(0x9be77e8)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x66abf88)', 'groups', 'userregexp', 'HASH(0x9c7a978)') called at Bugzilla/Install/DB.pm line 432
	Bugzilla::Install::DB::update_table_definitions('HASH(0x2711bb0)') called at ./checksetup.pl line 170

2.18.6 -> 5.0 FAIL

Updating column userregexp in table groups ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
DBD::mysql::db do failed: BLOB/TEXT column 'userregexp' can't have a default value [for Statement "ALTER TABLE groups ALTER COLUMN userregexp
							   SET DEFAULT ''"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x65ef498)', 'groups', 'userregexp', 'HASH(0x68c1b90)', 'HASH(0x677b230)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x65ef498)', 'groups', 'userregexp', 'HASH(0x68c1b90)') called at Bugzilla/Install/DB.pm line 432
	Bugzilla::Install::DB::update_table_definitions('HASH(0x2654b80)') called at ./checksetup.pl line 170

2.20.7 -> 5.0 FAIL

Updating column userregexp in table groups ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
DBD::mysql::db do failed: BLOB/TEXT column 'userregexp' can't have a default value [for Statement "ALTER TABLE groups ALTER COLUMN userregexp
							   SET DEFAULT ''"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x584e5a8)', 'groups', 'userregexp', 'HASH(0x5b7e8f0)', 'HASH(0x5cfbed0)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x584e5a8)', 'groups', 'userregexp', 'HASH(0x5b7e8f0)') called at Bugzilla/Install/DB.pm line 432
	Bugzilla::Install::DB::update_table_definitions('HASH(0x18b3b80)') called at ./checksetup.pl line 170

2.22.7 -> 5.0 FAIL

Updating column milestoneurl in table products ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
DBD::mysql::db do failed: BLOB/TEXT column 'milestoneurl' can't have a default value [for Statement "ALTER TABLE products ALTER COLUMN milestoneurl
							   SET DEFAULT ''"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x63f0498)', 'products', 'milestoneurl', 'HASH(0x98d2f48)', 'HASH(0x98d2cf0)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x63f0498)', 'products', 'milestoneurl', 'HASH(0x98d2f48)') called at Bugzilla/Install/DB.pm line 457
	Bugzilla::Install::DB::update_table_definitions('HASH(0x2455b80)') called at ./checksetup.pl line 170

3.0.11 -> 5.0 PASS
3.2.10 -> 5.0 PASS

3.4.14 -> 5.0 FAIL

Updating column setter_id in table flags ...
Old: mediumint
New: mediumint NOT NULL
DBD::mysql::db do failed: Cannot change column 'setter_id': used in a foreign key constraint 'fk_flags_setter_id_profiles_userid' [for Statement "ALTER TABLE flags CHANGE COLUMN 
					   setter_id setter_id mediumint NOT NULL"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x574c5e8)', 'flags', 'setter_id', 'HASH(0x5a94220)', 'HASH(0x5a66db0)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x574c5e8)', 'flags', 'setter_id', 'HASH(0x5a94220)') called at Bugzilla/Install/DB.pm line 627
	Bugzilla::Install::DB::update_table_definitions('HASH(0x17b1b80)') called at ./checksetup.pl line 170

3.6.13 -> 5.0 FAIL

Updating column setter_id in table flags ...
Old: mediumint
New: mediumint NOT NULL
DBD::mysql::db do failed: Cannot change column 'setter_id': used in a foreign key constraint 'fk_flags_setter_id_profiles_userid' [for Statement "ALTER TABLE flags CHANGE COLUMN 
					   setter_id setter_id mediumint NOT NULL"] at Bugzilla/DB.pm line 713.
	Bugzilla::DB::bz_alter_column_raw('Bugzilla::DB::Mysql=HASH(0x696c068)', 'flags', 'setter_id', 'HASH(0x9f0db58)', 'HASH(0x6d35cf8)', undef) called at Bugzilla/DB.pm line 672
	Bugzilla::DB::bz_alter_column('Bugzilla::DB::Mysql=HASH(0x696c068)', 'flags', 'setter_id', 'HASH(0x9f0db58)') called at Bugzilla/Install/DB.pm line 627
	Bugzilla::Install::DB::update_table_definitions('HASH(0x29d1bb0)') called at ./checksetup.pl line 170

4.0.18 -> 5.0 PASS
4.2.14 -> 5.0 PASS
4.4.9  -> 5.0 PASS

The BLOB/TEXT columns unable to have a default value seems to be due to the new version of MySQL than when these versions were used so not sure we need to worry with those. The flags.setter error is alleviated if I first drop the constraint, make the NOT NULL change and then add the foreign key constraint back. Not sure how we can fix that in the code systematically.

LpSolit, any ideas?

dkl

Comment 7

[Frédéric Buclin]

Updating column setter_id in table flags ...
Old: mediumint
New: mediumint NOT NULL


I cannot reproduce this issue. I upgraded from 3.6 to master without any problem (with bug 1169767 fixed).

I didn't investigate the "BLOB/TEXT column 'userregexp' can't have a default value" problem yet.

Comment 8

[Frédéric Buclin]

(In reply to Frédéric Buclin from comment #7)
> Updating column setter_id in table flags ...
> Old: mediumint
> New: mediumint NOT NULL
> 
> 
> I cannot reproduce this issue. I upgraded from 3.6 to master without any
> problem (with bug 1169767 fixed).

I also cannot reproduce when upgrading from 3.6 to 5.0.