Closed Bug 244756 Opened 20 years ago Closed 20 years ago

checksetup.pl fails to remove product index from milestones table in upgrade from 2.14.x

Categories

(Bugzilla :: Installation & Upgrading, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Bugzilla 2.18

People

(Reporter: bzadmin, Assigned: mkanat)

Details

Attachments

(2 files)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020604
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020604

I upgraded my BZ 2.14.5 to 2.17.7 and have two products (ABC, GHJ) under the
control of bugzilla. I created a new milestone 2.17.7 for product ABC. Then I
would create zhe same milestone for the product GHJ but BZ say

Error Message:
DBD::mysql::st execute failed: Duplicate entry '2.17.7' for key 1 [for
Statement "INSERT INTO milestones ( value, product_id, sortkey ) VALUES (
'2.17.7', 13, 0)"] at Bugzilla/DB.pm line 66
Bugzilla::DB::SendSQL('INSERT INTO milestones ( value, product_id, sortkey )
VALUES ...') called at /var/www/a-o/bugzilla-2.17.7/editmilestones.cgi line 311

The reason for this is checksetup.pl creates a new index for the milestones
table but doesn't delete the old primary index.

The old primary index was product and value. product isn't any longer used by BZ
2.17.7 and is always empty or null (?). When I create the second milestone with
the same value as the first the milestone exists already because the field
product is empty.

The new unique index is product_id and value

I could solve the problem on my box by deleting the old primary index from BZ 2.14.5

Reproducible: Always
Steps to Reproduce:
1.Install 2.14.5 and use 2 different products with the same or different milestones
2.Update BZ to Version 2.17.7
3.Create one (same value like 2.17.7) milestone for both products




Error Message:
DBD::mysql::st execute failed: Duplicate entry '2.17.7' for key 1 [for
Statement "INSERT INTO milestones ( value, product_id, sortkey ) VALUES (
'2.17.7', 13, 0)"] at Bugzilla/DB.pm line 66
Bugzilla::DB::SendSQL('INSERT INTO milestones ( value, product_id, sortkey )
VALUES ...') called at /var/www/a-o/bugzilla-2.17.7/editmilestones.cgi line 311
preliminarily setting this to blocking 2.18 until we can confirm it.  If this is
really happening, we'll want to fix this before release.
Flags: blocking2.18+
OS: Linux → All
Hardware: PC → All
Summary: primary db index after upgrade to much → checksetup.pl fails to remove product index from milestones table in upgrade from 2.14.x
Target Milestone: --- → Bugzilla 2.18
Hmm... Cannot reproduce this; checksetup also has the following:

<http://lxr.mozilla.org/mozilla/source/webtools/bugzilla/checksetup.pl#3257>
--
    print "Fixing Indexes and Uniqueness.\n";
    $dbh->do("ALTER TABLE milestones DROP INDEX product");
    $dbh->do("ALTER TABLE milestones ADD UNIQUE (product_id, value)");
...
    DropField("milestones", "product");
--

Seems to produce the perfect result for me. 

bzadmin@topitd.de: Do you have any additional information on this?
Hmmmh ... I have done a lot of retries this morning, but I am not able to
reproduce the error with a bugzilla 2.14.5 and 2.17.7. I asked my partner but he
could not give me further infos.

If I setup my database as I described before - the script checksetup.pl does not
run anymore.

But I could not reproduce the mistake by updating the 2.14.5 version. I have no
further idea to reproduce the error.

Sorry
Hy, I made a further update of another bugzilla installation and I have won some
more infomation about the error

Deleting unused field product from table milestones ...
[Mon Jun 21 08:58:05 2004] checksetup.pl: DBD::mysql::db do failed: Duplicate
entry '---' for key 1 at ./checksetup.pl line 2279.

The complete checksetup output:

./checksetup.pl

Checking perl modules ...
Checking for       AppConfig (v1.52)   ok: found v1.52
Checking for             CGI (v2.93)   ok: found v3.01
Checking for    Data::Dumper (any)     ok: found v2.121
Checking for    Date::Format (v2.21)   ok: found v2.21
Checking for             DBI (v1.32)   ok: found v1.41
Checking for      DBD::mysql (v2.1010) ok: found v2.9003
Checking for      File::Spec (v0.82)   ok: found v0.87
Checking for      File::Temp (any)     ok: found v0.14
Checking for        Template (v2.08)   ok: found v2.10
Checking for      Text::Wrap (v2001.0131) ok: found v2001.09291

The following Perl modules are optional:
Checking for              GD (v1.20)   ok: found v2.11
Checking for     Chart::Base (v0.99)   ok: found v2.2
Checking for     XML::Parser (any)     ok: found v2.34
Checking for       GD::Graph (any)     ok: found v1.43
Checking for GD::Text::Align (any)     ok: found v1.18
Checking for     PatchReader (any)      not found

If you want to see pretty HTML views of patches, you should install the
PatchReader module, which can be downloaded at:
http://search.cpan.org/CPAN/authors/id/J/JK/JKEISER/PatchReader-0.9.2.tar.gz
When you get it, do the following to install:
tar xzvf PatchReader-0.9.2.tar.gz
cd PatchReader-0.9.2
perl Makefile.PL
make install

Checking user setup ...

OPTIONAL NOTE: If you want to be able to use the
 'difference between two patches' feature of Bugzilla (requires
 the PatchReader Perl module as well), you should install
 patchutils from http://cyberelk.net/tim/patchutils/


This version of Bugzilla contains some variables that you may want
to change and adapt to your local settings. Please edit the file
'localconfig' and rerun checksetup.pl

The following variables are new to localconfig since you last ran
checksetup.pl:   index_html mysqlpath cvsbin interdiffbin diffpath db_sock



./checksetup.pl

Checking perl modules ...
Checking for       AppConfig (v1.52)   ok: found v1.52
Checking for             CGI (v2.93)   ok: found v3.01
Checking for    Data::Dumper (any)     ok: found v2.121
Checking for    Date::Format (v2.21)   ok: found v2.21
Checking for             DBI (v1.32)   ok: found v1.41
Checking for      DBD::mysql (v2.1010) ok: found v2.9003
Checking for      File::Spec (v0.82)   ok: found v0.87
Checking for      File::Temp (any)     ok: found v0.14
Checking for        Template (v2.08)   ok: found v2.10
Checking for      Text::Wrap (v2001.0131) ok: found v2001.09291

The following Perl modules are optional:
Checking for              GD (v1.20)   ok: found v2.11
Checking for     Chart::Base (v0.99)   ok: found v2.2
Checking for     XML::Parser (any)     ok: found v2.34
Checking for       GD::Graph (any)     ok: found v1.43
Checking for GD::Text::Align (any)     ok: found v1.18
Checking for     PatchReader (any)      not found

If you want to see pretty HTML views of patches, you should install the
PatchReader module, which can be downloaded at:
http://search.cpan.org/CPAN/authors/id/J/JK/JKEISER/PatchReader-0.9.2.tar.gz
When you get it, do the following to install:
tar xzvf PatchReader-0.9.2.tar.gz
cd PatchReader-0.9.2
perl Makefile.PL
make install

Checking user setup ...
Removing existing compiled templates ...
Precompiling templates ...
The following parameters are no longer used in Bugzilla, and so have been
removed from your parameters file and appended to old-params.txt:
nummilestones, mostfreqhtml, errorhtml, expectbigqueries, curmilestone,
queryagainstshadowdb, usedependencies, bodyhtml, despotbaseurl,
usebuggroupsentry, preferlists, usebuggroups, footerhtml, usedespot, bannerhtml,
useLDAP, capitalizelists, blurbhtml, entryheaderhtml, strictvaluechecks, headerhtml
Checking for    MySQL Server (v3.23.41) ok: found v3.23.49-log

Creating table user_group_map ...
Creating table series_data ...
Creating table flagtypes ...
Creating table series_categories ...
Creating table group_control_map ...
Creating table group_group_map ...
Creating table user_series_map ...
Creating table query_page ...
Creating table series ...
Creating table flagexclusions ...
Creating table bug_group_map ...
Creating table flags ...
Creating table flaginclusions ...
Creating table quips ...
Updating field resolution in table bugs ...
Adding new field isobsolete to table attachments ...
Adding new field attach_id to table bugs_activity ...
Removing invalid login cookies...
Deleting unused field cryptpassword from table logincookies ...
Removing restrictions on bugs for assignee and qacontact...
Deleting unused field qacontact_accessible from table bugs ...
Deleting unused field assignee_accessible from table bugs ...
Adding new field work_time to table longdescs ...
Adding new field estimated_time to table bugs ...
Adding new field remaining_time to table bugs ...
Adding new field isprivate to table longdescs ...
Adding new field isprivate to table attachments ...
Adding new field alias to table bugs ...
Deleting unused field watchfordiffs from table namedqueries ...
Updating database to use product IDs.
Adding new field id to table products ...
Adding new field product_id to table components ...
Adding new field product_id to table versions ...
Adding new field product_id to table milestones ...
Adding new field product_id to table bugs ...
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::st execute failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 2070.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::st fetchrow_arrayref
failed: fetch() without execute() at ./checksetup.pl line 2072.
Adding new field product_id to table attachstatusdefs ...
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 2267.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::st fetchrow_array failed:
Table 'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 3196.
Updating the database to use component IDs.
Adding new field id to table components ...
Adding new field component_id to table bugs ...
Fixing Indexes and Uniqueness.
[Mon Jun 21 08:58:04 2004] checksetup.pl: DBD::mysql::db do failed: Can't DROP
'product'. Check that column/key exists at ./checksetup.pl line 3222.
Removing, renaming, and retyping old product and component fields.
Deleting unused field program from table components ...
Deleting unused field program from table versions ...
Deleting unused field product from table milestones ...
[Mon Jun 21 08:58:05 2004] checksetup.pl: DBD::mysql::db do failed: Duplicate
entry '---' for key 1 at ./checksetup.pl line 2279.
Deleting unused field product from table bugs ...
Deleting unused field component from table bugs ...
[Mon Jun 21 08:58:06 2004] checksetup.pl: DBD::mysql::st execute failed: Table
'fit.attachstatusdefs' doesn't exist at ./checksetup.pl line 2070.
[Mon Jun 21 08:58:06 2004] checksetup.pl: DBD::mysql::st fetchrow_arrayref
failed: fetch() without execute() at ./checksetup.pl line 2072.
Updating field product in table products ...
Updating field type name in table products ...
old: varchar(64)
new: varchar(64) not null
Updating field value in table components ...
Updating field type name in table components ...
old: tinytext
new: varchar(64) not null
Adding indexes for products and components tables.
Fixing creation time on attachments...
Warning - could not determine correct creation time for attachment 2 [details] [diff] [review] on bug 7
Warning - could not determine correct creation time for attachment 3 [details] [diff] [review] on bug 8
Warning - could not determine correct creation time for attachment 4 [details] [diff] [review] on bug 11
[snip]
.....
[snap]
Warning - could not determine correct creation time for attachment 210 [details] [diff] [review]10 on bug 1652
Warning - could not determine correct creation time for attachment 211 [details] [diff] [review]11 on bug 1664
Warning - could not determine correct creation time for attachment 212 [details] [diff] [review]12 on bug 1673
Done - converted 212 attachments
Updating field type creation_ts in table attachments ...
old: timestamp(14)
new: datetime NOT NULL
Adding new field last_changed to table groups ...
Adding new field id to table groups ...
Adding new field refreshed_when to table profiles ...

WARNING - GROUPSET ACTIVITY ON BUG 928 CONTAINS DELETED GROUPS
 
WARNING - GROUPSET ACTIVITY ON BUG 929 CONTAINS DELETED GROUPS
 
WARNING - GROUPSET ACTIVITY ON BUG 930 CONTAINS DELETED GROUPS
 
WARNING - GROUPSET ACTIVITY ON BUG 931 CONTAINS DELETED GROUPS
Deleting unused field groupset from table profiles ...
Deleting unused field blessgroupset from table profiles ...
Deleting unused field groupset from table bugs ...
Deleting unused field bit from table groups ...
Migrating old chart data into database ...
Adding group admin ...
 
Looks like we don't have an administrator set up yet.  Either this is your
first time using Bugzilla, or your administrator's privileges might have accidently
been deleted.
Enter the e-mail address of the administrator: bzadmin@topitd.de
'bzadmin@topitd.de' already has an account.
Make this user the administrator? [Y/n] y
 
'stz.kienle@daimlerchrysler.com' is now set up as an administrator account.
Adding full-text index for short_desc column in bugs table...
Adding full-text index for thetext column in longdescs table...
Removing paths from filenames in attachments table...
Done.
Resizing attachments.filename from mediumtext to varchar(100).
Updating field type filename in table attachments ...
old: mediumtext not null
new: varchar(100) not null
Removing shadowlog table
Updating field count in table votes ...
Reminder: Bugzilla now requires version 8.7 or later of sendmail.
The actual error I could note on an update of bugzilla 2.14.5 with bz 2.17.6.
First time I saw the error during a update of bz 2.14.5 with bz 2.17.7
Version: unspecified → 2.17.6
i can reproduce hard the problem with our company data. i do some more retries
to figure out some details for the problem
company data corrupted

undef error - DBD::mysql::st execute failed: You have an error in your SQL
syntax near 'WHERE isbuggroup' at line 1 [for Statement "SELECT DISTINCT
groups.id, name, description, bug_group_map.group_id IS NOT NULL,
user_group_map.group_id IS NOT NULL, isactive, membercontrol, othercontrol FROM
groups LEFT JOIN bug_group_map ON bug_group_map.group_id = groups.id AND bug_id
= 71 LEFT JOIN user_group_map ON user_group_map.group_id = groups.id AND user_id
= 0 AND NOT isbless LEFT JOIN group_control_map ON group_control_map.group_id =
groups.id AND group_control_map.product_id = WHERE isbuggroup"] at
Bugzilla/DB.pm line 66 Bugzilla::DB::SendSQL('SELECT DISTINCT groups.id, name,
description, bug_group_map.g...') called at Bug.pm line 332
Bug::groups('Bug=HASH(0x875ecfc)') called at
data/template/en/custom/bug/edit.html.tmpl line 405 eval {...} called at
data/template/en/custom/bug/edit.html.tmpl line 405 eval {...} called at
data/template/en/custom/bug/edit.html.tmpl line 16
Template::Provider::__ANON__('Template::Context=HASH(0x865f340)') called at
/usr/lib/perl5/Template/Document.pm line 141 eval {...} called at
/usr/lib/perl5/Template/Document.pm line 139
Template::Document::process('Template::Document=HASH(0x8805264)','Template::Context=HASH(0x865f340)')
called at /usr/lib/perl5/Template/Context.pm line 340 eval {...} called at
/usr/lib/perl5/Template/Context.pm line 320
Template::Context::process('Template::Context=HASH(0x865f340)','bug/edit.html.tmpl')
called at data/template/en/custom/bug/show.html.tmpl line 68 eval {...} called
at data/template/en/custom/bug/show.html.tmpl line 16
Template::Provider::__ANON__('Template::Context=HASH(0x865f340)') called at
/usr/lib/perl5/Template/Document.pm line 141 eval {...} called at
/usr/lib/perl5/Template/Document.pm line 139
Template::Document::process('Template::Document=HASH(0x875ed98)','Template::Context=HASH(0x865f340)')
called at /usr/lib/perl5/Template/Context.pm line 340 eval {...} called at
/usr/lib/perl5/Template/Context.pm line 320
Template::Context::process('Template::Context=HASH(0x865f340)','Template::Document=HASH(0x875ed98)')
called at /usr/lib/perl5/Template/Service.pm line 97 eval {...} called at
/usr/lib/perl5/Template/Service.pm line 94
Template::Service::process('Template::Service=HASH(0x865d4cc)','bug/show.html.tmpl','HASH(0x8195074)')
called at /usr/lib/perl5/Template.pm line 76
Template::process('Bugzilla::Template=HASH(0x865d280)','bug/show.html.tmpl','HASH(0x8195074)')
called at /var/www/a-o/fit-test/show_bug.cgi line 115
I reproduced this on landfill using one of the older databases still sitting
around on there.  Tracked it down to the code that converts products from names
to IDs.  At the time that was checked in, attachment statuses were still around,
so it's converting the product field on the attachment status table to product_id.

Patch coming up.
Assignee: zach → justdave
Status: UNCONFIRMED → NEW
Ever confirmed: true
(In reply to comment #8)
> Patch coming up.

Errr...  sorry, that comment was meant for bug 245101.

I'll come back to this though.
I'm with Jouni (comment 2), I can't reproduce this upgrading an old 2.14.x
database from landfill to the tip.

Can you run "SHOW INDEX FROM milestones;" on the backup copy of your database
(from before the upgrade) and post it here?

Here's the output of that from landfill from before upgrading:

mysql> show index from milestones;
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
| Table      | Non_unique | Key_name | Seq_in_index | Column_name | Collation |
Cardinality | Sub_part | Packed | Comment |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
| milestones |          0 | product  |            1 | product     | A         |
       NULL |     NULL | NULL   |         |
| milestones |          0 | product  |            2 | value       | A         |
          1 |     NULL | NULL   |         |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
2 rows in set (0.00 sec)

and after:

mysql> show index from milestones;
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
| Table      | Non_unique | Key_name   | Seq_in_index | Column_name | Collation
| Cardinality | Sub_part | Packed | Comment |
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
| milestones |          0 | product_id |            1 | product_id  | A        
|        NULL |     NULL | NULL   |         |
| milestones |          0 | product_id |            2 | value       | A        
|           1 |     NULL | NULL   |         |
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
2 rows in set (0.00 sec)
I'm removing the blocking flag on this since we can't reproduce it.  If we get
enough evidence provided to point out what the problem actually is, we can still
get it in before release though.

Since we've had two people follow the existing steps to reproduce and not be
able to reproduce it, I will resolve this WORKSFORME in a week if there's
nothing additional posted which leads us to the problem.
Flags: blocking2.18+ → blocking2.18-
mysql> show index from milestones;
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
| Table      | Non_unique | Key_name | Seq_in_index | Column_name | Collation |
Cardinality | Sub_part | Packed | Comment |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
| milestones |          0 | PRIMARY  |            1 | product     | A         |
       NULL |     NULL | NULL   |         |
| milestones |          0 | PRIMARY  |            2 | value       | A         |
        118 |     NULL | NULL   |         |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+---------+
2 rows in set (0.00 sec)

after update and run of checksetup.pl

mysql> show index from milestones;
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
| Table      | Non_unique | Key_name   | Seq_in_index | Column_name | Collation
| Cardinality | Sub_part | Packed | Comment |
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
| milestones |          0 | PRIMARY    |            1 | product     | A        
|        NULL |     NULL | NULL   |         |
| milestones |          0 | PRIMARY    |            2 | value       | A        
|         118 |     NULL | NULL   |         |
| milestones |          0 | product_id |            1 | product_id  | A        
|        NULL |     NULL | NULL   |         |
| milestones |          0 | product_id |            2 | value       | A        
|         118 |     NULL | NULL   |         |
+------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+---------+
4 rows in set (0.00 sec)

my Milestones table after checksetup run

mysql> select * from milestones;
+--------------------+--------------------+---------+------------+
| value              | product            | sortkey | product_id |
+--------------------+--------------------+---------+------------+
| ---                | TestProduct - Auto |       0 |          1 |
| Beginn C2.2-Muster | ABC-TA             |       1 |          2 |
| Beginn D-Muster    | ABC-TA             |       2 |          2 |
| ---                | PRE-DUMP DCF 1.1   |       0 |          9 |
| 200204             | TestProduct - Auto |       5 |          1 |
| SWAb 251           | PRE-DUMP DCF 1.0   |      30 |          8 |

see attached file
defect milestone table after update 2.14.5 to 2.17.7
PRIMARY....  that's the difference...

hmmm. 

/me goes digging through cvs blame

okay, the milestones table has never *ever* had a "PRIMARY KEY" in the entire
history of Bugzilla.  Was that set up manually on your milestones table by any
chance?  Were there any local hacks on your system that might have caused it?

Anyone know if an older version of MySQL might have had some quirk that caused a
unique index on a table with no primary key to become the primary key?
We installed first time a BZ 2.14 (may 2001) and made a update upto BZ 2.14.5 
(february 2003). We made no changes at the database. the database structure is 
original BZ 2.14 updated with 2.14.5

There are no local hacks on our system that might have caused it. The only 
changes we have done are at the bugzilla code for the graphical user interface.
Hate to do this, but I can find no way to reproduce this, and haven't come up
with any good leads of any plausible way it might have happened, either.  Unless
someone else also reports it, I'll have to write this off as a fluke.
Status: NEW → RESOLVED
Closed: 20 years ago
Resolution: --- → WORKSFORME
Target Milestone: Bugzilla 2.18 → ---
I run into the same problem updating from Bugzilla 2.13 to 2.18rc2. When I 
edit milestones in 2.18rc2 the field product is NOT filled by bugzilla. So 
entering a second milestone (same value) to another product its´t possible 
(same error like in desciption to this bug).

Is the field product not longer in use by bugzilla and should be deleted by 
checksetup.pl ?

Here is our index of milestones in 2.18rc2:
mysql> show index from milestones;
+------------+------------+------------+--------------+-------------+----------
-+-------------+----------+--------+------+------------+---------+
| Table      | Non_unique | Key_name   | Seq_in_index | Column_name | 
Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+------------+--------------+-------------+----------
-+-------------+----------+--------+------+------------+---------+
| milestones |          0 | PRIMARY    |            1 | product     | 
A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| milestones |          0 | PRIMARY    |            2 | value       | 
A         |         657 |     NULL | NULL   |      | BTREE      |         |
| milestones |          0 | product_id |            1 | product_id  | 
A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| milestones |          0 | product_id |            2 | value       | 
A         |         657 |     NULL | NULL   |      | BTREE      |         |
+------------+------------+------------+--------------+-------------+----------
-+-------------+----------+--------+------+------------+---------+
4 rows in set (0.00 sec)

mysql>
in editmilestones.cgi I have found the sql-insert to the tabelle milestones:

 # Add the new milestone
   SendSQL("INSERT INTO milestones ( " .
           "value, product_id, sortkey" .
           " ) VALUES ( " .
           SqlQuote($milestone) . ", $product_id, $sortkey)");

The column product is not filled. 
Does the tabelle not have the column product any longer ? Has this changed 
from BZ 2.13 to 2.18 and checksetup.pl does not recognized this ?
Reopening this, as per discussion on the developer's list.

Further information to reproduce:

I'm doing all my upgrading on a second machine to be assured that everything is 
working well (customizations, etc.) before it goes live. To facilitate this, I 
copied the database over... and this seems to be the source of the problem.

On the original machine: (truncated on the right for readability)

mysql> show index from milestones;
+------------+------------+----------+--------------+-------------+-----------+
| Table      | Non_unique | Key_name | Seq_in_index | Column_name | Collation |
+------------+------------+----------+--------------+-------------+-----------+
| milestones |          0 | product  |            1 | product     | A         |
| milestones |          0 | product  |            2 | value       | A         |
+------------+------------+----------+--------------+-------------+-----------+
2 rows in set (0.01 sec)


Looks good, looks like it should work. In transferring to the other machine,
however (mysqldump -c -q), the output file shows this:


CREATE TABLE milestones (
  value varchar(20) NOT NULL default '',
  product varchar(64) NOT NULL default '',
  sortkey smallint(6) NOT NULL default '0',
  PRIMARY KEY  (product,value)                  <--- here is the problem
) TYPE=MyISAM;


The above sql creates this:

mysql> show index from milestones;
+------------+------------+----------+--------------+-------------+-----------+
| Table      | Non_unique | Key_name | Seq_in_index | Column_name | Collation |
+------------+------------+----------+--------------+-------------+-----------+
| milestones |          0 | PRIMARY  |            1 | product     | A         |
| milestones |          0 | PRIMARY  |            2 | value       | A         |
+------------+------------+----------+--------------+-------------+-----------+
2 rows in set (0.00 sec)

Trying to update this causes the errors in checksetup.pl as were described by 
the original poster. That it is a defect in mysqldump (which is probably used 
by a lot of Bugzilla admins for archiving/backup purposes) and not Bugzilla 
itself explains a lot about why it could not be reproduced.

Locally, I am running:

% mysqldump --version
mysqldump  Ver 8.13 Distrib 3.23.37, for hp-hpux11.00 (hppa2.0w)

I do not know if newer versions of mysqldump have the same issues or not, but 
this at least should allow it to be reproduced, and therefore fixed.

Dave: you'll probably want to re-set the "blocking2.18" flag.
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Target Milestone: --- → Bugzilla 2.18
Version: 2.17.6 → unspecified
yup, indeed.

Bug in mysqldump.  Yippee.  That certainly explains why we couldn't reproduce it
though.  We'll have to do something to detect which set of keys is there and
drop the correct ones.
Flags: blocking2.18- → blocking2.18+
Whiteboard: bug awaiting patch
Arrgh, this is nasty. So, the problem is that mysqldump makes tables with no PK
and a Unique index into having a PK.

Shane, for BZ, does this only happen to the milestones table? If so, we can use
some DBI functions to get the indexes, and then drop/re-create any ones that
call themselves PRIMARY.

Either that, or we could drop all indexes if we're doing an upgrade, and
re-create them... I'm not too sure about that, either. I'd have to look more
carefully at that part of checksetup...
Here is a VERY specific fix for this bug. Basically, we just drop any index
called "PRIMARY" that exists in the milestones table, but we suppress errors.
Assignee: justdave → mkanat
Status: REOPENED → ASSIGNED
Comment on attachment 168255 [details] [diff] [review]
Very specific fix for this bug

I figure I'll ask justdave for review, since this is a 2.18 blocker. Although
I'd also take a review from vladd, if you're around, vladd, and you feel
qualified... or from *anybody*, really. :-)
Attachment #168255 - Flags: review?(justdave)
(In reply to comment #21)
> Shane, for BZ, does this only happen to the milestones table?

In upgrading from 2.16.7 to 2.18rc3, I only noticed it happening on the 
Milestones table. This is not to say that it cannot happen elsewhere, only that 
it is happening *here* on *this* upgrade.

Your bugfix is a nicer version of what I had to do to get past this problem 
locally (I didn't worry about making the warning silent) so it should probably 
do the trick in this case.
Whiteboard: bug awaiting patch → patch awaiting review
Comment on attachment 168255 [details] [diff] [review]
Very specific fix for this bug

We might need/want to look at more general fixes in the long term. Since
mysqldump inserts in a lot of places keys that were not there originally, this
creates the opportunity for conflicts waiting to happen. We could make sure
that we index every table on a primary key, or drop all of them and create
them, like Max suggested.

However, for 2.18, I think this very specific fix will do.
Attachment #168255 - Flags: review+
Flags: approval?
Flags: approval2.18?
Whiteboard: patch awaiting review → patch awaiting approval
Flags: approval?
Flags: approval2.18?
Flags: approval2.18+
Flags: approval+
Whiteboard: patch awaiting approval → patch awaiting checkin
Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.317; previous revision: 1.316
done

Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.289.2.15; previous revision: 1.289.2.14
done
Status: ASSIGNED → RESOLVED
Closed: 20 years ago20 years ago
Resolution: --- → FIXED
Whiteboard: patch awaiting checkin
Attachment #168255 - Flags: review?(justdave)
QA Contact: matty_is_a_geek → default-qa
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: