364697 - dataloss warning from UTF8 conversion is not precise/scary enough

Status: RESOLVED FIXED

(Bugzilla :: Installation & Upgrading, defect)

Description

[timeless]

WARNING: We are about to convert your table storage format to UTF8. This
          allows Bugzilla to correctly store and sort international characters.
          However, if you have any non-UTF-8 data in your database,
          it ***WILL BE DELETED*** by this process. So, before
          you continue with checksetup.pl, if you have any non-UTF-8
          data (or even if you're not sure) you should press Ctrl-C now
          to interrupt checksetup.pl, and run contrib/recode.pl to make all
          the data in your database into UTF-8. You should also back up your
          database before continuing.


I asked justdave what would happen if I had my own custom hacked table or column to the bugzilla database with an encoding that wasn't UTF8. would checksetup claim that it was going to convert it to UTF8 and not?

his answer was that it would indeed convert that, too

he discovered a dead table in our bugs database that way

My view is that: if it's going to do that, i think it should spell that out

specifically, i'd like it to list all the tables/columns that are affected. this way if I have a second application sharing the bugzilla database (yes i know, that's wrong, i'm describing behaviors I know other people will do, not something i myself would do), i'll realize it at this time instead of after the frier starts.

I asked justdave: how trivial is it for bugzilla to list all fields which it's converting, especially the ones which bugzilla doesn't think it owns. but it isn't doing it when it's warning. by the time it does the conversion, it's too late, your data is already in the frier

He indicates: it lists all of them off while it's converting them. It could probably be made to do a dry run pass and gather the names.

recode converts everything in the DB, too. that's the one that choked on my rogue table actually. because the rogue table had no primary key.
and recode needs a PK to be able to update the right row when it makes changes
even if it's just a multi-column unique index.

Expected results:
1. list all tables/fields that will be upgraded (hopefully someone will properly recognize and panic at this stage)
2. if any fit justdave's definition of not having a primary key warn that recode will fail * - this is really ideally covered in other bug, but i don't trust my web browser so i'm including it here

Comment 1

[Dave Miller [:justdave]]

Even if we're completely wrong and it does only do tables it knows about, it should explicitly say so just to ease fears.  I don't know about the UTF-8 storage conversion, but I know recode tried to update a table that didn't belong to Bugzilla when I ran it.  (That table got nuked because it didn't belong there).

Comment 2

[Max Kanat-Alexander]

Okay, so we have two options. One is to explain in the message that every table in the database will be converted. The other is to make it only convert tables that exist in the schema.

The reason that I made it convert all tables is that I wanted to get custom columns and tables that users had added. This is not an easy process to run manually, and it would be difficult for those customizers to do it.

So I suppose the best solution is to just add that piece of information to the message.

Comment 3

[Max Kanat-Alexander]

Attachment: v1

Yes, this *is* the most trivial blocker ever. :-)

Comment 4

[Max Kanat-Alexander]

Checking in Bugzilla/DB/Mysql.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Mysql.pm,v  <--  Mysql.pm
new revision: 1.49; previous revision: 1.48
done