It often happens that people want to switch from their current bug-tracker to Bugzilla. It would be nice to have a contrib/ script that understood various different other systems and could do an initial copy of another system into Bugzilla, for migration purposes.
Created attachment 406140 [details] [diff] [review] Work In Progress So far, this only touches installation code, so after completion and testing, I plan to check this in as a module owner (without review). It's way too huge for anybody to have time to review, anyhow, and I can't imagine how you would test a GNATS migration (whereas I can do it pretty well, and I have a client who wants it and can verify whether or not it was done correctly).
Created attachment 406152 [details] [diff] [review] WIP 2 Further work--this now parses the history of a GNATS bug.
Created attachment 406183 [details] [diff] [review] WIP 3 The architecture's starting to fall into place a bit more. This reads users and bugs, now.
Created attachment 406377 [details] [diff] [review] WIP 4 This now fully reads bugs, users, and products (though not attachments yet). It also fully creates users and products (though there are some problems with bugs as of yet).
Created attachment 406404 [details] [diff] [review] WIP 5 This version currently is importing all bugs from my small test (18 bugs in two products), though it isn't yet handling attachments.
Created attachment 406566 [details] [diff] [review] v1 This version is essentially complete, importing everything from the other bug-tracker and implementing all the options that I plan to. However, it needs more testing. Also, I've discovered that running it under --dry-run increments the AUTO_INCREMENT counter on tables, because even if we rollback a transaction, we still get AUTO_INCREMENT updated. :-( So I'm going to have to write some DB methods to fix that.
Created attachment 406616 [details] [diff] [review] v2 A bunch of bug-fixes over v1.
(In reply to comment #6) > Also, I've discovered that running it under --dry-run increments the > AUTO_INCREMENT counter on tables, because even if we rollback a transaction, we > still get AUTO_INCREMENT updated. :-( So I'm going to have to write some DB > methods to fix that. This can't be done without dangerous cheating. In Oracle, SEQUENCE values never get back if you rollback a transaction. Surely you're not trying to hide dry-run attempts?
It can be done relatively non-dangerously, after the transaction is rolled back. At least, in MySQL and PostgreSQL it can. I assume there's a similar method in Oracle. The idea is that dry-run attempts shouldn't modify the database AT ALL.
Created attachment 406631 [details] [diff] [review] v3 This version fully imports the GNATS database that I am testing on.
Created attachment 406637 [details] [diff] [review] v4 And this version also does the rollback of the auto_increment values. xiaoou--I need an implementation of get_set_serial_sql (which you can see used and defined for other DBs in this patch) for Oracle.
CC'ing mockodin so he can know about the new Bugzilla::DB method I added.
Created attachment 406638 [details] [diff] [review] v5 And this version passes runtests, and can be checked in once the blockers are resolved.
Holding approval until blockers are resolved.
(In reply to comment #9) > It can be done relatively non-dangerously, after the transaction is rolled > back. At least, in MySQL and PostgreSQL it can. I assume there's a similar > method in Oracle. Are you assuming (a) database being in exclusive mode during migrate.pl, or (b) dry-run can be done using read transactions only, without UPDATEs rolled back?
Created attachment 406643 [details] [diff] [review] get_set_serial_sql for oracle v1
(In reply to comment #15) > Are you assuming (a) database being in exclusive mode during migrate.pl, or (b) > dry-run can be done using read transactions only, without UPDATEs rolled back? The way that dry-run works is that it rolls back the transaction and *then* resets the serial counters to MAX(column) + 1. I assume that the database isn't under heavy production load when running under dry-run. And I do mention that --dry-run shouldn't be run on a live database.
Comment on attachment 406643 [details] [diff] [review] get_set_serial_sql for oracle v1 xiaoou: This looks like it will increment the sequence by $value, instead of setting the next value to $value. Am I missing something?
Comment on attachment 406643 [details] [diff] [review] get_set_serial_sql for oracle v1 The Oracle docs say: "To restart the sequence at a different number, you must drop and re-create it." Is that what we should be doing here instead?
Created attachment 406849 [details] [diff] [review] v6 I made some improvements to the GNATS migrator in terms of version number and username parsing.
Created attachment 406855 [details] [diff] [review] v7 This version uses less memory during import and has some improvements in attachment parsing for GNATS.
Created attachment 406977 [details] [diff] [review] v8 This version introduces the ability to set resolutions for particular translated statuses, and moves more code up into the generic framework from the GNATS migrator.
Created attachment 406980 [details] [diff] [review] v9 Fix a few bugs in the GNATS migrator.
Created attachment 406981 [details] [diff] [review] v9.1 The original v9 I attached had extra local changes in it.
Created attachment 407231 [details] [diff] [review] v10 Improve the version parsing of the GNATS migrator, and make value-mapping case-insensitive (because Bugzilla is case-insensitive about field values anyway).
Comment on attachment 406643 [details] [diff] [review] get_set_serial_sql for oracle v1 This doesn't look like it's setting the sequence's value, it looks like it's incrementing the sequence by the value passed in, which is not what I wanted.
Created attachment 407489 [details] [diff] [review] get_set_serial_sql for oracle v2 Yeah, v1 was assuming that the current sequence is 0. This v2 will set the sequence value.
Checking in Bugzilla.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla.pm,v <-- Bugzilla.pm new revision: 1.80; previous revision: 1.79 done RCS file: /cvsroot/mozilla/webtools/bugzilla/migrate.pl,v done Checking in migrate.pl; /cvsroot/mozilla/webtools/bugzilla/migrate.pl,v <-- migrate.pl initial revision: 1.1 done Checking in Bugzilla/DB.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB.pm,v <-- DB.pm new revision: 1.129; previous revision: 1.128 done RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Migrate.pm,v done Checking in Bugzilla/Migrate.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Migrate.pm,v <-- Migrate.pm initial revision: 1.1 done Checking in Bugzilla/DB/Schema/Mysql.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Schema/Mysql.pm,v <-- Mysql.pm new revision: 1.23; previous revision: 1.22 done Checking in Bugzilla/DB/Schema/Pg.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Schema/Pg.pm,v <-- Pg.pm new revision: 1.16; previous revision: 1.15 done Checking in Bugzilla/Install/Filesystem.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Filesystem.pm,v <-- Filesystem.pm new revision: 1.40; previous revision: 1.39 done Checking in Bugzilla/Install/Requirements.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v <-- Requirements.pm new revision: 1.72; previous revision: 1.71 done RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Migrate/Gnats.pm,v done Checking in Bugzilla/Migrate/Gnats.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Migrate/Gnats.pm,v <-- Gnats.pm initial revision: 1.1 done Checking in contrib/bzdbcopy.pl; /cvsroot/mozilla/webtools/bugzilla/contrib/bzdbcopy.pl,v <-- bzdbcopy.pl new revision: 1.9; previous revision: 1.8 done Removing contrib/gnats2bz.pl; /cvsroot/mozilla/webtools/bugzilla/contrib/gnats2bz.pl,v <-- gnats2bz.pl new revision: delete; previous revision: 1.10 done Removing contrib/gnatsparse/README; /cvsroot/mozilla/webtools/bugzilla/contrib/gnatsparse/README,v <-- README new revision: delete; previous revision: 1.2 done Removing contrib/gnatsparse/gnatsparse.py; /cvsroot/mozilla/webtools/bugzilla/contrib/gnatsparse/gnatsparse.py,v <-- gnatsparse.py new revision: delete; previous revision: 1.6 done Removing contrib/gnatsparse/magic.py; /cvsroot/mozilla/webtools/bugzilla/contrib/gnatsparse/magic.py,v <-- magic.py new revision: delete; previous revision: 1.1 done Removing contrib/gnatsparse/specialuu.py; /cvsroot/mozilla/webtools/bugzilla/contrib/gnatsparse/specialuu.py,v <-- specialuu.py new revision: delete; previous revision: 1.1 done Checking in template/en/default/global/messages.html.tmpl; /cvsroot/mozilla/webtools/bugzilla/template/en/default/global/messages.html.tmpl,v <-- messages.html.tmpl new revision: 1.93; previous revision: 1.92 done Checking in template/en/default/global/user-error.html.tmpl; /cvsroot/mozilla/webtools/bugzilla/template/en/default/global/user-error.html.tmpl,v <-- user-error.html.tmpl new revision: 1.287; previous revision: 1.286 done Checking in template/en/default/setup/strings.txt.pl; /cvsroot/mozilla/webtools/bugzilla/template/en/default/setup/strings.txt.pl,v <-- strings.txt.pl new revision: 1.17; previous revision: 1.16 done
And Xiaoou's patch: Checking in Bugzilla/DB/Schema/Oracle.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Schema/Oracle.pm,v <-- Oracle.pm new revision: 1.12; previous revision: 1.11 done
Added to the release notes in bug 547466.