244913 - Ability to combine (merge) two different Bugzilla installations

Status: NEW

(Bugzilla :: Bug Import/Export & Moving, enhancement, P2)

Description

[Joel Peshkin]

When 2 distinct installations need to be combined, there are a number of steps
that must be taken...

1) Make a temp copy of the source bugzilla
2) for each user in the source bugzilla, see if the same user exists in the
target and either update the source's userid or create the user in the target
and update the sources userid.  After updating the source's userid, change the
userid in all other tables that reference userids.
3) repeat step 2 for group ids, product ids, component ids, keyword ids, etc..
4) for each bug in the source bugzilla, change its bug id to an available bug id
in the target bugzilla, appending a comment to it identifying the old number and
updating all references to that bug number to remain consistent.
5) copy each bug and its comments and associated tables to the target.

This should all live in a contrib/ script

Comment 1

[Christian Reis]

bugzilla-submit does some of the hard work of pushing the bug, but the problem
is it uses the post_bug.cgi frontend. 

Would your script be something that did the work via the database directly, or
would it push things over a CGI wire?

Comment 2

[Joel Peshkin]

It would have to be direct database access.
There is way too much reconciliation to do it any other way.

Comment 3

[Joel Peshkin]

Functions this should have....

GetOrAddRecord(table, idfield, sourceid, matchfield, match) -- returns the new
id that sourceid should change to by either locating a record in the target
whose matchfield already matches match and returning its idfield or inserting a
new record in the target with all common fields from the source EXCEPT idfield
and returning idefield for that record after insertion.

ChangeUserId(olduserid, newuserid) -- changes source tables (all of them)
replacing userid.  The list of fields containing userids could be kept in a hash
so the function does not even need to be hardcoded.  It would have to have a
mechanism to ensure that, if the new value conflicts with another record yet to
be converted, that the other is shifted to a new autonumber first.

ChangeKeywordId(oldid, newid)
etc...

This makes the procedure something like

$sth = $source->prepare("SELECT userid, login_name FROM profiles");
$sth->execute();
while (my ($userid, $login) = $sth->fetchrow_array()) {
    my $newid = GetOrAddRecord("profiles","userid",$userid,"login_name",$login);
    ChangeUserID($userid, $newid);
}

and this would repeat for each such item.

Comment 4

[Olav Vitters]

Here is something specific for moving whole product(s) to another Bugzilla
installation (including keywords, creating versions, components, users, ...).
Destination database format must be 2.16 (source also, but can be upgraded to
2.16 using checksetup.pl). Also, source database format was 2.10, so it only
moves the tables used in that database format (no tables created after 2.10),
although the other tables probably won't require that much work.

Anyway see:
http://cvs.gnome.org/viewcvs/bugmasters/tools/mvbugs/

It'll also update the bug numbers and attachments mentioned in comments, create
the product, users, etc. It still is very specific (don't use it if you are not
running 2.16), and needs some changes to be usable for another installation.

Comment 5

[Kevin Benton]

Merging two Bugzilla installations is anything but fun, especially if they're not the same version, and when user names aren't exact matches or worse - when passwords don't match.  Should the script die?  Ask the user what to do?  Also, how do you deal with back-ends of different revisions?  Passwords aren't stored in the same format in MySQL 4.0 versus 4.1 and I have no idea about Postgres...  It's something I'm dealing with here, however, I don't expect to start helping with this till at least April-May time-frame due to current workload.

Comment 6

[Frédéric Buclin]

related bug: bug 123130.

Comment 7

[Max Kanat-Alexander]

Attachment: v1

Here is a contrib script that merges two Bugzillas. It doesn't do Flags, but that wouldn't be too hard.

It also doesn't do Whines and Series, but adding those would be more difficult.

This should work on 3.0 as well as 3.2, I think.

Comment 8

[Max Kanat-Alexander]

The one change required for 3.0 is to make "Install::Util" into "Install::DB".

Comment 9

[Max Kanat-Alexander]

Comment on attachment 303017 [details]
v1

This script has two bugs:

1) For some reason rows aren't added in the correct order, at least for the bugs table. (That is, the order of bug_ids doesn't map to their creation_ts).

2) The regex replacement for "attachment" and "bug" in comments needs to be case-insensitive.

Comment 10

[Max Kanat-Alexander]

By the way, in modern times this should be done by creating a "Bugzilla" module for migrate.pl.