562315 - Scripts for migration from SourceForge to Bugzilla

Status: RESOLVED DUPLICATE of bug 168593

(Bugzilla :: Documentation, enhancement)

Description

[Fabian Groffen]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en; rv:1.9.0.18) Gecko/2010021619 Camino/2.0.2 (like Firefox/3.0.18)
Build Identifier: Bugzilla-3.6

Feel free to close this bug if this is not the right place for this information.

I'm currently working on a migration from the SourceForge bugtracker to Bugzilla.  Our current attitute is not very demanding regarding the import of bugs; anything that improves from no import at all is better.  This results in the scripts/procedure laid out here not to be a perfect 1-to-1 migration.

Maybe this information is interesting enough to put somewhere in some documentation place (a wiki, wherever) such that others can benefit from the steps I've made to get this far.

We started with the XML dump from the sourceforge project as can be done by an administrator.  It includes a lot more than just the tracker items.

On this XML file, I ran the xslt proc as I will attach.  It basically selects the tracker items out of the file and transforms them to the format that Bugzilla (version 3.6) exports itself for bugs.  The proc does a very lazy map of the status and resolutions from sourceforge to Bugzilla's one.  If you added any, it probably needs tweaking.

% xsltproc --novalid --nodtdattr sf2bz.xslt monetdb_export.xml > bugzilla-sfbugs.xml

Because I failed to format timestamps in the xslt translation, this is done afterwards with an AWK script.  Since it's processing anyway, at the same time email addresses can be converted, if required.

% awk -f bzpostprocess.awk bugzilla-sfbugs.xml > bugzilla-final-sfbugs.xml

Next the bugzilla-final-sfbugs.xml can be imported with importxml.pl tool from Bugzilla.  Bug #562308 and bug #562314 contain enhancements that make the import result look nicer.  E.g.:

% ./importxml.pl --post_comments --bug_page="tracker.php?aid=" bugzilla-final-sfbugs.xml

This would do the normal import, but recreate all comments as real comments, and create working links/references to the original bug on SourceForge.  E.g.:

  This bug was previously known as _bug_ 602271 at
  http://sourceforge.net/support/tracker.php?aid=602271


Reproducible: Always

Comment 1

[Fabian Groffen]

Attachment: XSLT proc

Comment 2

[Fabian Groffen]

Attachment: AWK postprocessing script

Comment 3

[Max Kanat-Alexander]

Thanks for your contributions! :-)

Instead of creating custom scripts and using importxml.pl, the correct way to create an importer into Bugzilla 3.6 is using the migrate.pl script. The API of a Bugzilla::Migrate module is described here:

  http://www.bugzilla.org/docs/3.6/en/html/api/Bugzilla/Migrate.html

Once you've taken a look at that, if you'd like to get your contributions reviewed and made a part of Bugzilla, see our development process for details on how to do that:

  http://wiki.mozilla.org/Bugzilla:Developers

Comment 4

[Fabian Groffen]

I totally missed out on migrate.pl, my bad.  Its approach looks like a much better fit for doing a migration if one's very serious about retaining as much details as possible.

Given that I've already reached my goal (with manual addition of users, products and components -- which were to be done anyway -- migrate old bugs for reference), I'm affraid I won't be writing a sourceforge migrator.