508974 - Create migration script from Bouncer database to Mirror Brain database

Status: RESOLVED WONTFIX

(Webtools :: Bouncer, defect)

Description

[Michael Morgan [:morgamic]]

In order to test MB we need to set up our products in MB and slam it.  First part of that is attaining parity between the two in terms of size and volume.

Steps:
1. map bouncer tables to mb tables (flatten if necessary)
2. create corresponding entries for mirrors, weights, products, etc. in the MB db

Comment 1

[Michael Morgan [:morgamic]]

MB SQL is here:
http://svn.mirrorbrain.org/svn/mirrorbrain/trunk/sql/schema-mysql.sql

Frank - we'll talk about this next week.

Comment 2

[Michael Morgan [:morgamic]]

Frank - how are things progressing?  Can you give an estimate on when you'll be done?

Comment 3

[Frank Griswold [:griswolf] [:fgriswold]]

We are going to do this in three parts:
Part 1: Use script (attached) to convert the bouncer data in situ to mirrorbrain structure, all within mysql. This creates several tables and populates three of them: file, server, file_server (a mapping table)

Part 2: Use mirror-brain scripts pulled from their svn to create and partly populate the postgres side:
  URL: http://svn.mirrorbrain.org/svn/mirrorbrain/trunk/sql/schema-postgresql.sql
  URL: http://svn.mirrorbrain.org/svn/mirrorbrain/trunk/sql/initialdata-postgresql.sql

Part 3: Convert the data from part 1 to postgres

Comment 4

[Frank Griswold [:griswolf] [:fgriswold]]

Attachment: Converts bouncer to mirror-brain schema, within mysql database

Comment 5

[K Lars Lohn [:lars] [:klohn]]

Attachment: a python script to migrate from MySQL Bouncer to PosgreSQL MirrorBrain

Attached is Python script that will migrate a MySQL based Bouncer instance to the schema of MirrorBrain running under PostgreSQL.

It is likely that some tweaks are going to be necessary as we learn more about MirrorBrain.

> ./bouncer2mb.py --help
Bouncer to MirrorBrain 0.1
	    --bouncerDatabaseHost
		the hostname of the bouncer database server (default: localhost)
	    --bouncerDatabaseName
		the name of the database within the server (default: bouncer)
	    --bouncerDatabasePassword
		the password for the database user (default: )
	    --bouncerDatabaseUserName
		the user name for the database servers (default: root)
	-?, --help
		print this list
	    --logFileErrorLoggingLevel
		logging level for the log file (10 - DEBUG, 20 - INFO, 30 - WARNING, 40 - ERROR, 50 - CRITICAL) (default: 10)
	    --logFileLineFormatString
		python logging system format for log file entries (default: %(asctime)s %(levelname)s - %(message)s)
	    --logFileMaximumBackupHistory
		maximum number of log files to keep (default: 50)
	    --logFileMaximumSize
		maximum size in bytes of the log file (default: 1000000)
	    --logFilePathname
		full pathname for the log file (default: ./bouncer2mb.log)
	    --mirrorBrainDatabaseHost
		the hostname of the bouncer database server (default: localhost)
	    --mirrorBrainDatabaseName
		the name of the database within the server (default: mirrorbrain)
	    --mirrorBrainDatabasePassword
		the password for the database user (default: )
	    --mirrorBrainDatabaseUserName
		the user name for the database servers (default: root)
	    --stderrErrorLoggingLevel
		logging level for the logging to stderr (10 - DEBUG, 20 - INFO, 30 - WARNING, 40 - ERROR, 50 - CRITICAL) (default: 20)
	    --stderrLineFormatString
		python logging system format for logging to stderr (default: %(asctime)s %(levelname)s - %(message)s)

Comment 6

[Frank Griswold [:griswolf] [:fgriswold]]

Comment on attachment 403505 [details]
Converts bouncer to mirror-brain schema, within mysql database

replaced by python script also attached

Comment 7

[Jeremy Orem [:oremj]]

Is this bug "FIXED"?

Comment 8

[K Lars Lohn [:lars] [:klohn]]

This bug isn't "FIXED" until the script is executed and we work out any changes that need to be made.  I'm just waiting for someone to use it and tell me how it went...

Comment 9

[Fred Wenzel [:wenzel]]

We have decided in late 2009 that Bouncer is currently serving our needs relatively well and we won't switch to MirrorBrain at this time.