1526371 - Dylan needs a newer database dump to locally test the STMO data delivery pipeline

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Infrastructure, enhancement)

Description

[Dylan Hardison [:dylan] (he/him)]

Basically I have some complicated-ish queries to write and I need some production-like data to experiment with.

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Steps (roughly?)

• database copy of prod
• run sanitizeme.pl --keep-groups on that
• zip, encrypt, and put on a g-drive.

Comment 2

[Bob Micheletto [:bobm]]

Sanitizing a copy of production currently.

Comment 3

[Bob Micheletto [:bobm]]

sanitizeme.pl died a couple of times on security bugs and needed to be restarted. It's still running.

Comment 4

[Bob Micheletto [:bobm]]

sanitizedme.pl died on 'Removing attachment data' but started again from insidergroup comments and attachments. Hopefully it's not stuck:

$ scripts/sanitizeme.pl --execute --keep-groups
Deleting 0 bugs in security groups...
Done
Deleting 'insidergroup' comments and attachments...
Deleting sensitive user account data...
Removing attachment data...
DBD::mysql::db do failed: Lost connection to MySQL server during query [for Statement "UPDATE attach_data SET thedata = ''"] at scripts/sanitizeme.pl line 118.

$ scripts/sanitizeme.pl --execute --keep-groups
Deleting 0 bugs in security groups...
Done
Deleting 'insidergroup' comments and attachments...

Comment 5

[Bob Micheletto [:bobm]]

(In reply to Bob Micheletto [:bobm] from comment #4)

sanitizedme.pl died on 'Removing attachment data' but started again from insidergroup comments and attachments. Hopefully it's not stuck.

It appears to be stuck.

$ scripts/sanitizeme.pl --execute --keep-groups
Deleting 0 bugs in security groups...
Done
Deleting 'insidergroup' comments and attachments...
Deleting sensitive user account data...
Removing attachment data...
DBD::mysql::db do failed: Lost connection to MySQL server during query [for Statement "UPDATE attach_data SET thedata = ''"] at scripts/sanitizeme.pl line 118.

$ scripts/sanitizeme.pl --execute --keep-groups
Deleting 0 bugs in security groups...
Done
Deleting 'insidergroup' comments and attachments...
DBD::mysql::db do failed: Lost connection to MySQL server during query [for Statement "UPDATE bugs_fulltext SET comments = comments_noprivate"] at scripts/sanitizeme.pl line 118.

$ scripts/sanitizeme.pl --execute --keep-groups
Deleting 0 bugs in security groups...
Done
Deleting 'insidergroup' comments and attachments...

Comment 6

[Dylan Hardison [:dylan] (he/him)]

(we answered the above problems in IRC)

Recipients of the data should be:

• [e-mail]
• [e-mail]
• [e-mail]

Comment 7

[David Lawrence [:dkl]]

Got it thanks.

Comment 9

[Dylan Hardison [:dylan] (he/him)]

As kohei is a contractor and under NDA, he may have access to the sanitized db dump.