1427884 - Add upgrade tests from 4.2 to Harmony on mysql

Status: RESOLVED FIXED

(Bugzilla :: Installation & Upgrading, enhancement)

Description

[Dylan Hardison [:dylan] (he/him)]

We need to keep a copy of a 4.2 database dump somewhere (preferably not in the repo)
and add a test that loads it into a fresh database.

We're starting from 4.2 as it is easier than 5.0 or master will be, but we'll need to cover those as well.

We'll need to do this for postgresql eventually too, but let's start small.

To do this we'll need to add a new test to .circleci/config.yml,
use test_bmo as an example.

Something like this:

  test_bmo:
    parallelism: 1
    working_directory: /app
    docker:
      - <<: *bmo_slim_image
        environment:
          <<: *bmo_env
          BZ_QA_ANSWERS_FILE:  /app/.circleci/checksetup_answers.txt
          TWD_HOST: localhost
          TWD_PORT: 4444
          TWD_BROWSER: firefox
      - <<: *mysql_image
        environment: *mysql_env
      - image: memcached:latest
    steps:
      - checkout
      - run: |
          mv /opt/bmo/local /app/local
          curl -L -Obugzilla.sql https://something.example.com/bugzilla.sql
          mysqladmin create -u$BMO_db_user -p$BMO_db_pass -h $BMO_db_host $BMO_db_name
          mysql -u$BMO_db_user -p$BMO_db_pass -h $BMO_db_host $BMO_db_name < bugzilla.sql
          perl checksetup.pl

Comment 1

[Dylan Hardison [:dylan] (he/him)]

I realize how problematic those env names are, so I'm going to do bug 1427887.

Comment 2

[Dylan Hardison [:dylan] (he/him)]

eseyman added the data to https://github.com/bugzilla/harmony-test-fixtures/blob/master/db-dumps/.
I'm needing to add the 'mysql' client commands to our base testing image, then this should be a reality.

Comment 3

[Dylan Hardison [:dylan] (he/him)]

Attachment: PR

This patch adds a failing test case. You can run it locally if you have 'circleci' installed, or see the results on the circleci site (I'll add a link in a subsequent comment once it has finished running).

Next steps: If this PR can be approved and landed, next we can try to fix the underlying problem, which is probably just this:

DBD::mysql::db selectrow_hashref failed: Unknown column 'first_patch_reviewed_id' in 'field list' [for Statement "SELECT profiles.userid,profiles.login_name,profiles.realname,profiles.mybugslink AS showmybugslink,profiles.disabledtext,profiles.disable_mail,profiles.extern_id,profiles.is_enabled,DATE_FORMAT(last_seen_date, '%Y-%m-%d') AS last_seen_date,profiles.password_change_required,profiles.password_change_reason,profiles.mfa,profiles.mfa_required_date,first_patch_reviewed_id,review_request_count,feedback_request_count,needinfo_request_count,comment_count,creation_ts,first_patch_bug_id,last_activity_ts,last_statistics_ts FROM profiles WHERE login_name = ?"] at /app/Bugzilla/Object.pm line 157.

Comment 4

[Dylan Hardison [:dylan] (he/him)]

If I disable the ContributorEngagement extension, we get a bit farther:

DBD::mysql::db selectrow_hashref failed: Unknown column 'review_request_count' in 'field list' [for Statement "SELECT profiles.userid,profiles.login_name,profiles.realname,profiles.mybugslink AS showmybugslink,profiles.disabledtext,profiles.disable_mail,profiles.extern_id,profiles.is_enabled,DATE_FORMAT(last_seen_date, '%Y-%m-%d') AS last_seen_date,profiles.password_change_required,profiles.password_change_reason,profiles.mfa,profiles.mfa_required_date,review_request_count,feedback_request_count,needinfo_request_count,comment_count,creation_ts,first_patch_bug_id,last_activity_ts,last_statistics_ts FROM profiles WHERE login_name = ?"] at/app/Bugzilla/Object.pm line 157.
Error: Exited with code 255

This suggests a more general problem, e.g. ignoring columns provided by extensions at this part of the migration process perhaps.

Comment 5

[Dylan Hardison [:dylan] (he/him)]

Update! I've split off the db migration code to scripts/migrate-database.pl. If I run it once, I get this error:

DBD::mysql::db selectrow_hashref failed: Unknown column 'first_patch_reviewed_id' in 'field list' [for Statement "SELECT profiles.userid,profiles.login_name,profiles.realname,profiles.mybugslink AS showmybugslink,profiles.disabledtext,profiles.disable_mail,profiles.extern_id,profiles.is_enabled,DATE_FORMAT(last_seen_date, '%Y-%m-%d') AS last_seen_date,profiles.password_change_required,profiles.password_change_reason,profiles.mfa,profiles.mfa_required_date,first_patch_reviewed_id,review_request_count,feedback_request_count,needinfo_request_count,comment_count,creation_ts,first_patch_bug_id,last_activity_ts,last_statistics_ts FROM profiles WHERE login_name = ?"] at /Users/dylan/src/bteam/bmo/bmo/scripts/../Bugzilla/Object.pm line 157.

Bug if I run it a second time, it works and I have a working install!