Closed Bug 1504903 Opened 7 years ago Closed 4 years ago

Upgrade FxA database to MySQL 5.7

Categories

(Cloud Services :: Server: Firefox Accounts, task)

Product:
Cloud Services
Component:
Server: Firefox Accounts
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rfkelly, Assigned: jbuck)

References

(
URL
)

Details

To help ensure future db migrations go smoothly, let's update FxA to be running on MySQL 5.7 rather than 5.6.
> To help ensure future db migrations go smoothly, let's update FxA to be running on MySQL 5.7 rather than 5.6. To be clear, the migration tool in question exhibits the same bug in the very latest patch level of 5.7.
From our investigations so far, it sounds like we don't really expect this update to help with anything. Should we remove it as a blocker for the db-migration-problems metabug in Bu 1504901?
Flags: needinfo?(jrgm)
> Should we remove it as a blocker for the db-migration-problems metabug in Bug 1504901? I'm going to just go ahead and remove it as a blocker.
Flags: needinfo?(jrgm)
Assignee: nobody → jrgm
Status: NEW → ASSIGNED
Assignee: jrgm → jbuckley
Group: services-infra

On 2021-02-07 from 9-9:20PM ET the Firefox Accounts team will be performing a database upgrade from MySQL 5.6 to MySQL 5.7 resulting in a 20 minute downtime. Firefox Accounts users will not be able to log in, create new accounts, or subscribe to paid products during this 20 minute downtime.

This upgrade is required due to MySQL 5.6 reaching it’s end-of-life on 2021-02-05. We are performing it during the lowest period of active users on a weekly basis.

If this time period conflicts with any planned activities your team has, please let us know so we can reschedule. If you’d like to follow along, please watch this bug and join #fxa on Slack during the downtime for the festivities.

Component: Operations: Deployment Requests - DEPRECATED → Server: Firefox Accounts
Summary: Update FxA database to MySQL 5.7 → Upgrade FxA database to MySQL 5.7

Upgrade complete!

There was 19 minutes of downtime, from 9:10PM to 9:29PM ET. This was 9 minutes longer than planned, see details below.

  • The oauth db had 4 minutes of downtime (9:10PM to 9:14PM) as pre-upgrade testing showed, but the auth db had 15 minutes of downtime (9:14PM to 9:29PM). It's unfortunate that the downtime didn't overlap neatly as I was hoping, but more annoyingly there was several minutes of Waiting for page_cleaner to finish flushing of buffer pool before the auth db shut down. My pre-upgrade testing didn't show this happening, but it would've been prudent to reserve more time for unexpected issues.
  • :sven was paged for kintowe being down, but shouldn't have been since this was a known maintenance window
  • Sentry deleted a bunch of our old error events for some reason
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.