1437924 - socorro deploy: 309

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

We want to do a Socorro -prod deploy today tagged 309.

It consists of the following:

(current tag: 308 - 36492caf)
c288c317: Update sqlalchemy from 1.2.0 to 1.2.2
25651e8a: Update alembic from 0.9.6 to 0.9.7
0f90ba14: Update elasticsearch from 1.9.0 to 6.1.1
fb32887a: Update elasticsearch-dsl from 0.0.11 to 6.1.0
b1668d7c: Update raven from 6.4.0 to 6.5.0
e0a1b925: Update pyquery from 1.3.0 to 1.4.0
f19cb02a: Update django-waffle from 0.12.0 to 0.13.0
3ba8f69a: Update django from 1.11.7 to 2.0.2
dc0aa91b: Update datadog from 0.17.0 to 0.18.0
97a040f0: Update newrelic from 2.98.0.81 to 2.100.0.84
3fb85b16: Update django-pipeline from 1.6.13 to 1.6.14
76454507: Update humanfriendly from 4.4.1 to 4.8
5706e152: Update pytest from 3.3.1 to 3.4.0
084213ab: Update pytest from 3.3.2 to 3.4.0
b1596ee4: Update pinax-eventlog from 2.0.2 to 2.0.3
9733839d: Update awscli from 1.14.23 to 1.14.32
3c850034: Update sphinx from 1.6.5 to 1.6.6
caa93281: Update typing from 3.6.2 to 3.6.4
12ecde3a: Update babel from 2.5.1 to 2.5.3
00fc0265: Update commonmark from 0.5.4 to 0.7.4
1baafec8: Update pytest-selenium from 1.11.3 to 1.11.4
48b57855: Update pytest-variables from 1.7.0 to 1.7.1
b0f36b62: Update pytest-xdist from 1.21.0 to 1.22.0
7fe9690a: Revert "Update django from 1.11.7 to 2.0.2"
dafcac73: Revert "Update elasticsearch-dsl from 0.0.11 to 6.1.0"
6e196618: Revert "Update elasticsearch from 1.9.0 to 6.1.1"
ae3f317a: fixes bug 1429663 - drop pytest-catchlog
090afbe2: fixes bug 1436537 - fixes sha to tag with in deploy_bug.py
0784a9ab: fixes bug 1435068 - remove tcbs and tcbs_build tables
f3d792b6: fix bug 1299465 - nix CrashData classes we're not using
ce2b0910: fix bug 1435551 - stop saving to processed_crashes_* tables
41337225: fixes bug 1435551 - remove processed_crashes* tables
f41846fa: fixes bug 1437414 - clarify crash data situation in getting started
eb74bde7: fixes bug 1437632 - make test run interactive
1a7bb50a: Fix bug 1437390: Append correlations as text nodes.
9f30ec03: Fix bug 1437223: Add config for Pyup API key.
(next tag: 309 - 9030ee13)

After deploy:

* update -prod admin node
* perform alembic migrations
* run alembic migrations on -new-stage and -new-prod

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Tagging myself and Mike for verifying -stage.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I did some changes that covered the tests and dev stuff. Those are fine.

I ran the migrations on -stage for the tcbs* tables and the processed_crashes* tables. Those are fine.

I'm good to go for this deploy once -stage is up to date.

Comment 3

[Osmose [:osmose, :mkelly]]

I verified bug 1437390 on staging. I don't think bug 1437223 has anything we can verify beyond the dependency job not crashing.

The library updates might warrant a run through the system checklist, but that's a judgement call I'll leave to willkg. Otherwise, this looks good from my end.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

My bad--I've gone through most of the system checklist with -stage already to cover the lib updates.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

-stage updated and it looks good.

Miles, Brian: Can one of you two ping me in IRC and do a socorro -prod deploy today?

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pretty sure Miles deployed 309 and that the deploy is done now.

Given that, I moved on. I updated the -prod admin node from 1:307.1518029020-1 to 1:308.1518539778-1.

I ran alembic migrations:

[centos@prod ~]$ ./bin/alembic_run.sh upgrade heads
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 0f31c225e765 -> 3c5d40ea7c5a, bug 1435068 remove tcbs and tcbs_build table
INFO  [alembic.runtime.migration] Running upgrade 3c5d40ea7c5a -> 1feab2fbed4c, bug 1435551 processed_crashes

The db went from like 430gb to 120gb!

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Miles, Brian: Did someone run the alembic migrations in the new infrastructure? Are there other things this bug is waiting on or can we close it out?

Comment 8

[Miles Crabill [:miles]]

Migrations have been run in -new-stage and -new-prod. Marking this fixed.