898536 - Stage needs signature summary migration

Status: RESOLVED FIXED

(Socorro :: Database, task)

Description

[Lonnen :lonnen]

Signature Summary cron job is failing on stage. Needs the following migration run:

https://github.com/mozilla/socorro/blob/82b794aae1292c0611513c0b68d5f7be6cd9a390/alembic/versions/11cd71153550_bug_895928_signature.py

Comment 1

[Matt Pressman [:mpressman]]

ran on stage:
(venv-alembic)[[e-mail]1 application]$ PYTHONPATH=. alembic  -c ~/.alembic-config  upgrade +1
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade 4d4b48f86d4b -> 2cbb29fc9b77, bug 880378 make update_adu throw exceptions

Comment 2

[Peter Bengtsson [:peterbe]]

If that created the new function, I guess we can close this.

Perhaps a followup bug about adding the `upgrade` script as part of the release scripts. Kinda like we currently do `syncdb` for the django part. I.e. something that can't be run on jenkins.

Comment 3

[Lonnen :lonnen]

Unfortunately I don't think this migration solved it:

Traceback (most recent call last):
  File "/data/socorro/application/socorro/cron/crontabber.py", line 703, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "/data/socorro/application/socorro/cron/base.py", line 174, in main
    function(when)
  File "/data/socorro/application/socorro/cron/base.py", line 213, in _run_proxy
    self.run(connection, date)
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 50, in run
    self.run_proc(connection, [target_date])
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 21, in run_proc
    cursor.callproc(self.get_proc_name(), signature)
ProgrammingError: relation "signature_summary_products" does not exist

Comment 4

[Lonnen :lonnen]

Unfortunately Selena is sick right now. I've ack'd the alert, but I think we'll need her help to sort this out.

Comment 5

[Lonnen :lonnen]

The wrong migration was run, I expect bug 895928 Migration for signature summary tables

Comment 7

[Matt Pressman [:mpressman]]

(venv-alembic)[[e-mail]1 application]$ PYTHONPATH=. alembic  -c ~/.alembic-config  upgrade head
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade 2cbb29fc9b77 -> 3e70bfd1ab0c, bug 889151 add exploitability_report to backfill_matviews
INFO  [alembic.migration] Running upgrade 3e70bfd1ab0c -> 471c6efadde, index on report_date on exploitability_reports
INFO  [alembic.migration] Running upgrade 471c6efadde -> 54e898dc3251, bug 878109 shut camino down
INFO  [alembic.migration] Running upgrade 54e898dc3251 -> 11cd71153550, bug 895928 Signature Summary tables
(venv-alembic)[[e-mail]1 application]$

Comment 8

[Lonnen :lonnen]

New error:

Traceback (most recent call last):
  File "/data/socorro/application/socorro/cron/crontabber.py", line 703, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "/data/socorro/application/socorro/cron/base.py", line 174, in main
    function(when)
  File "/data/socorro/application/socorro/cron/base.py", line 213, in _run_proxy
    self.run(connection, date)
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 50, in run
    self.run_proc(connection, [target_date])
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 21, in run_proc
    cursor.callproc(self.get_proc_name(), signature)
ProgrammingError: permission denied for relation signature_summary_products
CONTEXT:  SQL statement "INSERT INTO signature_summary_products (
    signature_id
    , product_version_id
    , product_name
    , version_string
    , report_count
    , report_date
)
SELECT
    signature_id
    , product_version_id
    , product_name
    , version_string
    , count(*) AS report_count
    , updateday AS report_date
FROM reports_clean
    JOIN product_versions USING (product_version_id)
WHERE
    date_processed::date = updateday
GROUP BY product_version_id, product_name, version_string, signature_id, updateday"
PL/pgSQL function update_signature_summary(date,boolean) line 23 at SQL statement

Comment 9

[Lonnen :lonnen]

I've alerted oncall that this will persist and we don't expect to solve it before Monday. Selena may have some insight for us then.

Comment 10

[Selena Deckelmann :selenamarie :selena]

Fixed with these commands: 

 ALTER TABLE signature_summary_flash_version OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_uptime OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_architecture OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_installations OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_products OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_os OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_process_type OWNER TO breakpad_rw;
GRANT SELECT ON ALL TABLES  IN SCHEMA public to breakpad_ro; 
GRANT ALL on ALL TABLES IN SCHEMA public to breakpad_rw; 

I'd like to make this automatic in alembic.. first step is probably to have alembic connect as the breakpad_rw role instead of postgres. 

Created bug 899355 for the long-term fix. Adding those lines to the migration now.

Comment 11

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/1e05e57b177465867015a2c6d54611ee0def554f
Merge pull request #1371 from selenamarie/bug898536-fix-sigsum-permissions

Fixes bug 898536 signature summary table permissions

Comment 12

[Lonnen :lonnen]

Changes for this bug are going out in 56.