Closed Bug 898536 Opened 8 years ago Closed 8 years ago

Stage needs signature summary migration

Categories

(Socorro :: Database, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: lonnen, Assigned: selenamarie)

References

Details

(Whiteboard: [DB Changes][qa-])

Signature Summary cron job is failing on stage. Needs the following migration run:

https://github.com/mozilla/socorro/blob/82b794aae1292c0611513c0b68d5f7be6cd9a390/alembic/versions/11cd71153550_bug_895928_signature.py
Assignee: nobody → mpressman
ran on stage:
(venv-alembic)[postgres@socorro1.stage.db.phx1 application]$ PYTHONPATH=. alembic  -c ~/.alembic-config  upgrade +1
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade 4d4b48f86d4b -> 2cbb29fc9b77, bug 880378 make update_adu throw exceptions
If that created the new function, I guess we can close this.

Perhaps a followup bug about adding the `upgrade` script as part of the release scripts. Kinda like we currently do `syncdb` for the django part. I.e. something that can't be run on jenkins.
Unfortunately I don't think this migration solved it:

Traceback (most recent call last):
  File "/data/socorro/application/socorro/cron/crontabber.py", line 703, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "/data/socorro/application/socorro/cron/base.py", line 174, in main
    function(when)
  File "/data/socorro/application/socorro/cron/base.py", line 213, in _run_proxy
    self.run(connection, date)
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 50, in run
    self.run_proc(connection, [target_date])
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 21, in run_proc
    cursor.callproc(self.get_proc_name(), signature)
ProgrammingError: relation "signature_summary_products" does not exist
Unfortunately Selena is sick right now. I've ack'd the alert, but I think we'll need her help to sort this out.
Target Milestone: --- → 55
The wrong migration was run, I expect bug 895928 Migration for signature summary tables
Duplicate of this bug: 898507
(venv-alembic)[postgres@socorro1.stage.db.phx1 application]$ PYTHONPATH=. alembic  -c ~/.alembic-config  upgrade head
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade 2cbb29fc9b77 -> 3e70bfd1ab0c, bug 889151 add exploitability_report to backfill_matviews
INFO  [alembic.migration] Running upgrade 3e70bfd1ab0c -> 471c6efadde, index on report_date on exploitability_reports
INFO  [alembic.migration] Running upgrade 471c6efadde -> 54e898dc3251, bug 878109 shut camino down
INFO  [alembic.migration] Running upgrade 54e898dc3251 -> 11cd71153550, bug 895928 Signature Summary tables
(venv-alembic)[postgres@socorro1.stage.db.phx1 application]$
New error:

Traceback (most recent call last):
  File "/data/socorro/application/socorro/cron/crontabber.py", line 703, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "/data/socorro/application/socorro/cron/base.py", line 174, in main
    function(when)
  File "/data/socorro/application/socorro/cron/base.py", line 213, in _run_proxy
    self.run(connection, date)
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 50, in run
    self.run_proc(connection, [target_date])
  File "/data/socorro/application/socorro/cron/jobs/matviews.py", line 21, in run_proc
    cursor.callproc(self.get_proc_name(), signature)
ProgrammingError: permission denied for relation signature_summary_products
CONTEXT:  SQL statement "INSERT INTO signature_summary_products (
    signature_id
    , product_version_id
    , product_name
    , version_string
    , report_count
    , report_date
)
SELECT
    signature_id
    , product_version_id
    , product_name
    , version_string
    , count(*) AS report_count
    , updateday AS report_date
FROM reports_clean
    JOIN product_versions USING (product_version_id)
WHERE
    date_processed::date = updateday
GROUP BY product_version_id, product_name, version_string, signature_id, updateday"
PL/pgSQL function update_signature_summary(date,boolean) line 23 at SQL statement
I've alerted oncall that this will persist and we don't expect to solve it before Monday. Selena may have some insight for us then.
See Also: → 898943
Fixed with these commands: 

 ALTER TABLE signature_summary_flash_version OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_uptime OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_architecture OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_installations OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_products OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_os OWNER TO breakpad_rw;
 ALTER TABLE signature_summary_process_type OWNER TO breakpad_rw;
GRANT SELECT ON ALL TABLES  IN SCHEMA public to breakpad_ro; 
GRANT ALL on ALL TABLES IN SCHEMA public to breakpad_rw; 

I'd like to make this automatic in alembic.. first step is probably to have alembic connect as the breakpad_rw role instead of postgres. 

Created bug 899355 for the long-term fix. Adding those lines to the migration now.
Assignee: mpressman → sdeckelmann
Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/1e05e57b177465867015a2c6d54611ee0def554f
Merge pull request #1371 from selenamarie/bug898536-fix-sigsum-permissions

Fixes bug 898536 signature summary table permissions
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Whiteboard: [DB Changes][qa-]
Changes for this bug are going out in 56.
Target Milestone: 55 → 56
You need to log in before you can comment on or make changes to this bug.