initialization routine needed for default rows in control tables.

RESOLVED WONTFIX

Status

Socorro
Database
RESOLVED WONTFIX
6 years ago
2 years ago

People

(Reporter: lars, Assigned: selenamarie)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
In testing the new Python invocation of the Postgres stored procedure 'weekly_report_partitions', it appears that the stored procedure is incomplete.  It successfully creates the new 'reports' table weekly partitions, but it misses the subordinate tables, 'extensions' and 'plugins_reports'.
(Reporter)

Updated

6 years ago
Blocks: 701253
Hmmm, I tested this in dev.  Are you seeing this in prod, or on stage, or somewhere else?
(Reporter)

Comment 2

6 years ago
I'm seeing in a fresh install of a brand new empty database.  Specifically in the database in the vagrant virtual machine.
This problem appears to be caused by the "reports_partition_info" table being empty on vagrant.  That table needs to be populated (hand).  Rhelmer?
Assignee: nobody → rhelmer
(Reporter)

Comment 4

6 years ago
So there are a set of tables in Postgres that are used to control how socorro works. It isn't sufficient to just create an empty database and run Socorro. We need a way to insert default rows into the control tables after creating the empty database. 

I'm assuming that we're getting away with this right now because when we load sample data, we're just happen to be getting the control data, too.
Summary: weekly_report_partitions stored proc incomplete → initialization routine needed for default rows in control tables.
(In reply to [:jberkus] Josh Berkus from comment #3)
> This problem appears to be caused by the "reports_partition_info" table
> being empty on vagrant.  That table needs to be populated (hand).  Rhelmer?

If someone wants to help to add this sort of thing, each CSV in this file corresponds to a table:
https://github.com/mozilla/socorro/tree/master/tools/dataload

The dataload tool is documented here:
http://socorro.readthedocs.org/en/latest/populatepostgres.html

Also there is documentation about each table and how it's populated here:
http://socorro.readthedocs.org/en/latest/databasetablesbysource.html#databasetablesbysource-chapter

Ideally I'd like to be able to run the dataload tool on a fresh schema import and get 100% Socorro functionality. As it stands today, we know we're missing some documented tables, and I suspect that we have some undocumented dependencies too. Also, the dates on the test data and backfilling processes etc. need to be taken into account (this is something puppet can easily do for us, currently it calls backfill_matviews() for instance).

This probably deserves it's own bug, I'll file that momentarily.
(In reply to K Lars Lohn [:lars] [:klohn] from comment #4)
> So there are a set of tables in Postgres that are used to control how
> socorro works. It isn't sufficient to just create an empty database and run
> Socorro. We need a way to insert default rows into the control tables after
> creating the empty database. 
> 
> I'm assuming that we're getting away with this right now because when we
> load sample data, we're just happen to be getting the control data, too.

Yes if you load a dump derived from prod, you'll get all these important little details. I do believe that being able to create a DB "from scratch" is really important for development purposes, external users, and just for understanding how the system works though, so the fact that this does not work is a valid thing to bring up.
So, need to do a few things here:

1. add reports_partition_info to the list of "manually populated tables". (Josh)

2. Add sensible error condition to the stored procedure if the table is empty (Josh)

3. Add table data to the empty-db load routine (rhelmer?)
No longer blocks: 701253
Depends on: 720738
Selena is working on something similar to this (feel free to dupe etc)
Assignee: rhelmer → sdeckelmann
(Reporter)

Comment 9

2 years ago
old and outdated - refile if this is still needed...
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.