934325 - weekly socorro1.dev.db.phx1 refresh threw error while trying to dump relation daily_hangs

Status: RESOLVED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Matt Pressman [:mpressman]]

From the log output
ERROR:  relation "daily_hangs" does not exist
	LINE 1: COPY  ( SELECT * FROM daily_hangs WHERE report_date >= '2013...

It looks like daily_hangs no longer exists. This prevented the extractdb.tgz file from being built and subsequently decompressed while loading the minidb.

Comment 1

[Matt Pressman [:mpressman]]

This portion of the python script compresses the individual dumps into a compressed file that is then loaded into the dev instance. Since the individual dumps were not removed, when it came time to load them it succeeded as they were in the same location where they would have been if they had been unarchived.

Since the script did complete successfully in dumping a scrubbed minidb dump, everything may be just fine, but I wanted to document the error in case there are any issues not noticed.

Comment 2

[Sheeri Cabral [:sheeri]]

Adding a needinfo so that Selena is aware of the daily_hangs table not existing.

Comment 3

[Selena Deckelmann :selenamarie :selena]

That table was removed September 6, 2012: https://github.com/mozilla/socorro/commit/5a318a1a8f40a004cf53926093b399080eed1fb7

Let me know if you need any particular help from m.e

Comment 4

[Sheeri Cabral [:sheeri]]

What are next steps? Remove that part of the python script?

Comment 5

[Selena Deckelmann :selenamarie :selena]

(In reply to Sheeri Cabral [:sheeri] from comment #4)
> What are next steps? Remove that part of the python script?

Please let me know:

* which script this is
* where it is in revision control
* how it is called (cron? on which machine?)
* how it is configured (configuration file? or maybe just parameters passed to cron

Comment 6

[Sheeri Cabral [:sheeri]]

Selena - that was a comment for Matt...I'm confused, are you suggesting you want to put in the fix for this? I figured as the DB admins we'd fix it, trying not to put more work on your plate when it's clearly administrative stuff that we can handle.

But just in case you wanted to know:

The script that has the daily_hangs table in it is:
 /data/socorro/application/scripts/staging/extractMiniDB.py

It's called by cron on the dev server - /etc/cron.d/dev_refresh, which calls /var/lib/pgsql/bin/dev_refresh.sh, which calls the extractMiniDB.py script. Currently it's in revision control in the postgres2 module; we haven't yet moved it to the postgres module, which we're in the process of doing in another bug.

But that's all for Matt and myself to handle. The only information we needed from you was if the lack of daily_hangs table was a problem, which it is not.

Comment 7

[Sheeri Cabral [:sheeri]]

(In reply to Selena Deckelmann :selenamarie :selena from comment #3)
> That table was removed September 6, 2012:
> https://github.com/mozilla/socorro/commit/
> 5a318a1a8f40a004cf53926093b399080eed1fb7
> 
> Let me know if you need any particular help from m.e

Nope, we got this. Thanx!

Comment 8

[Sheeri Cabral [:sheeri]]

Removing cc of selena as this is not work on her plate (as per an IRC convo with her).

Comment 9

[Matt Pressman [:mpressman]]

There are two places where the daily_hangs view is referenced in the python extract and load scripts. The first was identified and removed from extractMiniDB.py. The second is in loadMiniBonDev.py. I have removed the reference from loadMiniBonDev.py and I'm now running loadMiniBonDev.py on socorro1.dev.db.phx1 to get the data loaded.

Comment 10

[Matt Pressman [:mpressman]]

loadMiniBonDev.py ran successfully after removing the reference to daily_hangs

Comment 11

[Matt Pressman [:mpressman]]

Changed dev_refresh.sh to reference modified extractMiniDB.py and loadMiniBonDev.py that had removed reference to the daily_hangs matview and successfully run in bug 935635