796153 - Upgrade to 9.2: New behavior in >=9.1 Postgres errors out on multiple attempts to set temp_buffers

Status: RESOLVED FIXED

(Socorro :: Database, task)

Description

[Selena Deckelmann :selenamarie :selena]

Raised the issue on pgsql-hackers, work-around is to have temp_buffers set only once per session, which is challenging for the backfill routines.

Comment 1

[Selena Deckelmann :selenamarie :selena]

Another work around is to set temp_buffers to exactly the same value in all the PL code that uses it. Trying that out on :rhelmer's branch.

Comment 2

[Selena Deckelmann :selenamarie :selena]

Sent rhelmer the patch. Closing!

Comment 3

[Selena Deckelmann :selenamarie :selena]

The following functions set temp_buffers explicitly: 

backfill_daily_crashes
backfill_one_day
backfill_one_day
backfill_reports_duplicates
update_adu
update_build_adu
update_correlations
update_crashes_by_user
update_crashes_by_user_build
update_daily_crashes
update_explosiveness
update_home_page_graph
update_home_page_graph_build
update_nightly_builds
update_os_versions
update_os_versions_new_reports
update_rank_compare
update_reports_clean
update_reports_duplicates
update_signatures
update_tcbs
update_tcbs_build

Working on removing temp_buffers from these.

Comment 4

[Selena Deckelmann :selenamarie :selena]

r? https://github.com/mozilla/socorro/pull/899

We probably want to add the ability to set temp_buffers and workmem at the start of a session in the crons. I need some help with that.

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to Selena Deckelmann :selena from comment #4)
> r? https://github.com/mozilla/socorro/pull/899
> 
> We probably want to add the ability to set temp_buffers and workmem at the
> start of a session in the crons. I need some help with that.

These should all be in:
https://github.com/rhelmer/socorro/tree/master/socorro/cron

e.g.:

https://github.com/rhelmer/socorro/blob/master/socorro/cron/dailyMatviews.py
https://github.com/rhelmer/socorro/blob/master/socorro/cron/reportsClean.py
https://github.com/rhelmer/socorro/blob/master/socorro/cron/duplicates.py
https://github.com/rhelmer/socorro/blob/master/socorro/cron/updateADUs.py

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #5)
> (In reply to Selena Deckelmann :selena from comment #4)
> > r? https://github.com/mozilla/socorro/pull/899
> > 
> > We probably want to add the ability to set temp_buffers and workmem at the
> > start of a session in the crons. I need some help with that.
> 
> These should all be in:
> https://github.com/rhelmer/socorro/tree/master/socorro/cron
> 
> e.g.:
> 
> https://github.com/rhelmer/socorro/blob/master/socorro/cron/dailyMatviews.py
> https://github.com/rhelmer/socorro/blob/master/socorro/cron/reportsClean.py
> https://github.com/rhelmer/socorro/blob/master/socorro/cron/duplicates.py
> https://github.com/rhelmer/socorro/blob/master/socorro/cron/updateADUs.py

Oops s/rhelmer/mozilla/ ^ :)

Comment 7

[Selena Deckelmann :selenamarie :selena]

Thanks, Rob! I'll try to fix this up.

Comment 8

[Selena Deckelmann :selenamarie :selena]

:rhelmer - r? https://github.com/selenamarie/socorro/commit/f1093e8ca953b7eb83ddaf80d61ed8fd67282f15

Comment 9

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/bea24d657215574eafba8a8c1a92431c5dbce985
bug 796153 Removed SET memory params from functions
* Moved to 28.0
* Update version numbers in scripts

Comment 10

[Selena Deckelmann :selenamarie :selena]

\o/

Comment 11

[Peter Bengtsson [:peterbe]]

All these upgrades to old cron scripts need to be ported to the new crontabber based ones:
This commit in particular (I think)
https://github.com/mozilla/socorro/commit/2d7910b25b59696dfb2ab994818e901f40a9cc20

crontabber doesn't have a way of setting "session based" things but crontabber makes a new connection for each postgres based app so it should be easy to add lines like::

    connection.execute('SET TEMP_BUFFERS = %s', self.config.databaseTempBuffers)

Comment 12

[Selena Deckelmann :selenamarie :selena]

There's a separate ticket open for the crontabber changes: bug 805625