Last Comment Bug 676420 - please clone and prune try
: please clone and prune try
Product: Graveyard
Classification: Graveyard
Component: Server Operations (show other bugs)
: other
: All All
-- major (vote)
: ---
Assigned To: Shyam Mani [:fox2mike] (AFK until March 10)
: matthew zeier [:mrz]
Depends on: 554656 633161
Blocks: 672231
  Show dependency treegraph
Reported: 2011-08-03 16:11 PDT by Aki Sasaki [:aki]
Modified: 2015-03-12 08:17 PDT (History)
7 users (show)
aki: needs‑treeclosure+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Description User image Aki Sasaki [:aki] 2011-08-03 16:11:57 PDT
Try is slow, and at >4000 heads.
We need to prune those heads or re-clone try, but John O'Duinn needs the pushloghtml stats from existing try before we do that.

I think the path to success involves

a) moving or cloning existing try to a temporary location, until John O'Duinn can gather stats, and then nuke the site from orbit.
b) pruning the heads from hg.m.o/try, or recloning try.

Adding bug 554656 and bug 633161 as "depends on" bugs, even though they don't strictly block; more for reference.
Comment 1 User image Ted Mielczarek [:ted.mielczarek] 2011-08-04 04:36:36 PDT
The point of the work in bug 633161 was to be able to strip old heads without having to completely reset the repo. All indications were that it seemed to work, I think we should continue with that instead of blowing away the entire repo, since it can be done without any impact on pushing to try.
Comment 2 User image Dumitru Gherman [:dumitru] 2011-08-04 10:32:07 PDT
I can follow the path to success described by Aki by
a) cloning
Comment 3 User image Dumitru Gherman [:dumitru] 2011-08-04 10:47:31 PDT
Sorry for the last comment, I hit enter to save changes accidentally.
According to a TreeClosure is needed. CC'ing arr to confirm and let me know when can we do this.

Also, is a) necessary or can I jump to b) directly?
Comment 4 User image Aki Sasaki [:aki] 2011-08-04 10:54:42 PDT
a) is necessary to get stats from Try; otherwise we have to block on John O'Duinn's schedule, which will push this out several days.

bug 633161 has a script ( that Ted says will work on an open Try tree. has details on how to use it.

I'll negotiate with the sheriff for a soft downtime just in case.
Comment 5 User image Shyam Mani [:fox2mike] (AFK until March 10) 2011-08-04 12:41:27 PDT
a) is done, we have a backup of the db.

Ted's script will run into issues with try + pushlog hook enabled..he ran into that while testing, so we're going to have to reclone try, I guess.
Comment 6 User image Shyam Mani [:fox2mike] (AFK until March 10) 2011-08-05 01:06:03 PDT
I think this was supposed to happen on Monday, Aug 8th at 0600 PDT. Not sure if notices were sent. CC'ing catlee who's supposed to be on buildduty on Monday.
Comment 7 User image Aki Sasaki [:aki] 2011-08-05 11:59:09 PDT
Notice sent.
Not sure if catlee is aware of this yet; if it's approaching 6am PDT Monday and he's not around, feel free to call my cell (in phonebook) =\
Comment 8 User image Shyam Mani [:fox2mike] (AFK until March 10) 2011-08-08 06:55:39 PDT
Sweet. Following steps from the wiki :

[hg@dm-svn01 mozilla]$ time hg clone -U mozilla-central try

real    7m24.144s
user    0m3.694s
sys     0m7.957s

[hg@dm-svn01 mozilla]$ cp /repo/hg/mozilla/mozilla-central/.hg/pushlog2.db /repo/hg/mozilla/try/.hg/pushlog2.db 
[hg@dm-svn01 mozilla]$ chown hg:scm_level_1 /repo/hg/mozilla/try/.hg/pushlog2.db 
[hg@dm-svn01 mozilla]$ chmod g+w /repo/hg/mozilla/try/.hg/pushlog2.db
[hg@dm-svn01 mozilla]$ cd /repo/hg/mozilla/try/.hg/ 
[hg@dm-svn01 .hg]$ sqlite3 pushlog2.db 
SQLite version 3.3.6
Enter ".help" for instructions
sqlite> delete from changesets;
sqlite> delete from pushlog; 
sqlite> .exit

[root@dm-svn01 mozilla]# chown -R hg:scm_level_1 try
[root@dm-svn01 mozilla]#

And the copy on the varnish server has been rebuilt as well. 

This should be good. I have a copy of the pushlogdb and can put it up someplace when John needs it.

Note You need to log in before you can comment on or make changes to this bug.