Last Comment Bug 688240 - pushing to try is really slow again
: pushing to try is really slow again
Status: RESOLVED DUPLICATE of bug 708632
:
Product: mozilla.org Graveyard
Classification: Graveyard
Component: Server Operations (show other bugs)
: other
: x86_64 Linux
: -- normal (vote)
: ---
Assigned To: Corey Shields [:cshields]
: Corey Shields [:cshields]
Mentors:
: 695454 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-09-21 12:10 PDT by Rafael Ávila de Espíndola (:espindola) (not reading bugmail)
Modified: 2015-03-12 08:17 PDT (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-09-21 12:10:55 PDT
$ time hg push -f try
pushing to ssh://hg.mozilla.org/try
searching for changes
remote: waiting for lock on repository /repo/hg/mozilla/try held by 'dm-svn02.mozilla.org:10522'
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 5 files (+1 heads)
remote: Looks like you used try syntax, going ahead with the push.
remote: If you don't get what you expected, check http://people.mozilla.org/~lsblakk/trychooser/ for help with building your trychooser request.
remote: Thanks for helping save resources, you're the best!
remote: Trying to insert into pushlog.
remote: Please do not interrupt...
remote: Inserted into the pushlog db successfully.

real	2m24.139s
user	0m0.875s
sys	0m0.071s
Comment 1 :Ehsan Akhgari (busy, don't ask for review please) 2011-09-21 12:23:28 PDT
Resetting the Talos benchmark hurt me today, so it would be really painful for me to see the stuff that I've been working on try disappear by resetting try.  :(
Comment 2 Corey Shields [:cshields] 2011-10-05 10:28:20 PDT
I this still a problem?
Comment 3 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-05 11:33:46 PDT
I have noticed a push being slow today. I will time it next time I push something.
Comment 4 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-10 08:07:14 PDT
Last try was

real	1m45.594s
user	0m1.094s
sys	0m0.185s
Comment 5 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-18 08:06:22 PDT
now it is

real	2m58.934s
user	0m0.941s
sys	0m0.056s
Comment 6 Corey Shields [:cshields] 2011-10-18 08:34:22 PDT
Ben, any thoughts from your hg archeology?
Comment 7 Ben Kero [:bkero] 2011-10-18 08:50:20 PDT
I haven't been looking at try as much as fixing the underlying problems with varnish, which appear to be completed now, so I can focus on this.

try pushes get sent from nginx on dm-hg02 directly to apache on dm-vcview04.  dm-vcview04 apache is extremely busy, pegging all 8 cores with a loadavg hovering around 20.  The load is entire CPU-bound, with very little iowait.  I'm wondering if the load itself could explain slow push times.
Comment 8 Ben Kero [:bkero] 2011-10-18 13:13:17 PDT
*** Bug 695454 has been marked as a duplicate of this bug. ***
Comment 9 Rafael Ávila de Espíndola (:espindola) (not reading bugmail) 2011-10-21 07:58:42 PDT
looks like it just got really bad:

remote: waiting for lock on repository /repo/hg/mozilla/try held by 'dm-svn02.mozilla.org:30578'
remote: abort: repository /repo/hg/mozilla/try: timed out waiting for lock held by dm-svn02.mozilla.org:30578
abort: unexpected response: empty string
Comment 10 Chris Cooper [:coop] 2011-12-12 14:42:28 PST
Bug 708632 is a more recent incarnation of this.

*** This bug has been marked as a duplicate of bug 708632 ***

Note You need to log in before you can comment on or make changes to this bug.