Closed Bug 462706 Opened 16 years ago Closed 16 years ago

hg push throwed unknown exception "database is locked"

Categories

(mozilla.org Graveyard :: Server Operations, task)

x86
Linux
task
Not set
blocker

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: MatsPalmgren_bugz, Unassigned)

Details

Followup from bug 462696.  Somehow we got into a state of
"database is locked" and no-one could push.  After I got the following
error I had to abandon this working tree.

pushing to ssh://hg.mozilla.org/mozilla-central
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: error: pretxnchangegroup.z_linearhistory hook raised an exception:
database is locked
remote: transaction abort!
remote: rollback completed
remote: ** unknown exception encountered, details follow
remote: ** report bug details to http://www.selenic.com/mercurial/bts
remote: ** or mercurial@selenic.com
remote: ** Mercurial Distributed SCM (version 1.0.2)
remote: Traceback (most recent call last):
remote:   File "/usr/bin/hg", line 20, in ?
remote:     mercurial.dispatch.run()
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
20, in run
remote:     sys.exit(dispatch(sys.argv[1:]))
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
29, in dispatch
remote:     return _runcatch(u, args)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
45, in _runcatch
remote:     return _dispatch(ui, args)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
364, in _dispatch
remote:     ret = _runcommand(ui, options, cmd, d)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
417, in _runcommand
remote:     return checkargs()
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
373, in checkargs
remote:     return cmdfunc()
remote:   File "/usr/lib/python2.4/site-packages/mercurial/dispatch.py", line
356, in <lambda>
remote:     d = lambda: func(ui, repo, *args, **cmdoptions)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/commands.py", line
2514, in serve
remote:     s.serve_forever()
remote:   File "/usr/lib/python2.4/site-packages/mercurial/sshserver.py", line
40, in serve_forever
remote:     while self.serve_one(): pass
remote:   File "/usr/lib/python2.4/site-packages/mercurial/sshserver.py", line
47, in serve_one
remote:     if impl: impl()
remote:   File "/usr/lib/python2.4/site-packages/mercurial/sshserver.py", line
194, in do_unbundle
remote:     r = self.repo.addchangegroup(fp, 'serve', self.client_url())
remote:   File "/usr/lib/python2.4/site-packages/mercurial/localrepo.py", line
2056, in addchangegroup
remote:     url=url)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/localrepo.py", line
123, in hook
remote:     return hook.hook(self.ui, self, name, throw, **args)
remote:   File "/usr/lib/python2.4/site-packages/mercurial/hook.py", line 107,
in hook
remote:     args, throw) or r
remote:   File "/usr/lib/python2.4/site-packages/mercurial/hook.py", line 51, in
_pythonhook
remote:     r = obj(ui=ui, repo=repo, hooktype=name, **args)
remote:   File "/usr/lib/python2.4/site-packages/mozhghooks/pushlog.py", line
52, in log
remote:     conn.commit()
remote: pysqlite2.dbapi2.OperationalError: database is locked
abort: unexpected response: empty string
Perhaps aravind can comment on why the pushlog db was locked. Otherwise I have no idea what's broken.
I'm hitting this again right now -- trying to push a trivial change that disables a test, to try to fix mac unittest oranges (per bug 464141 comment 11).

I tried on two different machines, and I also tried pushing a different unrelated change (removing the executable bit from a random .js file), and I failed in all cases.

Aravind / Ted, any ideas?
(For reference, see also http://www.selenic.com/mercurial/bts/issue1372 which Mats filed on this issue -- it was closed since it's a bug in Mozilla's hooks)
Assignee: nobody → server-ops
Severity: critical → blocker
Component: Hg: Customizations → Server Operations
QA Contact: hg.customizations → mrz
Severity: blocker → critical
Component: Server Operations → Hg: Customizations
(sorry, not sure why I didn't get a mid-air warning about that).
Severity: critical → blocker
Component: Hg: Customizations → Server Operations
I don't see anything actually having that file open on the read/write servers.

The read-only webheads serving hg.m.o seem to be dead though.  The last time
this happened, fixing those servers seems to have helped.  Trying that now.
Assignee: server-ops → aravind
FWIW, I'm still hitting this problem, pushing a change I've committed to my local repo.

I'll try again using a fresh repo, in case there's any accumulated crud from the previous failure...
Severity: blocker → critical
Component: Server Operations → Hg: Customizations
(sorry to change component/severity -- bugzilla seems to be witholding its mid-air warnings from mrbkap and me.)
Assignee: aravind → server-ops
Severity: critical → blocker
Component: Hg: Customizations → Server Operations
Woo-hoo -- it worked using a fresh repo! not sure if the working was due to the fresh repo, or due to the elapsed time that it took to get a fresh repo.  But, my push succeeded, which is what counts. :)

Thanks aravind! Closing this bug so it stops paging IT.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
bumped up the memory on the VMs serving the read-only content, lets see if the front end is more stable now.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.