1265029 - replication log not available; all writes disabled

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Trying to push to m-c, I get errors:
[:~/mozilla/unified] 255 $ hg push -v -r . central
pushing to ssh://hg.mozilla.org/mozilla-central
searching for changes
1 changesets found
uncompressed size of bundle content:
     860 (changelog)
     857 (manifests)
     147  js/src/builtin/Array.js
     172  js/src/builtin/String.js
     662  js/src/jsapi.h
    1135  js/src/jsarray.cpp
     140  js/src/jsarray.h
    2058  js/src/jsobj.cpp
    2677  js/src/jsstr.cpp
     138  js/src/jsstr.h
     161  js/src/vm/SelfHosting.cpp
     596  js/xpconnect/tests/chrome/test_xrayToJS.xul
remote: replication log not available; all writes disabled
remote: pretxnopen.vcsreplicator hook failed
abort: push failed on remote



I'm not the only one hitting this. Closing all trees (though that doesn't matter much since writes are disabled anyway).

Comment 1

[Gregory Szorc [:gps]]

Fixed.

Comment 2

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Reopening.
Seeing #vcs alerts like
11:41 AM <nagios-scl3> Fri 11:41:25 PDT [5440] hgweb1.dmz.scl3.mozilla.com:procs - hg vcsreplicator consumer is CRITICAL: PROCS CRITICAL: 1 process with regex args vcsreplicator-consumer (http://m.mozilla.org/procs+-+hg+vcsreplicator+consumer)
11:41 AM <nagios-scl3> Fri 11:41:45 PDT [5441] hgweb2.dmz.scl3.mozilla.com:procs - hg vcsreplicator consumer is CRITICAL: PROCS CRITICAL: 3 processes with regex args vcsreplicator-consumer (http://m.mozilla.org/procs+-+hg+vcsreplicator+consumer)

And adw reports that his successful push to fx-team isn't showing up on hg.m.o:
11:38 AM <adw> KWierso: i just pushed to fx-team but it's not showing up at https://hg.mozilla.org/integration/fx-team
11:39 AM <adw> no errors on my end
11:39 AM <KWierso> and it actually went through?
11:39 AM <KWierso> did you get an actual URL to your commits that you pushed?
11:39 AM <adw> yeah. i was waiting for the tree to reopen
11:39 AM <adw> https://hg.mozilla.org/integration/fx-team/rev/009d7df1ba6d
11:39 AM <Gijs> gps: ^^^
11:39 AM <KWierso> yeah, sounds like maybe the vcs-replicator got screwy?

Comment 3

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

There's some lag in the replicators, but gps says they should catch up shortly.

Comment 4

[Gregory Szorc [:gps]]

Everything except try and the user repos are caught up.