Closed Bug 755067 Opened 8 years ago Closed 8 years ago
_scheduler errors on 'assert state _json is not None'
Since this morning we've been hitting this on bm36's build_scheduler: 2012-05-14 11:01:10-0700 [-] Unhandled Error Traceback (most recent call last): File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap self.__bootstrap_inner() File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner self.run() File "/usr/lib64/python2.6/threading.py", line 484, in run self.__target(*self.__args, **self.__kwargs) --- <exception caught here> --- File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/twisted/python/threadpool.py", line 207, in _worker result = context.call(ctx, function, *args, **kwargs) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/twisted/enterprise/adbapi.py", line 429, in _runInteraction result = interaction(trans, *args, **kw) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/buildbot-0.8.2_hg_a2045101fe7a_production_0.8-py2.6.egg/buildbot/schedulers/timed.py", line 238, in _check_timer s = self.get_state(t) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/buildbot-0.8.2_hg_a2045101fe7a_production_0.8-py2.6.egg/buildbot/schedulers/base.py", line 90, in get_state return self.parent.db.scheduler_get_state(self.schedulerid, t) File "/builds/buildbot/build_scheduler/lib/python2.6/site-packages/buildbot-0.8.2_hg_a2045101fe7a_production_0.8-py2.6.egg/buildbot/db/connector.py", line 656, in scheduler_g et_state assert state_json is not None exceptions.AssertionError: Reconfig's happened at : 2012-05-14 12:27:08-0700 [-] configuration update complete 2012-05-14 12:27:08-0700 [-] configuration update complete 2012-05-14 14:59:24-0700 [-] configuration update complete The double reconfig at 12:27 is a little odd, but otherwise this isn't a smoking gun. I checked the scheduler db and none of the rows in scheduler table have a state field of length 0. Restarting build_scheduler resolved the problem. Fallout from bug 754885 ?
Or the db going ro ?
Fallout from bug 754885 - I deleted all the pine schedulers since they had a huge backlog of changes to go through. The test schedulers all got recreated when I restarted the tests scheduler master, but I neglected to restart the build scheduler master,so it still had a reference to the old (deleted) pine build scheduler.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.