Closed Bug 1014443 Opened 11 years ago Closed 11 years ago

buildbot master's pulse queue is full and filled up /dev/shm.

Categories

(SeaMonkey :: Release Engineering, defect)

Product:
SeaMonkey
Component:
Release Engineering
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ewong, Assigned: ewong)

References

Details

sea-master1's pulse log has filled up. +++ This bug was initially created as a clone of Bug #831145 +++ The SeaMonkey buildbot master twistd.log contains: 2013-01-15 20:02:05-0800 [buildbot.status.web.baseweb.RotateLogSite] Could not accept new connection (EMFILE) 2013-01-15 20:02:05-0800 [twisted.spread.pb.PBServerFactory] Could not accept new connection (EMFILE) and it repeats. Going into /proc/<pid>/fd there are a lot of entries: (an example) lrwx------ 1 seabld seabld 64 Jan 15 20:23 975 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703QkoKOI lrwx------ 1 seabld seabld 64 Jan 15 20:23 976 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703cN3Fsr lrwx------ 1 seabld seabld 64 Jan 15 20:23 977 -> /dev/shm/queue/pulse/tmp/1358218872-2172-157039um5Xu lrwx------ 1 seabld seabld 64 Jan 15 20:23 978 -> /dev/shm/queue/pulse/tmp/1358218872-2172-157033L2ayj lrwx------ 1 seabld seabld 64 Jan 15 20:23 979 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703XVHD7g lrwx------ 1 seabld seabld 64 Jan 15 20:23 98 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703aguZ_I lrwx------ 1 seabld seabld 64 Jan 15 20:23 980 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703CWqU5w lrwx------ 1 seabld seabld 64 Jan 15 20:23 981 -> /dev/shm/queue/pulse/tmp/1358218872-2172-157034mKICI lrwx------ 1 seabld seabld 64 Jan 15 20:23 982 -> /dev/shm/queue/pulse/tmp/1358218872-2172-157034KKooR lrwx------ 1 seabld seabld 64 Jan 15 20:23 983 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703fWbDUq lrwx------ 1 seabld seabld 64 Jan 15 20:23 984 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703nPj2IV lrwx------ 1 seabld seabld 64 Jan 15 20:23 987 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703QUFmnP lrwx------ 1 seabld seabld 64 Jan 15 20:23 988 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703D7LsO9 lrwx------ 1 seabld seabld 64 Jan 15 20:23 989 -> /dev/shm/queue/pulse/tmp/1358218872-2172-1570342Di_r lrwx------ 1 seabld seabld 64 Jan 15 20:23 99 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703pkb5UT lrwx------ 1 seabld seabld 64 Jan 15 20:23 990 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703LGADGX lrwx------ 1 seabld seabld 64 Jan 15 20:23 991 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703dfRJ4r lrwx------ 1 seabld seabld 64 Jan 15 20:23 992 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703HPmNu4 lrwx------ 1 seabld seabld 64 Jan 15 20:23 993 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703gu90ZF lrwx------ 1 seabld seabld 64 Jan 15 20:23 994 -> /dev/shm/queue/pulse/tmp/1358218872-2172-157033MsTtw lrwx------ 1 seabld seabld 64 Jan 15 20:23 995 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703on31Ya lrwx------ 1 seabld seabld 64 Jan 15 20:23 996 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703pnWCHE lrwx------ 1 seabld seabld 64 Jan 15 20:23 999 -> /dev/shm/queue/pulse/tmp/1358218872-2172-15703yTB0aJ thus our tmpfs: tmpfs 1037876 1037876 0 100% /dev/shm My first guess is something to do with pulse.
Running the following: /builds/buildbot/master01/bin/python2.7 /builds/buildbot/master01/tools/buildbot-helpers/pulse_publisher.py --passwords <passwordfile>
Once completed, I rebooted sea-master1. It's now back up and looks good. Waiting until all slaves have connected to the master before closing this.
All slaves attached (aside for sea-win32-01, which is down due to hw issues.)
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.