493752 - Reduce buildbot logging

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Chris AtLee [:catlee]]

Attachment: Disable maybeStartBuild messages

buildbot logs way too much stuff to twistd.log.  The log file is rotated every few minutes on production-master.

Comment 1

[Chris AtLee [:catlee]]

Comment on attachment 378366 [details] [diff] [review]
Disable maybeStartBuild messages

Disabling maybeStartBuild messages reduces the size of the logs by around 75%

Comment 2

[Chris AtLee [:catlee]]

We could also use daily log files instead of log files capped at a certain size limit:

application = service.Application('buildmaster')

from twisted.python.log import ILogObserver, FileLogObserver
from twisted.python.logfile import DailyLogFile

logfile = DailyLogFile("twistd.log", "logs")
application.setComponent(ILogObserver, FileLogObserver(logfile).emit)

BuildMaster(basedir, configfile).setServiceParent(application)

Comment 3

[[e-mail] (:bhearsum)]

(In reply to comment #2)
> We could also use daily log files instead of log files capped at a certain size
> limit:
> 
> application = service.Application('buildmaster')
> 
> from twisted.python.log import ILogObserver, FileLogObserver
> from twisted.python.logfile import DailyLogFile
> 
> logfile = DailyLogFile("twistd.log", "logs")
> application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
> 
> BuildMaster(basedir, configfile).setServiceParent(application)

I think this is a great idea.

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 378366 [details] [diff] [review]
Disable maybeStartBuild messages

this is made of win

Comment 5

[Chris AtLee [:catlee]]

Comment on attachment 378366 [details] [diff] [review]
Disable maybeStartBuild messages

changeset:   40:b1bef83b3c4f

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #3)
> (In reply to comment #2)
> > We could also use daily log files instead of log files capped at a certain size
> > limit:
> > 
> > application = service.Application('buildmaster')
> > 
> > from twisted.python.log import ILogObserver, FileLogObserver
> > from twisted.python.logfile import DailyLogFile
> > 
> > logfile = DailyLogFile("twistd.log", "logs")
> > application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
> > 
> > BuildMaster(basedir, configfile).setServiceParent(application)
> 
> I think this is a great idea.

This doesn't work with twisted 8.0.1

Comment 7

[Axel Hecht [:Pike]]

Fixing this might give perf room to poll l10n repos more often, which is bug 498654.

Comment 8

[Chris Cooper [:coop] (he/him)]

I was forced to restart production-master this morning when l10n nightly updates + places project brought it to a standstill, so I pull in this patch on the restart.

Comment 9

[Chris Cooper [:coop] (he/him)]

I had to revert back to the previously running changeset (30). There were 12 new changesets of which this patch was only one, and something was seriously tanking the waterfall, pollers went missing, etc.

Comment 10

[Chris AtLee [:catlee]]

This is now running on our production buildbot masters.

Comment 11

[Axel Hecht [:Pike]]

Do we have an upstream ticket for this?