794790 - Make TalosFactory chmod steps less verbose

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ed Morley [:emorley]]

Background to bug 765224: Many of the buildbot logs are extremely verbose even in the case of success, causing larger log sizes, thereby meaning slower TBPL parsing (and of late, timeouts) - as well as making it harder for someone to look at the log and ascertain what actually failed without trying a bunch of searches.

One place I think we can cut back is in our uses of chmod, eg:

{
========= Started chmod files (see msys bug) (results: 0, elapsed: 0 secs) (at 2012-09-27 01:05:56.205492) =========
'chmod' '-v' '-R' 'a+rwx' '.'
 in dir c:\talos-slave\test\../talos-data (timeout 1200 secs)
 watching logfiles {}
 argv: ['chmod', '-v', '-R', 'a+rwx', '.']
 environment:
<snip>
mode of `.' changed to 0777 (rwxrwxrwx)
mode of `./count_and_reboot.py' changed to 0777 (rwxrwxrwx)
mode of `./firefox' changed to 0777 (rwxrwxrwx)
mode of `./firefox/AccessibleMarshal.dll' changed to 0777 (rwxrwxrwx)
mode of `./firefox/application.ini' changed to 0777 (rwxrwxrwx)
mode of `./firefox/blocklist.xml' changed to 0777 (rwxrwxrwx)
mode of `./firefox/breakpadinjector.dll' changed to 0777 (rwxrwxrwx)
mode of `./firefox/chrome.manifest' changed to 0777 (rwxrwxrwx)
mode of `./firefox/components' changed to 0777 (rwxrwxrwx)
mode of `./firefox/components/binary.manifest' changed to 0777 (rwxrwxrwx)
mode of `./firefox/components/browsercomps.dll' changed to 0777 (rwxrwxrwx)
mode of `./firefox/crashreporter-override.ini' changed to 0777 (rwxrwxrwx)
mode of `./firefox/crashreporter.exe' retained as 0777 (rwxrwxrwx)
}

Comment 1

[Ed Morley [:emorley]]

Attachment: Don't use chmod -v

Comment 2

[Chris AtLee [:catlee]]

Some context as to why we've run with verbose settings here, and other related steps.

- When something goes wrong, having as much information as possible about what happened on the machine is important. Often times you can't reproduce issues reliably, and it may be impossible from non-verbose logs to determine what the problem was.

- Adding verbose logging keeps buildbot from killing jobs that run for a long time without generating output.

Comment 3

[Ed Morley [:emorley]]

(In reply to Chris AtLee [:catlee] from comment #2)
> - When something goes wrong, having as much information as possible about
> what happened on the machine is important. Often times you can't reproduce
> issues reliably, and it may be impossible from non-verbose logs to determine
> what the problem was.

Agreed; though the instances where the errors (which will still get displayed with non verbose) aren't clear without context are hopefully less frequent - and I've only removed verbose mode for some of the simpler (yet log spamming) steps. We can always add them back in if we have any problems. (Even better would be if buildbot had a way of not displaying stdout if the exit code was zero; then we could just leave as verbose in case the extra context was needed).

> - Adding verbose logging keeps buildbot from killing jobs that run for a
> long time without generating output.

All of the steps for which I have removed verbose output have been under a minute (and some a matter of seconds) :-)

Comment 4

[Ed Morley [:emorley]]

Comment on attachment 665321 [details] [diff] [review]
Don't use chmod -v

http://hg.mozilla.org/build/buildbotcustom/rev/bb63a869fb76