661409 - Turn off emails for successful try builds or for all builds (with try syntax)

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Lukas Blakk [:lsblakk] use ?needinfo]

This came up in a discussion with mconnor today - why not only email on fail for try?  So I think it would be possible to adjust http://hg.mozilla.org/build/buildbotcustom/file/532be9b13696/bin/try_mailer.py#l168 to not send mail on success, or alternately/as well to allow for a try-syntax flag about email desires that is handled in the try_mailer (instead of in try_parser) that allows you to disable emails altogether or just the successful ones: 

-f --fail-only  
-n --no-emails

http://mxr.mozilla.org/build/source/buildbotcustom/status/generators.py would make sure you still get your email on push that shows you the link where your builds and your tbpl results will be.

Comment 1

[Mike Connor [:mconnor]]

Can we default to -f, and have those who want all 186 emails specify --all-emails ?

Comment 2

[Lukas Blakk [:lsblakk] use ?needinfo]

why yes, yes we could - I like how you think!

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [needs testing] only send email on failures

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [tested] try_mailer only mails on non-successful build by default, checks syntax for -e or -n flags

Ran this through staging, am able to get all/none/or only non-success emails as per flags:

-e --all-emails
-n --no-emails

with the default now being only getting emails when something does not succeed.

Comment 5

[Chris AtLee [:catlee]]

Comment on attachment 537971 [details] [diff] [review]
[tested] try_mailer only mails on non-successful build by default, checks syntax for -e or -n flags

>+    tm_parser = ArgumentParser()
>+    tm_parser.add_argument("-e", "--all-emails", dest="all_emails", help="request all emails", action="store_true")
>+    tm_parser.add_argument("-n", "--no-emails", dest="silence", help="request no emails at all", action="store_true")
>+    tm_parser.set_defaults(
>+        all_emails=False,
>+        silence=False,
>+        )
>+
>     builder_path, build_number = sys.argv[-2:]

Is there a reason this is using sys.argv rather than args?

>     build = getBuild(builder_path, build_number)
> 
>+    # check the commit message for syntax regarding email prefs
>+    comment_args = build.source.changes[-1].comments.split("try: ")[1].split()
>+    tm_options, args = tm_parser.parse_known_args(comment_args)

I know that we're going to be requiring try syntax when this is deployed, but I'd be happier if this was written to not require it. If "try: " isn't found in your comments, then .split("try: ")[1] will raise a KeyError.

Looks good other than that.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [tested] try_mailer only mails on non-successful build by default, checks syntax for -e or -n flags

Thanks for catching that - added checking for "try: " and empty "" if not in the comments. Also ran a quick test on staging-master using args instead of sys.argv and it looks good.

Comment 7

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 538007 [details] [diff] [review]
[tested] try_mailer only mails on non-successful build by default, checks syntax for -e or -n flags

landed on default:
http://hg.mozilla.org/build/buildbotcustom/rev/789d1a28cd8e

Comment 8

[Lukas Blakk [:lsblakk] use ?needinfo]

This is now in production, closing.

Comment 9

[Aki Sasaki (not active)]

Backed out http://hg.mozilla.org/build/buildbotcustom/rev/789d1a28cd8e at Catlee's request due to lots of try_mailer exceptions.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

So looks like this is an example of the exceptions that were seen:

2011-06-10 03:23:33-0700 [Broker,1411,10.2.91.156] Log output: Running ['/tools/buildbot-0.8.2/bin/python', '/tools/buildbotcustom-0.8.0/buildbotcustom/bin/log_uploader.py', 'stage.mozilla.org', '-u', 'trybld', '-i', '/home/cltbld/.ssh/trybld_dsa', '-b', 'try', '-p', 'linux-debug', '--product', 'firefox', '--try', '/builds/buildbot/try-trunk-master/try-linux-debug', '994']
	Running post_upload.py -b try -p firefox --revision 7740d33931eb --who [e-mail] --builddir try-linux-debug --release-to-try-builds /tmp/tmp.EWnwo14004 /tmp/tmp.EWnwo14004/try-linux-debug-build994.txt.gz
	http://stage.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-7740d33931eb/try-linux-debug/try-linux-debug-build994.txt.gz
	
	Traceback (most recent call last):
	  File "/tools/buildbotcustom-0.8.0/buildbotcustom/bin/try_mailer.py", line 198, in <module>
	    options.to.append(msgdict['author'])
	TypeError: 'NoneType' object is unsubscriptable
	
2011-06-10 03:23:33-0700 [Broker,1411,10.2.91.156] Unhandled Error
	Traceback (most recent call last):
	  File "/tools/python-2.6.5/lib/python2.6/threading.py", line 484, in run
	    self.__target(*self.__args, **self.__kwargs)
	  File "/tools/buildbot-0.8.2/lib/python2.6/site-packages/twisted/python/threadpool.py", line 207, in _worker
	    result = context.call(ctx, function, *args, **kwargs)
	  File "/tools/buildbot-0.8.2/lib/python2.6/site-packages/twisted/python/context.py", line 59, in callWithContext
	    return self.currentContext().callWithContext(ctx, func, *args, **kw)
	  File "/tools/buildbot-0.8.2/lib/python2.6/site-packages/twisted/python/context.py", line 37, in callWithContext
	    return func(*args,**kw)
	--- <exception caught here> ---
	  File "/tools/buildbotcustom-0.8.0/buildbotcustom/status/log_handlers.py", line 88, in handleLogs
	    subprocess.check_call(cmd, stdout=output, stderr=subprocess.STDOUT)
	  File "/tools/python-2.6.5/lib/python2.6/subprocess.py", line 498, in check_call
	    raise CalledProcessError(retcode, cmd)
	subprocess.CalledProcessError: Command '['/tools/buildbot-0.8.2/bin/python', '/tools/buildbotcustom-0.8.0/buildbotcustom/bin/try_mailer.py', 'stage.mozilla.org', '-u', 'trybld', '-i', '/home/cltbld/.ssh/trybld_dsa', '-b', 'try', '-p', 'linux-debug', '-f', '[e-mail]', '--to-author', '--product', 'firefox', '--try', '/builds/buildbot/try-trunk-master/try-linux-debug', '994']' returned non-zero exit status 1

Comment 11

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: v.2 - now with the author being set inside the msgdict != None

Put the author setting into the msgdict != None so there's never an attempt to set author on a NoneType object (which was causing the exception).

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 538818 [details] [diff] [review]
v.2 - now with the author being set inside the msgdict != None

http://hg.mozilla.org/build/buildbotcustom/rev/0649ccb25c3f on default

Comment 13

[Lukas Blakk [:lsblakk] use ?needinfo]

This has been re-enabled in production as of yesterday.