fwunit is failing, but the messages are too big

RESOLVED FIXED

Status

Infrastructure & Operations
RelOps
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: dustin, Assigned: dustin)

Tracking

Details

(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/4142] )

Nov 16 03:53:15 fwunit1.private.releng.scl3.mozilla.com postfix/postdrop[14057]: warning: uid=2918: File too large
Nov 16 03:53:15 fwunit1.private.releng.scl3.mozilla.com postfix/sendmail[14055]: fatal: root(2918): message file too big

Most likely, the crontask output should just say "good" or "bad", with the details left to the person trying to fix.
If possible I'd advocate for something like:

Passed Tests: 1200
Failed Tests:
 * test_aws_masters_to_product_delivery
 * test_scl3_masters_to_product_delivery

(aka: list the tests that failed, not the innermost details of them)

That can give us actionable, "what may have broken" without giving us lots of contextual spam that clogs postfix/inboxes.
Sounds good -- want to write up a nosetests plugin to do that?
Flags: needinfo?(bugspam.Callek)
Never wrote one, and have a lot of other priorities on my plate atm, but I'm not opposed to it; just would fall waaay down on my list of tasks atm.
Flags: needinfo?(bugspam.Callek)

Updated

4 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/4142]
commit df807496547832c6beec77e6f09c4e4a3f12572e
Author: Dustin J. Mitchell <dustin@mozilla.com>
Date:   Thu Nov 20 15:40:38 2014 +0000

    Bug 1100576: just include pass/fail in the cron mail from fwunit

diff --git a/crontask.sh b/crontask.sh
index 7a4c48f..b4a5dd2 100644
--- a/crontask.sh
+++ b/crontask.sh
@@ -19,4 +19,7 @@ fwunit

 # and run the tests
 cd ${BASE}/src/tests
-nosetests 2>&1
+if ! nosetests 2>/dev/null; then
+    echo "Tests failed (output suppressed)"
+    exit 1
+fi
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.