Closed Bug 1084163 Opened 5 years ago Closed 5 years ago

[mozharness] When make check fails, other tests are not run

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

defect
Not set

Tracking

(firefox34 fixed, firefox35 fixed, firefox-esr31 fixed, b2g-v2.0 fixed)

RESOLVED FIXED
Tracking Status
firefox34 --- fixed
firefox35 --- fixed
firefox-esr31 --- fixed
b2g-v2.0 --- fixed

People

(Reporter: glandium, Assigned: mshal)

References

Details

Attachments

(1 file)

Before mozharness:
  https://treeherder.mozilla.org/ui/#/jobs?repo=elm&revision=e5c5caa646ec
make check happened after make upload, so tests could start, and make check failure turned the build orange.

With mozharness:
 https://treeherder.mozilla.org/ui/#/jobs?repo=elm&revision=e8f90a5158c5
make check happens before make upload somehow, so tests don't start.

I guess make check should have a dependency on make upload.
Component: General Automation → Mozharness
Depends on: 1087102
Depends on: 1087104
This just removes 'make check' from the automation/build step. We'll run it as a separate step in mozharness, due to the complications of either moving sendchange into automation/build (which is needed to trigger tests before 'mach build' returns).
Assignee: nobody → mshal
Attachment #8509918 - Flags: review?(mh+mozilla)
Attachment #8509918 - Flags: review?(mh+mozilla) → review+
(In reply to Michael Shal [:mshal] from comment #1)
> This just removes 'make check' from the automation/build step. We'll run it
> as a separate step in mozharness

Was that ever done? Is there a bug #?
It looks like it was added in http://hg.mozilla.org/build/mozharness/rev/42d035f84c63 as part of bug 1055918 (and a collection of other bugs).
(In reply to Michael Shal [:mshal] from comment #5)
> It looks like it was added in
> http://hg.mozilla.org/build/mozharness/rev/42d035f84c63 as part of bug
> 1055918 (and a collection of other bugs).

Well, make check doesn't run on elm, so it's either not true, or something broke since then.
jlund, any idea why make check wouldn't be running on elm?
Flags: needinfo?(jlund)
(In reply to Michael Shal [:mshal] from comment #7)
> jlund, any idea why make check wouldn't be running on elm?

hmm, so https://tbpl.mozilla.org/php/getParsedLog.php?id=51749740&full=1&branch=elm shows that make -k check was run. it even highlights the errors:

21:21:45  WARNING - TEST-UNEXPECTED-FAIL | GTest unit test: failed
21:21:45     INFO - Finished running GTest tests.
21:21:45  WARNING - gtest TEST-UNEXPECTED-FAIL | gtest | test failed with return code 1
21:21:45     INFO - make[1]: *** [check] Error 1
21:21:45     INFO - make[1]: Leaving directory `/builds/slave/elm-lx-00000000000000000000000/build/src/obj-firefox/testing/gtest'
21:21:45     INFO - make: *** [check] Error 2
21:21:45     INFO - PASSED
21:21:46    ERROR - Return code: 2
21:21:46     INFO - TinderboxPrint: check<br/>1810/<em class="testfail">4</em>/0

though the overall build is not orange. *sigh* that's a bug. /me files https://bugzil.la/1093911
Flags: needinfo?(jlund)
adding the following to an upcoming ash run:

diff --git a/mozharness/mozilla/building/buildbase.py b/mozharness/mozilla/building/buildbase.py
index 8beecae..8888e57 100755
--- a/mozharness/mozilla/building/buildbase.py
+++ b/mozharness/mozilla/building/buildbase.py
@@ -1431,11 +1431,18 @@ or run without that action (ie: --no-{action})"

         parser = CheckTestCompleteParser(config=c,
                                          log_obj=self.log_obj)
-        self.run_command_m(command=cmd,
-                           cwd=dirs['abs_obj_dir'],
-                           env=env,
-                           output_parser=parser)
+        return_code = self.run_command_m(command=cmd,
+                                       cwd=dirs['abs_obj_dir'],
+                                       env=env,
+                                       output_parser=parser)
         parser.evaluate_parser()
+        if return_code:
+            self.return_code = self.worst_level(
+                EXIT_STATUS_DICT[TBPL_WARNING], self.return_code,
+                AUTOMATION_EXIT_CODES[::-1]
+            )
+            self.error("'make -k check' did not run successfully. Please check "
+                       "log for errors.")
You need to log in before you can comment on or make changes to this bug.