1248365 - Make Valgrind Mochitests run greener on Taskcluster

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Julian Seward [:jseward]]

Bug 1245566 adds Taskcluster scheduling for Mochitests on Valgrind,
but such tests largely run Orange at the moment.  This bug tracks work
needed to get to an initial green state, so that such runs can be
scheduled for production.

The required changes are expected to be some combination of extra
suppressions and bug fixes.

Comment 1

[Julian Seward [:jseward]]

Attachment: Suppressions and command-line-arg changes to reduce noise level

Without this patch, all 40 out of 40 mochitest tasks run orange on
taskcluster.  With it in place, at 19 out of 40 run green, for example
[1], and the remaining 21 do not have much noise.

The patch contains two components:

[a] new suppressions, largely for SSE functions to do with compositing

[b] A change of command line parameters:

    - so as not to trace into /usr/bin/lsb_release.  This runs Python,
      which creates a lot of noise because Valgrind doesn't interact well
      the Python implementation's memory management scheme.

    - changes the leak check report detail level from full to summary.
      This is because all runs report at least some leaks, and the 'full'
      mode printout makes the test harness orange-vs-not-orange detection
      always report "orange".  This isn't useful since we have other tools
      to detect leaks and it obscures runs which report uninitialised
      value errors, the detection of which is the main purpose of the runs.

Note that [a] affects both these new (mochitest-valgrind) runs and the
existing Valgrind short test in automation, although I don't think it
makes any difference to them because they don't report any of the
suppressed errors anyway.

Whereas [b] only applies to the new runs.  Flags for the existing
Valgrind short test are unaffected since they are set elsewhere.


[1]
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f5f13828f59d9e401ad2b79f7eefefb5862df007

Comment 2

[Nicholas Nethercote [inactive]]

Comment on attachment 8814405 [details] [diff] [review]
Suppressions and command-line-arg changes to reduce noise level

Review of attachment 8814405 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozdebug/mozdebug/mozdebug.py
@@ +285,5 @@
>  
>  
>  def get_default_valgrind_tool_specific_args():
>      return ['--partial-loads-ok=yes',
> +            '--leak-check=summary',

Should we just do --leak-check=no? The summary gives you just enough information to be worried but not enough to actually act... I'll let you decide.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/41a7afc03d20
Make Valgrind Mochitests run greener on Taskcluster.  r=n.nethercote.

Comment 4

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/41a7afc03d20