1370242 - Fix AWSY test_memory_usage.py failure on linux64-ccov.

Status: RESOLVED FIXED

(Testing :: Code Coverage, defect)

Description

[Greg Mierzwinski [:sparky]]

Attachment: live_backing_awsy_test_memory_usage_py_failure.log

This bug is for the work needed to be done to ensure that all the tests in the AWSY test suite are passing (or can be skipped). Currently, there is only one failure with the file test_memory_usage.py during the test_open_tabs test.

I've attached a log to the error. [1], and [2] have some debugging information available within their logs.

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d7ba3c5dd60bdbac0d51d749cab35a0348bcacbb

[2]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=343218ff6c942306ee5d1e07eb4945e515b64daa&selectedJob=104551423

Comment 1

[Greg Mierzwinski [:sparky]]

Here's a link to a log which contains more debugging information [0]. I've found that there seems to be items found at [1] but then at [2], nothing is recorded because totals_heap_allocated is empty.

[0]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=343218ff6c942306ee5d1e07eb4945e515b64daa
[1]: https://dxr.mozilla.org/mozilla-central/source/testing/awsy/awsy/parse_about_memory.py#22
[2]: https://dxr.mozilla.org/mozilla-central/source/testing/awsy/awsy/parse_about_memory.py#38

:bc, would you know why totals_heap_allocated is empty now? It contains information when we don't have GCOV code coverage enabled and that's the only difference that I know off.

If it's expected, code coverage doesn't need to have the tests upload perf data so we can disable that part of the test whenever code coverage is on.

Comment 2

[Bob Clary [:bc] (inactive)]

erahm: Any thoughts on why these measurements would be empty when GCOV is enabled?

Comment 3

[Eric Rahm [:erahm]]

(In reply to Bob Clary [:bc:] from comment #2)
> erahm: Any thoughts on why these measurements would be empty when GCOV is
> enabled?

gcov or lcov or ccov or whatever also broke the arena allocator tests (bug 1353882, comment 6). I think they're hooking/patching some memory functions and breaking them. If we can isolate this we should probably file an upstream bug.

Comment 4

[Bob Clary [:bc] (inactive)]

We could add command line options to testing/mozharness/scripts/awsy_script.py and testing/awsy/mach_commands.py to pass info to testing/awsy/awsy/test_memory_usage.py via the testvars json file which TestMemoryUsage.tearDown could use to skip trying to process the data.

Comment 5

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1370242 - Prevent performance data from being uploaded when collecting code coverage.

This patch is to prevent performance data from being collected when code coverage is running. It uses the environment variables that are set when code coverage is being collected to detect when it shouldn't gather the data.

Review commit: https://reviewboard.mozilla.org/r/146698/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/146698/

Comment 6

[Greg Mierzwinski [:sparky]]

This patch disables gathering performance data, and you can see an example of it's use here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9934fe84dfbfb0d5aefc62cfed6b0a0e7dc4d992

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8875293 [details]
Bug 1370242 - Prevent performance data from being uploaded when collecting code coverage.

https://reviewboard.mozilla.org/r/146698/#review150836

nice and simple

Comment 8

[Eric Rahm [:erahm]]

I still think someone working on the ccov builds should figure out why this is failing, it's the second instance of ccov breaking memory related tests (that I know of).

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4aaf7ad4fef8
Prevent performance data from being uploaded when collecting code coverage. r=jmaher

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

:gmierz, can you file a follow up bug pointing to the two memory issues we have hacked around for ccov?

Comment 11

[Greg Mierzwinski [:sparky]]

It has been made and is one of the blockers for this bug.

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4aaf7ad4fef8