1334461 - run mochitest-a11y on ubuntu 16.04 instead of 12.04

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

to update our operating systems to be modern, we have been slowly converting our tests to run on 16.04!  This is great as there are updated libraries, kernels, and other tools.

For a11y, this works great except for ASAN where we have leaks.  In fact 13 different tests hit a leak (what appears to be the same leak).  We can consider disabling the tests or fixing the leak, I would vote for fixing the leak and then reassess what remains (ideally nothing).

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

in accessible/tests/mochitest/actions/test_general.xul [0], we have this leak:
[task 2017-01-26T18:33:54.679829Z] 18:33:54     INFO - =================================================================
[task 2017-01-26T18:33:54.679921Z] 18:33:54     INFO - ==1125==ERROR: LeakSanitizer: detected memory leaks
[task 2017-01-26T18:33:54.679968Z] 18:33:54     INFO - Direct leak of 40 byte(s) in 1 object(s) allocated from:
[task 2017-01-26T18:33:54.680030Z] 18:33:54     INFO -     #0 0x4b2654 in calloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:66:3
[task 2017-01-26T18:33:54.680097Z] 18:33:54     INFO -     #1 0x7fe144afb9ae in pa_xmalloc0 (/usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-8.0.so+0x149ae)
[task 2017-01-26T18:33:54.680144Z] 18:33:54     INFO - -----------------------------------------------------
[task 2017-01-26T18:33:54.680180Z] 18:33:54     INFO - Suppressions used:
[task 2017-01-26T18:33:54.680219Z] 18:33:54     INFO -   count      bytes template
[task 2017-01-26T18:33:54.680258Z] 18:33:54     INFO -     257       3493 libc.so
[task 2017-01-26T18:33:54.680300Z] 18:33:54     INFO -     803      25600 nsComponentManagerImpl
[task 2017-01-26T18:33:54.680344Z] 18:33:54     INFO -      53       7632 mozJSComponentLoader::LoadModule
[task 2017-01-26T18:33:54.680391Z] 18:33:54     INFO -       1        384 pixman_implementation_lookup_composite
[task 2017-01-26T18:33:54.680566Z] 18:33:54     INFO -     355      14021 libfontconfig.so
[task 2017-01-26T18:33:54.681844Z] 18:33:54     INFO -       1         32 libdl.so
[task 2017-01-26T18:33:54.682381Z] 18:33:54     INFO -      17       4348 libglib-2.0.so
[task 2017-01-26T18:33:54.682890Z] 18:33:54     INFO -       1         28 libresolv.so
[task 2017-01-26T18:33:54.683379Z] 18:33:54     INFO - -----------------------------------------------------
[task 2017-01-26T18:33:54.683882Z] 18:33:54     INFO - SUMMARY: AddressSanitizer: 40 byte(s) leaked in 1 allocation(s).
[task 2017-01-26T18:33:54.985689Z] 18:33:54     INFO - TEST-INFO | Main app process: exit 0
[task 2017-01-26T18:33:54.986641Z] 18:33:54     INFO - TEST-INFO | LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2017-01-26T18:33:54.987267Z] 18:33:54     INFO - TEST-INFO | LeakSanitizer | This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2017-01-26T18:33:54.987397Z] 18:33:54    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at pa_xmalloc0

[0] https://public-artifacts.taskcluster.net/Uuw3Io36QLyJxCDFxapeiQ/0/public/logs/live_backing.log

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

same for:
accessible/tests/mochitest/actions/test_keys_menu.xul
 - https://public-artifacts.taskcluster.net/RFieHFQuSruSUXLs6RLZSw/0/public/logs/live_backing.log

accessible/tests/mochitest/attributes/test_obj_group.xul:
 - https://public-artifacts.taskcluster.net/XECTmTymQaGLfESMhhaN0w/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_contextmenu.xul:
 - https://public-artifacts.taskcluster.net/dhuu13ffSbGYNIegWVZtqQ/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_focus_contextmenu.xul:
 - https://public-artifacts.taskcluster.net/UzF6NRB2Sb2_P7iNyz6uCg/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_focus_general.html
 - https://public-artifacts.taskcluster.net/PNWhasCsSXGflffYqEnGkQ/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_focus_general.xul:
 - https://public-artifacts.taskcluster.net/eXrcoQCgR4-zphm_FaJcxQ/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_focus_listcontrols.xul
 - https://public-artifacts.taskcluster.net/RgrV7zcnQnmGbX7FGaZFaQ/0/public/logs/live_backing.log

accessible/tests/mochitest/events/test_focus_menu.xul
 - https://queue.taskcluster.net/v1/task/BJWNImvGSum1_u6NoTQ4dA/runs/0/artifacts/public/logs/live_backing.log

accessible/tests/mochitest/states/test_expandable.xul
 - https://public-artifacts.taskcluster.net/Ll_8uLEST-6U89bLFPDi_A/0/public/logs/live_backing.log

accessible/tests/mochitest/treeupdate/test_contextmenu.xul
 - https://queue.taskcluster.net/v1/task/BufcgywNSFmAAh-nyk75_w/runs/0/artifacts/public/logs/live_backing.log

accessible/tests/mochitest/treeupdate/test_menu.xul
 - https://queue.taskcluster.net/v1/task/BLklut5fQmaEpW_dt6IAdA/runs/0/artifacts/public/logs/live_backing.log

accessible/tests/mochitest/treeupdate/test_menubutton.xul:
 - https://public-artifacts.taskcluster.net/WuKX4dIZRjmNg_Y6c6LBZA/0/public/logs/live_backing.log

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: run a11y mochitests on ubuntu 16.04 and skip leaking tests on asan

adding a patch if we choose to run on 16.04 and skip tests.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

:kats, you had interest in running on 16.04, possibly looking at this leak mentioned here would save us some work in other suites.

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Yup I can try to debug this. I'm doing a llvm build now so that I can run ASAN locally. Leaving needinfo on me for now.

That being said, wouldn't it be better to add the leaked block to the list of ignored leaks rather than to skip the tests entirely on ASAN builds?

Comment 6

[Kartikaya Gupta (email:[e-mail])]

... in fact, the leaks file already lists existing leaks in libpulsecommon [1]. I suspect it's just the version number of the library that has changed in going to 16.04, so the leak is showing up as "new" again.

[1] http://searchfox.org/mozilla-central/rev/8fa84ca6444e2e01fb405e078f6d2c8da0e55723/build/sanitizers/lsan_suppressions.txt#69

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

oh, interesting, I wasn't aware of lsan_suppressions.txt :)

I just pushed to try to test if updating this fixes the leak:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b00fc6e6e0b0e2fee72d9b4110d081cb312e312c

Comment 8

[Kartikaya Gupta (email:[e-mail])]

I just checked a recent asan tc-M(a11y) mozilla-inbound log [1] which shows this:

[task 2017-01-27T14:35:05.831468Z] 14:35:05     INFO - -----------------------------------------------------
[task 2017-01-27T14:35:05.831775Z] 14:35:05     INFO - Suppressions used:
[task 2017-01-27T14:35:05.832011Z] 14:35:05     INFO -   count      bytes template
.. [snip] ..
[task 2017-01-27T14:35:05.837231Z] 14:35:05     INFO -       1         40 libpulsecommon-1.1.so
.. [snip] ..

So the single 40-byte leak is in fact a pre-existing problem even on Ubuntu 12.04, and is getting ignored because of that leak suppression entry. I think it should be fine to just update the suppression file to ignore it in both libpulsecommon-1.1.so and libpulsecommon-8.0.so, and then remove the -1.1.so entry after all the jobs have been migrated to 16.04. Does that sound reasonable?

[1] https://public-artifacts.taskcluster.net/bFUW00FkTuOwPIhkpjQpgg/0/public/logs/live_backing.log

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

yes, let me confirm on the try push and that would be an ideal route forward- thanks for digging this up and helping move this bug forward :)

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: update libpulsecommon in lsan_suppressions.txt and run a11y on 16.04

Comment 11

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8831154 [details] [diff] [review]
update libpulsecommon in lsan_suppressions.txt and run a11y on 16.04

Review of attachment 8831154 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/sanitizers/lsan_suppressions.txt
@@ +66,5 @@
>  leak:libp11-kit.so
>  leak:libpixman-1.so
>  leak:libpulse.so
>  leak:libpulsecommon-1.1.so
> +leak:libpulsecommon-8.0.so

Maybe add a comment here saying that libpulsecommon-1.1 is on Ubuntu 12.04 and -8.0 is on 16.04?

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6a065534a5a2
run mochitest-a11y on ubuntu 16.04 instead of 12.04. r=kats

Comment 13

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/6a065534a5a2