1399396 - Support code coverage builds on Android

Status: RESOLVED FIXED

(Testing :: Code Coverage, enhancement)

Description

[Marco Castelluccio [:marco]]

This should not be too different from Linux. I don't know much about the way tests work on Android though, I don't know if there are problems accessing the gcda files.

Comment 1

[Jan Henning [:JanH]]

(In reply to Marco Castelluccio [:marco] from comment #0)
> This should not be too different from Linux.

Out of curiosity, what about the Android-specific Java code?

Comment 2

[Marco Castelluccio [:marco]]

(In reply to Jan Henning [:JanH] from comment #1)
> (In reply to Marco Castelluccio [:marco] from comment #0)
> > This should not be too different from Linux.
> 
> Out of curiosity, what about the Android-specific Java code?

It will be different, this is only about the C/C++ code.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I think given the lack of Mozilla development resources on Android, we should assume Android has a lower priority than desktop and only work on it once we have more coverage on desktop.

Comment 4

[Marco Castelluccio [:marco]]

Attachment: WIP

We will require r17 in order to build the coverage build because of https://github.com/android-ndk/ndk/issues/411.

Here's a WIP patch.

Comment 5

[Gabriel Vîjială [:gabriel-v]]

We've switched from NDK's clang to our own: https://bugzilla.mozilla.org/show_bug.cgi?id=1435409#c4.

I've rebased and re-applied your patch with the new clang, here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=66a501ea5a6f3e937977fb8f0673c56fc876205a.

Build is failing with:

    configure:1496:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
    main(){return(0);}
    ^
    1 warning generated.
    /builds/worker/workspace/build/src/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux
-androideabi/bin/ld: error: cannot open /builds/worker/workspace/build/src/clang/lib/clang/6.0.1/lib/linux/libclang_rt.profile-arm-android.a: No such file or directory
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_writeout: error: undefined reference to 'llvm_gcda_start_file'
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_writeout: error: undefined reference to 'llvm_gcda_emit_function'
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_writeout: error: undefined reference to 'llvm_gcda_emit_arcs'
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_writeout: error: undefined reference to 'llvm_gcda_summary_info'
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_writeout: error: undefined reference to 'llvm_gcda_end_file'
    /tmp/conftest-7071d9.o:conftest.c:function __llvm_gcov_init: error: undefined reference to 'llvm_gcov_init'
    clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)
    configure: failed program was:

    #line 1494 "configure"
    #include "confdefs.h"

    main(){return(0);}
    configure: error: installation or configuration problem: C compiler cannot create executables.


It appears we don't have compiler-rt installed in that environment.

What are the next steps here?

Comment 6

[Marco Castelluccio [:marco]]

We are using the linux64-clang toolchain for this build (cross-compiling), so we only have the linux compiler-rt libraries.
We need to find a way to build them for Android too (either in the same toolchain build, or in a new one we define).
The script used to build Clang/LLVM/compiler-rt is at build/build-clang/build-clang.py.

Comment 7

[Kagami Rosylight 🏳️‍🌈🏳️‍⚧️ [:saschanaz] (they/them)]

Looks like we had ccov and then removed it? Right now we don't get C++ coverage from Android - e.g. widget/android/nsAppShell.cpp gets zero coverage data.

Comment 8

[Nicolas Guichard]

Attachment: Bug 1399396 - Remove leftover Java code coverage stuff. r?nalexander!

Java code coverage was removed in bug 1536044 because it was broken. We
still want to have at least C++ coverage for Android so this removes
the Java leftovers to re-enable the C++ coverage in the next commit.

Comment 9

[Nicolas Guichard]

Attachment: Bug 1399396 - Run tests with code coverage on Android. r?nalexander!

This only enables code coverage for the C++ bits for now.

For Rust I think we would need to add a -dev toolchain for Android to
be able to build with -Zprofile.

For Java we probably need something like JaCoCo.

Comment 10

[Nicolas Guichard]

Attachment: Bug 1399396 - repack_rust.py: fix providing multiple targets. r?glandium!

x.py takes a single comma-separated --target argument

Comment 11

[Nicolas Guichard]

Attachment: Bug 1399396 - Add Android Rust dev toolchain. r?glandium!

Instrumenting Rust code to gather test coverage data currently requires
compiling with -Zprofile which requires building our own patched Rust
toolchain.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/9360841b5206
https://hg.mozilla.org/integration/autoland/rev/6606f60833f6
Remove leftover Java code coverage stuff. r=nalexander,taskgraph-reviewers,bhearsum
https://github.com/mozilla-firefox/firefox/commit/5b146cdc4dbf
https://hg.mozilla.org/integration/autoland/rev/189e77457843
repack_rust.py: fix providing multiple targets. r=glandium
https://github.com/mozilla-firefox/firefox/commit/cee547537311
https://hg.mozilla.org/integration/autoland/rev/260a6d407689
Add Android Rust dev toolchain. r=glandium
https://github.com/mozilla-firefox/firefox/commit/bc709a870410
https://hg.mozilla.org/integration/autoland/rev/e5524d571767
Run tests with code coverage on Android. r=nalexander,geckoview-reviewers,taskgraph-reviewers,bhearsum

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/6606f60833f6
https://hg.mozilla.org/mozilla-central/rev/189e77457843
https://hg.mozilla.org/mozilla-central/rev/260a6d407689
https://hg.mozilla.org/mozilla-central/rev/e5524d571767

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/6606f60833f6
https://hg.mozilla.org/mozilla-central/rev/189e77457843
https://hg.mozilla.org/mozilla-central/rev/260a6d407689
https://hg.mozilla.org/mozilla-central/rev/e5524d571767

Comment 15

[Nicolas Guichard]

I missed something and the coverage-instrumented tests aren't actually being run on mozilla-central pushes yet. target_tasks_mozilla_central in target_tasks.py appears to be filtering them out.

Comment 16

[Nicolas Guichard]

Attachment: Bug 1399396 - Actually run Android CCov tests on mc push. r=#taskgraph-reviewers!

In my Try runs the tests ran but they didn't once my changes landed in
m-c. It appears they were filtered out here.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/bd500482918d
https://hg.mozilla.org/integration/autoland/rev/3cf5c26989c2
Actually run Android CCov tests on mc push. r=taskgraph-reviewers,jcristau

Comment 18

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/3cf5c26989c2

Comment 19

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/3cf5c26989c2