1661406 - "Could not determine Android NDK directory" for mobile artifact build

Status: VERIFIED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Trying to build GeckoView in artifact mode locally raises the following error when executing mach build:

 0:12.63 FAILURE: Build failed with an exception.
 0:12.63 * Where:
 0:12.63 Script '/Users/henrik/code/gecko/mobile/android/gradle/with_gecko_binaries.gradle' line: 55
 0:12.63 * What went wrong:
 0:12.63 A problem occurred configuring project ':geckoview'.
 0:12.63 > Could not create task ':geckoview:syncLibsFromDistDirForWithGeckoBinariesDebug'.
 0:12.64    > Could not create task of type 'SyncLibsAndUpdateGeneration'.
 0:12.64       > Could not determine Android NDK directory.  Set `ndk.dir` in `/Users/henrik/code/gecko/local.properties`
 0:12.64 * Try:
 0:12.64 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
 0:12.64 * Get more help at https://help.gradle.org
 0:12.64 BUILD FAILED in 2s
 0:13.10 Traceback (most recent call last):
 0:13.10   File "/Users/henrik/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
 0:13.10     return _run_code(code, main_globals, None,
 0:13.10   File "/Users/henrik/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
 0:13.10     exec(code, run_globals)
 0:13.10   File "/Users/henrik/code/gecko/python/mozbuild/mozbuild/action/file_generate.py", line 124, in <module>
 0:13.10     sys.exit(main(sys.argv[1:]))
 0:13.10   File "/Users/henrik/code/gecko/python/mozbuild/mozbuild/action/file_generate.py", line 72, in main
 0:13.10     ret = module.__dict__[method](output, *args.additional_arguments, **kwargs)
 0:13.10   File "/Users/henrik/code/gecko/mobile/android/gradle.py", line 57, in assemble_app
 0:13.10     return android('assemble-app')
 0:13.10   File "/Users/henrik/code/gecko/mobile/android/gradle.py", line 51, in android
 0:13.10     subprocess.check_call(cmd, env=env)
 0:13.10   File "/Users/henrik/.pyenv/versions/3.8.2/lib/python3.8/subprocess.py", line 364, in check_call
 0:13.10     raise CalledProcessError(retcode, cmd)
 0:13.10 subprocess.CalledProcessError: Command '['/Users/henrik/code/gecko/obj/mobile/_virtualenvs/init_py3/bin/python', '/Users/henrik/code/gecko/mach', 'android', 'assemble-app']' returned non-zero exit status 1.
 0:13.11 make[3]: *** [mobile/android/base/.deps/android_apks.stub] Error 1
 0:13.11 make[2]: *** [export] Error 2
 0:13.12 make[1]: *** [default] Error 2
 0:13.12 make: *** [build] Error 2

I also run mach bootstrap, but there is no Android NDK installed or configured for mach build. It's the first time I see this now, and it was working some weeks ago.

Running a mach clobber, removing the obj dir, and the mach virtualenv didn't help.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Just noticed that the NDK is only getting installed by mach bootstrap these days when you aren't in artifact mode:

https://hg.mozilla.org/integration/autoland/file/tip/python/mozboot/mozboot/osx.py#l373

Does it mean that something changed for the build process and that the NDK is required now?

Comment 2

[Mike Hommey [:glandium]]

No, it shouldn't be required.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Ok, so this is a side-effect from bug 1627796. Means it hasn't completely fixed the issue with artifact builds.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Nick, can you please have a look? Thanks.

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #4)

Nick, can you please have a look? Thanks.

Oh dear -- this is unexpected fallout from Bug 1661158. The local workaround is to back out Bug 1661158 and then Bug 1627796, and to accept my apologies.

The fix, which I will pursue shortly, is to warn in some way about this issue and to not try to work around the underlying Android-Gradle plugin issue when the NDK directory can't be found. For bonus points we could ensure that when we are doing local GeckoView substitution and the underlying Android-Gradle plugin issue will rear its ugly head, then we can work around.

If this turns out to be more complicated than I can manage today I'll ask for the tickets above to be backed out. NI to me so I don't forget.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

No worries. I temporarily installed the Android NDK to make it work. I assume for the final patch this requirement will not be added for artifacts build? Or will we be forced to download and install this 800MB package?

Comment 7

[Nick Alexander :nalexander [he/him]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #6)

No worries. I temporarily installed the Android NDK to make it work.

Yes, that will work around as well.

I assume for the final patch this requirement will not be added for artifacts build? Or will we be forced to download and install this 800MB package?

Correct, we will not start requiring the Android NDK for artifact builds.

Comment 8

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1661406 - Allow artifact builds without an NDK when working around Android-Gradle plugin bug substituting GeckoView. r?agi

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1661158

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0eb5fc0d1895
Allow artifact builds without an NDK when working around Android-Gradle plugin bug substituting GeckoView. r=agi,geckoview-reviewers

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/0eb5fc0d1895

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Works again. Thanks for the quick fix!