Closed Bug 1844838 Opened 2 years ago Closed 2 years ago

Android build fails on Windows when triggered outside MozillaBuild

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86_64
Windows 11
Type:
defect

Tracking

(firefox117 fixed)

Status:
RESOLVED FIXED
Milestone:
117 Branch
Tracking Flags:
Tracking Status
firefox117 --- fixed

People

(Reporter: saschanaz, Assigned: saschanaz)

Details

Attachments

(1 file)

Bug 1844838 - Use default python3 outside MozillaBuild on Windows r=m_kato
48 bytes, text/x-phabricator-request
Details | Review

Mozconfig:

mk_add_options AUTOCLOBBER=1
ac_add_options --disable-optimize

# Build GeckoView/Firefox for Android:
ac_add_options --enable-project=mobile/android
ac_add_options --target=x86_64

Build log:

> ./mach build
 0:01.15 W Clobber not needed.
 0:01.38 W Adding make options from C:\Users\sasch\Documents\GitHub\gecko-dev\mozconfig-android-x86-64
    AUTOCLOBBER=1
    MOZ_OBJDIR=C:/Users/sasch/Documents/GitHub/gecko-dev/obj-x86_64-unknown-linux-android
    OBJDIR=C:/Users/sasch/Documents/GitHub/gecko-dev/obj-x86_64-unknown-linux-android
    FOUND_MOZCONFIG=C:/Users/sasch/Documents/GitHub/gecko-dev/mozconfig-android-x86-64
    export FOUND_MOZCONFIG
  Parallelism determined by memory: using 64 jobs for 64 cores based on 63.9 GiB RAM and estimated job size of 1.0 GiB
 0:01.40 C:/Users/sasch/.mozbuild/mozmake/mozmake.exe -f client.mk -j64 -s
 0:04.81 Elapsed: 0.00s; From dist/private: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:04.81 Elapsed: 0.00s; From dist/public: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:04.85 Elapsed: 0.05s; From dist/xpi-stage: Kept 0 existing; Added/updated 56; Removed 0 files and 0 directories.
 0:05.28 Elapsed: 0.47s; From dist/bin: Kept 0 existing; Added/updated 1072; Removed 0 files and 0 directories.
 0:05.55 Elapsed: 0.73s; From _tests: Kept 0 existing; Added/updated 1317; Removed 0 files and 0 directories.
 0:07.09 Elapsed: 2.28s; From dist/include: Kept 0 existing; Added/updated 6894; Removed 0 files and 0 directories.
 0:07.38 ./GeneratedElementDocumentState.h.stub
 0:07.38 ./data_encoding_ffi_generated.h.stub
 0:07.39 ./mime_guess_ffi_generated.h.stub
 0:07.39 ./audioipc2_client_ffi_generated.h.stub
 0:07.39 ./audioipc2_server_ffi_generated.h.stub
 0:07.39 ./origin_trials_ffi_generated.h.stub
 0:07.39 ./webrender_ffi_generated.h.stub
 0:07.41 ./wgpu_ffi_generated.h.stub
 0:07.41 ./mapped_hyph.h.stub
 0:07.41 ./fluent_ffi_generated.h.stub
 0:07.41 ./l10nregistry_ffi_generated.h.stub
 0:07.42 ./localization_ffi_generated.h.stub
 0:07.42 ./fluent_langneg_ffi_generated.h.stub
 0:07.42 ./unic_langid_ffi_generated.h.stub
 0:07.42 ./IsCombiningDiacritic.h.stub
 0:07.45 ./ServoStyleConsts.h.stub
 0:07.47 ./mp4parse_ffi_generated.h.stub
 0:07.49 ./android_apks.stub
 0:07.50 ./MozURL_ffi.h.stub
 0:07.50 ./rust_helper.h.stub
 0:07.50 ./neqo_glue_ffi_generated.h.stub
 0:07.52 ./fog_ffi_generated.h.stub
 0:07.52 ./jog_ffi_generated.h.stub
 0:07.53 ./dap_ffi_generated.h.stub
 0:07.55 ./profiler_ffi_generated.h.stub
 0:07.57 ./gk_rust_utils_ffi_generated.h.stub
 0:07.58 widget/android/generated_jni_wrappers.stub
 0:08.17 toolkit/components/glean/GleanMetrics.h.stub
 0:08.17 toolkit/components/glean/GleanPings.h.stub
 0:09.54  0:00.84 C:/Users/sasch/Documents/GitHub/gecko-dev/gradlew.bat geckoview:generateJNIWrappersForGeneratedWithGeckoBinariesDebug -x lint
 0:09.57 Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=utf-8
 0:10.54 FAILURE: Build failed with an exception.
 0:10.54 * Where:
 0:10.54 Settings file 'C:\Users\sasch\Documents\GitHub\gecko-dev\settings.gradle' line: 33
 0:10.54 * What went wrong:
 0:10.54 A problem occurred evaluating settings 'gecko-dev'.
 0:10.54 > Could not run ./mach environment. Try running ./mach build first.
 0:10.54 * Try:
 0:10.54 > Run with --stacktrace option to get the stack trace.
 0:10.54 > Run with --info or --debug option to get more log output.
 0:10.55 > Run with --scan to get full insights.
 0:10.55 * Get more help at https://help.gradle.org
 0:10.55 BUILD FAILED in 896ms
 0:11.15 Traceback (most recent call last):
 0:11.15   File "<frozen runpy>", line 198, in _run_module_as_main
 0:11.16   File "<frozen runpy>", line 88, in _run_code
 0:11.16   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\file_generate.py", line 155, in <module>
 0:11.16     sys.exit(log_build_task(main, sys.argv[1:]))
 0:11.16              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:11.16   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\util.py", line 16, in log_build_task
 0:11.16     return f(*args, **kwargs)
 0:11.16            ^^^^^^^^^^^^^^^^^^
 0:11.16   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\file_generate.py", line 99, in main
 0:11.16     ret = module.__dict__[method](
 0:11.16           ^^^^^^^^^^^^^^^^^^^^^^^^
 0:11.16   File "C:/Users/sasch/Documents/GitHub/gecko-dev/mobile/android/gradle.py", line 52, in assemble_app
 0:11.16     return android("assemble-app")
 0:11.16            ^^^^^^^^^^^^^^^^^^^^^^^
 0:11.16   File "C:/Users/sasch/Documents/GitHub/gecko-dev/mobile/android/gradle.py", line 46, in android
 0:11.16     subprocess.check_call(cmd, env=env)
 0:11.16   File "C:\Users\sasch\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 413, in check_call
 0:11.16     raise CalledProcessError(retcode, cmd)
 0:11.16 subprocess.CalledProcessError: Command '['C:\\Users\\sasch\\.mozbuild\\srcdirs\\gecko-dev-1a7eab949d3b\\_virtualenvs\\build\\Scripts\\python.exe', 'C:/Users/sasch/Documents/GitHub/gecko-dev/mach', 'android', 'assemble-app']' returned non-zero exit status 1.
 0:11.16 mozmake[3]: *** [backend.mk:957: mobile/android/.deps/android_apks.stub] Error 1
 0:11.17 mozmake[3]: *** Waiting for unfinished jobs....
 0:13.27  0:00.68 C:/Users/sasch/Documents/GitHub/gecko-dev/gradlew.bat geckoview:generateJNIWrappersForGeneratedWithGeckoBinariesDebug
 0:13.29 Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=utf-8
 0:14.15 FAILURE: Build failed with an exception.
 0:14.16 * Where:
 0:14.16 Settings file 'C:\Users\sasch\Documents\GitHub\gecko-dev\settings.gradle' line: 33
 0:14.16 * What went wrong:
 0:14.16 A problem occurred evaluating settings 'gecko-dev'.
 0:14.16 > Could not run ./mach environment. Try running ./mach build first.
 0:14.16 * Try:
 0:14.16 > Run with --stacktrace option to get the stack trace.
 0:14.16 > Run with --info or --debug option to get more log output.
 0:14.16 > Run with --scan to get full insights.
 0:14.16 * Get more help at https://help.gradle.org
 0:14.16 BUILD FAILED in 793ms
 0:14.77 Traceback (most recent call last):
 0:14.77   File "<frozen runpy>", line 198, in _run_module_as_main
 0:14.77   File "<frozen runpy>", line 88, in _run_code
 0:14.77   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\file_generate.py", line 155, in <module>
 0:14.77     sys.exit(log_build_task(main, sys.argv[1:]))
 0:14.78              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:14.78   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\util.py", line 16, in log_build_task
 0:14.78     return f(*args, **kwargs)
 0:14.78            ^^^^^^^^^^^^^^^^^^
 0:14.78   File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\action\file_generate.py", line 99, in main
 0:14.78     ret = module.__dict__[method](
 0:14.78           ^^^^^^^^^^^^^^^^^^^^^^^^
 0:14.78   File "C:/Users/sasch/Documents/GitHub/gecko-dev/mobile/android/gradle.py", line 60, in generate_generated_jni_wrappers
 0:14.78     return android("generate-generated-jni-wrappers", *args)
 0:14.78            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:14.78   File "C:/Users/sasch/Documents/GitHub/gecko-dev/mobile/android/gradle.py", line 46, in android
 0:14.78     subprocess.check_call(cmd, env=env)
 0:14.78   File "C:\Users\sasch\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 413, in check_call
 0:14.78     raise CalledProcessError(retcode, cmd)
 0:14.78 subprocess.CalledProcessError: Command '['C:\\Users\\sasch\\.mozbuild\\srcdirs\\gecko-dev-1a7eab949d3b\\_virtualenvs\\build\\Scripts\\python.exe', 'C:/Users/sasch/Documents/GitHub/gecko-dev/mach', 'android', 'generate-generated-jni-wrappers']' returned non-zero exit status 1.
 0:14.78 mozmake[4]: *** [backend.mk:191: .deps/generated_jni_wrappers.stub] Error 1
 0:14.78 mozmake[3]: *** [C:/Users/sasch/Documents/GitHub/gecko-dev/config/recurse.mk:94: widget/android/export] Error 2
 0:14.78 mozmake[2]: *** [C:/Users/sasch/Documents/GitHub/gecko-dev/config/recurse.mk:34: export] Error 2
 0:14.80 mozmake[1]: *** [C:/Users/sasch/Documents/GitHub/gecko-dev/config/rules.mk:361: default] Error 2
 0:14.80 mozmake: *** [client.mk:60: build] Error 2
 0:14.83 W 0 compiler warnings present.
Traceback (most recent call last):
  File "c:\users\sasch\.mozbuild\srcdirs\gecko-dev-1a7eab949d3b\_virtualenvs\mach\lib\site-packages\glean\_subprocess\_process_dispatcher_helper.py", line 35, in <module>
    simple_log_level, func, args = pickle.loads(base64.b64decode(sys.argv[1]))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\sasch\.mozbuild\srcdirs\gecko-dev-1a7eab949d3b\_virtualenvs\mach\lib\site-packages\glean\__init__.py", line 18, in <module>
    from .glean import Glean
ModuleNotFoundError: No module named 'glean.glean'

I can't do the following in a straightforward way as there's no ./mach build --info...

 0:14.16 > Run with --stacktrace option to get the stack trace.
 0:14.16 > Run with --info or --debug option to get more log output.
 0:14.16 > Run with --scan to get full insights.
OS: Unspecified → Windows 11
Hardware: Unspecified → x86_64
Summary: Android build fails on Android → Android build fails on Windows

Managed to add --info via android_generate_generated_jni_wrappers in mach_commands.py.

> ./mach android generate-generated-jni-wrappers                                                                                                                                   
Collecting glean-sdk==53.0.0
  Using cached glean_sdk-53.0.0-py3-none-win_amd64.whl (1.5 MB)
Collecting semver>=2.13.0
  Using cached semver-3.0.1-py3-none-any.whl (17 kB)
Requirement already satisfied: glean-parser~=7.1 in c:\users\sasch\documents\github\gecko-dev\third_party\python\glean_parser (from glean-sdk==53.0.0) (7.2.1)
Requirement already satisfied: appdirs>=1.4 in c:\users\sasch\documents\github\gecko-dev\third_party\python\appdirs (from glean-parser~=7.1->glean-sdk==53.0.0) (1.4.4)
Requirement already satisfied: Click>=7 in c:\users\sasch\documents\github\gecko-dev\third_party\python\click (from glean-parser~=7.1->glean-sdk==53.0.0) (7.1.2)
Requirement already satisfied: diskcache>=4 in c:\users\sasch\documents\github\gecko-dev\third_party\python\diskcache (from glean-parser~=7.1->glean-sdk==53.0.0) (4.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in c:\users\sasch\documents\github\gecko-dev\third_party\python\jinja2 (from glean-parser~=7.1->glean-sdk==53.0.0) (2.11.3)
Requirement already satisfied: MarkupSafe<=2.0.1,>=1.1.1 in c:\users\sasch\documents\github\gecko-dev\third_party\python\markupsafe\src (from glean-parser~=7.1->glean-sdk==53.0.0) (2.0.1)
Requirement already satisfied: jsonschema>=3.0.2 in c:\users\sasch\documents\github\gecko-dev\third_party\python\jsonschema (from glean-parser~=7.1->glean-sdk==53.0.0) (4.17.3)
Requirement already satisfied: PyYAML>=5.3.1 in c:\users\sasch\documents\github\gecko-dev\third_party\python\pyyaml\lib (from glean-parser~=7.1->glean-sdk==53.0.0) (6.0.1)
Requirement already satisfied: attrs>=17.4.0 in c:\users\sasch\documents\github\gecko-dev\third_party\python\attrs (from jsonschema>=3.0.2->glean-parser~=7.1->glean-sdk==53.0.0) (23.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in c:\users\sasch\documents\github\gecko-dev\third_party\python\pyrsistent (from jsonschema>=3.0.2->glean-parser~=7.1->glean-sdk==53.0.0) (0.16.0)
Requirement already satisfied: six in c:\users\sasch\documents\github\gecko-dev\third_party\python\six (from pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0->jsonschema>=3.0.2->glean-parser~=7.1->glean-sdk==53.0.0) (1.13.0)
Installing collected packages: semver, glean-sdk
Successfully installed glean-sdk-53.0.0 semver-3.0.1
Collecting psutil<=5.9.4,>=5.4.2
  Using cached psutil-5.9.4-cp36-abi3-win_amd64.whl (252 kB)
Installing collected packages: psutil
Successfully installed psutil-5.9.4
Collecting zstandard<=0.19.0,>=0.11.1
  Using cached zstandard-0.19.0-cp311-cp311-win_amd64.whl (473 kB)
Installing collected packages: zstandard
Successfully installed zstandard-0.19.0
 0:00.77 C:/Users/sasch/Documents/GitHub/gecko-dev/gradlew.bat geckoview:generateJNIWrappersForGeneratedWithGeckoBinariesDebug --info
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=utf-8
Initialized native services in: C:\Users\sasch\.gradle\native
Initialized jansi services in: C:\Users\sasch\.gradle\native
The client will now receive all logging from the daemon (pid: 25872). The daemon log file: C:\Users\sasch\.gradle\daemon\7.5.1\daemon-25872.out.log
Starting 15th build in daemon [uptime: 25 mins 44.396 secs, performance: 100%, non-heap usage: 0% of 6 GiB]
Using 64 worker leases.
Now considering [C:\Users\sasch\Documents\GitHub\gecko-dev] as hierarchies to watch
Watching the file system is configured to be enabled if available
File system watching is active
Starting Build
Error running ./mach environment:

Process '[c:/mozilla-build/python3/python3.exe, C:\Users\sasch\Documents\GitHub\gecko-dev/mach, environment, --format, json, --verbose]' finished with non-zero exit value 1:

stdout:


stderr:
Traceback (most recent call last):
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\mach", line 125, in <module>
    main(sys.argv[1:])
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\mach", line 117, in main
    mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\mach", line 37, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\mach", line 23, in load_mach
    return mach_initialize.initialize(dir_path)
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\build/mach_initialize.py", line 167, in initialize
    _activate_python_environment(
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\build/mach_initialize.py", line 93, in _activate_python_environment
    mach_environment.activate()
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mach\mach\site.py", line 369, in activate
    self.ensure()
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mach\mach\site.py", line 356, in ensure
    self._build()
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mach\mach\site.py", line 394, in _build
    _create_venv_with_pthfile(
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mach\mach\site.py", line 1269, in _create_venv_with_pthfile
    shutil.rmtree(virtualenv_root)
  File "c:\mozilla-build\python3\lib\shutil.py", line 749, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\mozilla-build\python3\lib\shutil.py", line 622, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\mozilla-build\python3\lib\shutil.py", line 622, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\mozilla-build\python3\lib\shutil.py", line 622, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\mozilla-build\python3\lib\shutil.py", line 627, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\mozilla-build\python3\lib\shutil.py", line 625, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\sasch\\.mozbuild\\srcdirs\\gecko-dev-1a7eab949d3b\\_virtualenvs\\mach\\Lib\\site-packages\\glean\\glean_ffi.dll'


FAILURE: Build failed with an exception.

* Where:
Settings file 'C:\Users\sasch\Documents\GitHub\gecko-dev\settings.gradle' line: 33

* What went wrong:
A problem occurred evaluating settings 'gecko-dev'.
> Could not run ./mach environment. Try running ./mach build first.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 796ms
Watched directory hierarchies: []
Traceback (most recent call last):                                                                                                                                                                                                                                       
  File "c:\users\sasch\.mozbuild\srcdirs\gecko-dev-1a7eab949d3b\_virtualenvs\mach\lib\site-packages\glean\_subprocess\_process_dispatcher_helper.py", line 35, in <module>                                                                                               
    simple_log_level, func, args = pickle.loads(base64.b64decode(sys.argv[1]))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\sasch\.mozbuild\srcdirs\gecko-dev-1a7eab949d3b\_virtualenvs\mach\lib\site-packages\glean\__init__.py", line 18, in <module>
    from .glean import Glean
ModuleNotFoundError: No module named 'glean.glean'

That's... weird?

Summary: Android build fails on Windows → Android build fails on Windows when triggered outside MozillaBuild
Assignee: nobody → krosylight
Status: NEW → ASSIGNED
Pushed by krosylight@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7b6332c9bce4 Use default python3 outside MozillaBuild on Windows r=m_kato,geckoview-reviewers

https://hg.mozilla.org/mozilla-central/rev/7b6332c9bce4

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
status-firefox117: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 117 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: