Publish a full-megazord-forUnitTests for Apple ARM architectures
Categories
(Application Services :: General, task)
Tracking
(Not tracked)
People
(Reporter: jonalmeida, Assigned: tcampbell)
References
Details
(Whiteboard: [fxdroid][group6])
Attachments
(1 file, 2 obsolete files)
Running android junit tests for services-firefox-accounts on an M1 Mac fails the tests with the following error:
dlopen(/Users/jalmeida/Library/Caches/JNA/temp/jna16523238109202510986.tmp, 0x0009): tried: '/Users/jalmeida/Library/Caches/JNA/temp/jna16523238109202510986.tmp' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/jalmeida/Library/Caches/JNA/temp/jna16523238109202510986.tmp' (no such file), '/Users/jalmeida/Library/Caches/JNA/temp/jna16523238109202510986.tmp' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))
This seems to be the cause because we use -forUnitTests variant of the full megazord native library in the test variant of Android Components. When we unzipped the jar file to find these files in there:
$ exa --tree /Users/jalmeida/Downloads/full-megazord-forUnitTests-132.0
/Users/jalmeida/Downloads/full-megazord-forUnitTests-132.0
├── com
│ └── sun
│ └── jna
│ ├── aix-ppc
│ │ └── libjnidispatch.a
│ ├── aix-ppc64
│ │ └── libjnidispatch.a
│ ├── darwin-aarch64
│ │ └── libjnidispatch.jnilib
│ ├── darwin-x86-64
│ │ └── libjnidispatch.jnilib
│ ├── freebsd-x86
│ │ └── libjnidispatch.so
│ ├── freebsd-x86-64
│ │ └── libjnidispatch.so
│ ├── linux-aarch64
│ │ └── libjnidispatch.so
│ ├── linux-arm
│ │ └── libjnidispatch.so
│ ├── linux-armel
│ │ └── libjnidispatch.so
│ ├── linux-loongarch64
│ │ └── libjnidispatch.so
│ ├── linux-mips64el
│ │ └── libjnidispatch.so
│ ├── linux-ppc
│ │ └── libjnidispatch.so
│ ├── linux-ppc64le
│ │ └── libjnidispatch.so
│ ├── linux-riscv64
│ │ └── libjnidispatch.so
│ ├── linux-s390x
│ │ └── libjnidispatch.so
│ ├── linux-x86
│ │ └── libjnidispatch.so
│ ├── linux-x86-64
│ │ └── libjnidispatch.so
│ ├── openbsd-x86
│ │ └── libjnidispatch.so
│ ├── openbsd-x86-64
│ │ └── libjnidispatch.so
│ ├── sunos-sparc
│ │ └── libjnidispatch.so
│ ├── sunos-sparcv9
│ │ └── libjnidispatch.so
│ ├── sunos-x86
│ │ └── libjnidispatch.so
│ ├── sunos-x86-64
│ │ └── libjnidispatch.so
│ ├── win32-aarch64
│ │ └── jnidispatch.dll
│ ├── win32-x86
│ │ └── jnidispatch.dll
│ └── win32-x86-64
│ └── jnidispatch.dll
├── darwin
│ └── libmegazord.dylib
├── linux-x86-64
│ └── libmegazord.so
└── META-INF
├── AL2.0
├── LGPL2.1
├── LICENSE
└── MANIFEST.MF
Notably, the darwin directory contains a dylib that doesn't support arm64:
$ file /Users/jalmeida/Downloads/full-megazord-forUnitTests-132.0/darwin/libmegazord.dylib
/Users/jalmeida/Downloads/full-megazord-forUnitTests-132.0/darwin/libmegazord.dylib: Mach-O 64-bit dynamically linked shared library x86_64
So it seems like we would just need to publish the artifacts for that build since we already have the jna bindings for it (see above tree that contains darwin-aarch64 and darwin-x86_64 in the com.sun.jna package.
This also looks like a subset of bug 1889970.
|
Reporter | |
Comment 1•1 year ago
|
||
Renaming to "Apple ARM" because M1 is versioned CPU.
|
|
Reporter | |
Comment 2•1 year ago
|
||
For fun, we tried to change this line to use the full version of the megazord in unit tests as well but that failed with a different error where we are unable to find the megazord.dylib in any of the listed paths:
Unable to load library 'megazord':
dlopen(libmegazord.dylib, 0x0009): tried: 'libmegazord.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibmegazord.dylib' (no such file), '/Users/jalmeida/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/./libmegazord.dylib' (no such file), '/Users/jalmeida/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/../lib/libmegazord.dylib' (no such file), '/usr/lib/libmegazord.dylib' (no such file, not in dyld cache), 'libmegazord.dylib' (no such file), '/usr/lib/libmegazord.dylib' (no such file, not in dyld cache)
dlopen(libmegazord.dylib, 0x0009): tried: 'libmegazord.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibmegazord.dylib' (no such file), '/Users/jalmeida/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/./libmegazord.dylib' (no such file), '/Users/jalmeida/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/../lib/libmegazord.dylib' (no such file), '/usr/lib/libmegazord.dylib' (no such file, not in dyld cache), 'libmegazord.dylib' (no such file), '/usr/lib/libmegazord.dylib' (no such file, not in dyld cache)
dlopen(/Users/jalmeida/Library/Frameworks/megazord.framework/megazord, 0x0009): tried: '/Users/jalmeida/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/jalmeida/Library/Frameworks/megazord.framework/megazord' (no such file), '/Users/jalmeida/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Library/Frameworks/megazord.framework/megazord' (no such file, not in dyld cache)
dlopen(/Library/Frameworks/megazord.framework/megazord, 0x0009): tried: '/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/megazord.framework/megazord' (no such file), '/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Library/Frameworks/megazord.framework/megazord' (no such file, not in dyld cache)
dlopen(/System/Library/Frameworks/megazord.framework/megazord, 0x0009): tried: '/System/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/Library/Frameworks/megazord.framework/megazord' (no such file), '/System/Library/Frameworks/megazord.framework/megazord' (no such file, not in dyld cache)
This was the patch I used:
diff --git a/mobile/android/android-components/components/service/firefox-accounts/build.gradle b/mobile/android/android-components/components/service/firefox-accounts/build.gradle
--- a/mobile/android/android-components/components/service/firefox-accounts/build.gradle
+++ b/mobile/android/android-components/components/service/firefox-accounts/build.gradle
@@ -52,15 +52,15 @@ dependencies {
testImplementation project(':support-test')
testImplementation project(':support-test-libstate')
testImplementation ComponentsDependencies.androidx_test_core
testImplementation ComponentsDependencies.androidx_test_junit
testImplementation ComponentsDependencies.androidx_work_testing
testImplementation ComponentsDependencies.testing_robolectric
testImplementation ComponentsDependencies.testing_coroutines
- testImplementation ComponentsDependencies.mozilla_appservices_full_megazord_forUnitTests
+ testImplementation ComponentsDependencies.mozilla_appservices_full_megazord
testImplementation ComponentsDependencies.kotlin_reflect
}
|
|
Reporter | |
Comment 3•1 year ago
|
||
Building a local version of application services with rust.targets=darwin-aarch64 works for me locally now so I can unblock myself and also validates that we need to publish the darwin-aarch64 and darwin-x86_64 variants of full-megazord-forUnitTests.
|
|
Reporter | |
Comment 4•11 months ago
•
|
||
I ran into this today again when trying to review some appservices PRs. bdk, do you know what could be the cause?
My guess, based on comment 3 is that the line in our gradle file says that we are reading the projects to build over here:
The official packages are built in taskcluster include
linux-x86-64anddarwin-x86-64and the list is controlled by
taskcluster/kinds/module-build/kind.yml
The taskcluster kind.yml points to these targets:
rust.targets=arm,arm64,x86_64,x86,darwin,linux-x86-64
Maybe that darwin should be darwin-aarch64 then? I'm going to try and push a CI build on appservices to see if that Just Works™ EDIT: Haven't got around to this.
|
||
Comment 5•11 months ago
|
||
The taskcluster kind.yml points to these targets:
rust.targets=arm,arm64,x86_64,x86,darwin,linux-x86-64
Maybe that darwin should be darwin-aarch64 then? I'm going to try and push a CI build on appservices to see if that Just Works™.
I can never remember how this works exactly, but that seems right to me. If it works for you then I think we should go for it and commit the change.
|
|
Reporter | |
Updated•8 months ago
|
|
||
Updated•8 months ago
|
|
|
Reporter | |
Updated•8 months ago
|
| Comment hidden (obsolete) |
| Comment hidden (obsolete) |
|
|
Reporter | |
Updated•8 months ago
|
|
|
Reporter | |
Comment 8•6 months ago
|
||
tcampbell has made more progress than I have, so marking my patches as obsolete for now.
|
|
Reporter | |
Updated•6 months ago
|
| Comment hidden (obsolete) |
|
|
Reporter | |
Comment 10•6 months ago
|
||
(In reply to Jonathan Almeida [:jonalmeida] from comment #9)
Oh hrm, I re-assigned it, but I can also take over your patch if you need me to get it over the finish line.
I type before reading. Looks like there is active work on that WIP patch.
|
|
Reporter | |
Comment 11•2 months ago
|
||
From conversations with Nick, "the plan of record is to address this during the a-s monorepo work; we should have a sensible “build libmegazord.so” step for all Desktop architectures with that work."
I recall a previous conversation as well where I was convinced that trying to support this case wasn't as fruitful because we these tests to be instrumented instead of trying to build a test environment that is not similar to what users are executing on their devices.
Description
•