1669697 - gcm.c methods not defined with aarch64 compiled static NSS library

Status: RESOLVED INCOMPLETE

(NSS :: Build, defect)

Description

[Andrej]

Attachment: My thoughts on the error.

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36

Steps to reproduce:

1. Building NSS with following configurations:
   gyp -f ninja-observit "${NSS_SRC_DIR}/nss/nss.gyp"
   --depth "${NSS_SRC_DIR}/nss/"
   --generator-output=.
   -DOS=linux
   -Dnspr_lib_dir="${NSS_SRC_DIR}/nspr_build/dist/lib"
   -Dnspr_include_dir="${NSS_SRC_DIR}/nspr_build/dist/include/nspr"
   -Dnss_dist_dir="${NSS_SRC_DIR}/nss_build"
   -Dnss_dist_obj_dir="${NSS_SRC_DIR}/nss_build"
   -Dtarget_arch=aarch64
   -Dstatic_libs=1
   -Ddisable_dbm=1
   -Dsign_libs=0
   -Denable_sslkeylogfile=0
   -Ddisable_tests=1
   -Ddisable_libpkix=1
   -Dcc_is_gcc=1
   -Ddisable_arm_hw_aes=1
   -Duse_system_zlib=1
   -Dzlib_libs="-L$INSTALLTOP/zlib/1.2.11/$PLATFORM/lib -lz" &>> $NSS_SRC_DIR/build_output

2. Link static library to my application as follows:
   LDLIBS += -static-libstdc++ -lcurl -lz -lrt -Wl,--start-group -lssl -lcrmf -lnss_static -lpkcs12 -lcerthi -lpkcs7 -lcryptohi -lpk11wrap_static -lplc4 -lnsspki -lsmime -lnssutil -lplds4 -lcertdb -lnssb -lnssdev -lsoftokn_static -lsqlite -lfreebl_static -lhw-acc-crypto -lnspr4 -Wl,--end-group -ldl

Compilator for both the application and the NSS library
/opt/axis/acapsdk/3.01.0/aarch64/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++ -mcpu=cortex-a53+crc+crypto --sysroot=/opt/axis/acapsdk/3.01.0/aarch64/sysroots/aarch64-poky-linux

Actual results:

./checksrc -h ama
/opt/axis/acapsdk/3.01.0/aarch64/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++ -mcpu=cortex-a53+crc+crypto --sysroot=/opt/axis/acapsdk/3.01.0/aarch64/sysroots/aarch64-poky-linux build/deviceapi/src/Pinger.o build/deviceapi/src/PeriodicReporter.o build/report/src/HardwareReport.o build/report/src/FileReport.o build/report/src/AppErrorReport.o build/report/src/StartReport.o build/report/src/LicenseReport.o build/report/src/SoftwareReport.o build/report/src/ExitReport.o build/report/src/AppStatusReport.o build/app/src/AmaUtil.o build/app/src/AmaContext.o build/app/src/App.o build/app/src/AmaStart.o build/app/src/AppManager.o build/app/src/AmaResponseHandler.o build/app/src/JsonFieldUpdater.o build/app/src/AmaConstants.o build/app/src/AppErrors.o build/app/src/AmaApp.o build/cfg/src/ConfigurationUpdater.o build/cfg/src/ConfigurationToc.o build/command/src/CfgCmd.o build/command/src/RemoveCmd.o build/command/src/SetCustomerKeyCmd.o build/command/src/RebootCmd.o build/command/src/GetFileCmd.o build/command/src/StartAppCmd.o build/command/src/UnlockCmd.o build/command/src/AmaCmdReceiver.o build/command/src/AmaCommand.o build/command/src/InstallCmd.o build/command/src/RestartAppCmd.o build/command/src/ClearQCmd.o build/command/src/SetHolderIdCmd.o build/command/src/StopAppCmd.o build/command/src/AmaCmdQueue.o build/command/src/GetHwCmd.o build/command/src/LockCmd.o build/command/src/GetSwCmd.o build/ama.o -s -pthread -Lexternal/lib/axis_arm64 -lcurl -lrt -Wl,--start-group -lssl -lcrmf -lnss_static -lpkcs12 -lcerthi -lpkcs7 -lcryptohi -lpk11wrap_static -lplc4 -lnsspki -lsmime -lnssutil -lplds4 -lcertdb -lnssb -lnssdev -lsoftokn_static -lsqlite -lfreebl_static -lhw-acc-crypto -lnspr4 -Wl,--end-group -ldl -lz -o build/ama
/opt/axis/acapsdk/3.01.0/aarch64/sysroots/x86_64-pokysdk-linux/usr/libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/9.2.0/real-ld: external/lib/axis_arm64/libfreebl_static.a(freebl_static.gcm.o): in function gcmHash_InitContext': /home/norman/libs/nss/nss/out/Release/../../lib/freebl/gcm.c:100: undefined reference to gcm_HashInit_hw'
/opt/axis/acapsdk/3.01.0/aarch64/sysroots/x86_64-pokysdk-linux/usr/libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/9.2.0/real-ld: external/lib/axis_arm64/libfreebl_static.a(freebl_static.gcm.o): in function gcmHash_Final': /home/norman/libs/nss/nss/out/Release/../../lib/freebl/gcm.c:455: undefined reference to gcm_HashWrite_hw'
/opt/axis/acapsdk/3.01.0/aarch64/sysroots/x86_64-pokysdk-linux/usr/libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/9.2.0/real-ld: external/lib/axis_arm64/libfreebl_static.a(freebl_static.gcm.o): in function gcm_zeroX': /home/norman/libs/nss/nss/out/Release/../../lib/freebl/gcm.c:326: undefined reference to gcm_HashZeroX_hw'
collect2: error: ld returned 1 exit status
make: *** [Makefile:116: build/ama] Error 1

Expected results:

Successful compilation

Comment 1

[J.C. Jones [:jcj] (he/they)]

Not a security bug, unhiding for triage

Comment 2

[J.C. Jones [:jcj] (he/they)]

Thanks for the detailed commentary you attached.

The first thing I notice is that you've defined -Ddisable_arm_hw_aes on the NSS static build, which I believe is stopping the compilation of gcm-aarch64.c, where these references should be defined. In this case what should happen is a fallback to the software implementation, but as we see, your code is attempting to link the _hw accelerated methods anyway.

That is, I think, because when you're compiling your application, it's assuming it has hardware acceleration, but those methods weren't compiled when you compiled NSS. The stubs in gcm.c are there to try and avoid this problem, but clearly we somehow failed to compile the stubs, too.

So I think that's the mismatch that needs fixing here. Both the library compilation and the application compilation need to agree on whether to explicitly disable hardware acceleration for AES. If the stubs were there correctly, you'd be getting SEC_ERROR_LIBRARY_FAILURE instead of the link failure in this case.

Can you play around with those defines for your application and/or the library build and see what happens? And please report back. If I'm right, then we should change this bug to instead fix whatever case we've fallen into where the stubs failed to get built with -Ddisable_arm_hw_aes defined on aarch64.

Thanks!

Comment 3

[BugBot [:suhaib / :marco]]

A needinfo is requested from the reporter, however, the reporter is inactive on Bugzilla. Closing the bug as incomplete.

For more information, please visit auto_nag documentation.