Closed Bug 1880254 Opened 1 year ago Closed 1 year ago

error LNK2019: unresolved external symbol _InterlockedCompareExchange

Categories

(NSPR :: NSPR, defect, P2)

Product:
NSPR
Component:
NSPR
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jcristau, Assigned: KaiE)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 3 obsolete files)

Bug 1880254 - On Windows, build 32bit on a 64bit host, if USE_64 is unset. r=glandium
48 bytes, text/x-phabricator-request
Details | Review

Trying to update the windows workers used for nss builds in bug 1831552, I'm hitting nspr build failures on 32-bit windows, e.g. https://treeherder.mozilla.org/jobs?repo=nss-try&selectedTaskRun=UFbvsm5PQBiHNYVUEDmt0Q.0&revision=7355b70c4c9ee6d2887f16ee59092e49cc7a6faf&searchStr=nspr

Creating library libnspr4.lib and object libnspr4.exp
prmwait.obj : error LNK2019: unresolved external symbol _InterlockedCompareExchange referenced in function _NT_TimeProc
ntio.obj : error LNK2001: unresolved external symbol _InterlockedCompareExchange
pratom.obj : error LNK2019: unresolved external symbol _InterlockedExchange referenced in function _PR_AtomicSet
libnspr4.dll : fatal error LNK1120: 2 unresolved externals
../../../config/rules.mk:306: recipe for target 'libnspr4.dll' failed

The severity field is not set for this bug.
:KaiE, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(kaie)
Severity: -- → N/A
Flags: needinfo?(kaie)
Priority: -- → P2

The MS developer docs still document it
https://learn.microsoft.com/en-us/cpp/intrinsics/interlockedcompareexchange-intrinsic-functions?view=msvc-170
(no talk about deprecation etc).

The functions are described as intrinsic in the 32bit SDK environment.
As I understand it, that means "built in". I'd expect it to be available to the compiler immediately, without linking them from a library.

I'm reading the function is also available on 64bit, but handled differently.
Is it possible your build environment somewhere defaults to 64bit?

documenting the linker command line from the log:

link -nologo -DLL -SUBSYSTEM:WINDOWS -DYNAMICBASE -OUT:"libnspr4.dll" -MAP advapi32.lib ws2_32.lib mswsock.lib winmm.lib ./prvrsion.obj io/./prfdcach.obj io/./prmwait.obj io/./prmapopt.obj io/./priometh.obj io/./pripv6.obj io/./prlayer.obj io/./prlog.obj io/./prmmap.obj io/./prpolevt.obj io/./prprf.obj io/./prscanf.obj io/./prstdio.obj threads/./prcmon.obj threads/./prrwlock.obj threads/./prtpd.obj linking/./prlink.obj malloc/./prmalloc.obj malloc/./prmem.obj md/./prosdep.obj memory/./prshm.obj memory/./prshma.obj memory/./prseg.obj misc/./pralarm.obj misc/./pratom.obj misc/./prcountr.obj misc/./prdtoa.obj misc/./prenv.obj misc/./prerr.obj misc/./prerror.obj misc/./prerrortable.obj misc/./prinit.obj misc/./prinrval.obj misc/./pripc.obj misc/./prlog2.obj misc/./prlong.obj misc/./prnetdb.obj misc/./praton.obj misc/./prolock.obj misc/./prrng.obj misc/./prsystem.obj misc/./prthinfo.obj misc/./prtpool.obj misc/./prtrace.obj misc/./prtime.obj io/./prdir.obj io/./prfile.obj io/./prio.obj io/./prsocket.obj misc/./pripcsem.obj threads/./prcthr.obj threads/./prdump.obj threads/./prmon.obj threads/./prsem.obj threads/combined/./prucpu.obj threads/combined/./prucv.obj threads/combined/./prulock.obj threads/combined/./prustack.obj threads/combined/./pruthr.obj md/windows/./ntmisc.obj md/windows/./ntsec.obj md/windows/./ntsem.obj md/windows/./ntinrval.obj md/windows/./ntgc.obj md/windows/./ntio.obj md/windows/./ntthread.obj md/windows/./ntdllmn.obj md/windows/./win32_errors.obj md/windows/./w32ipcsem.obj md/windows/./w32poll.obj md/windows/./w32rng.obj md/windows/./w32shm.obj ./nspr.res

The following suggests that a necessary header is missing. For this to work, the compiler needs to be able to find the correct definition for that function.

cl -Foprmwait.obj -c -W3 -nologo -GF -Gy -FS -MD -GT -O2 -UDEBUG -U_DEBUG -DPACKAGE_NAME="" -DPACKAGE_TARNAME="" -DPACKAGE_VERSION="" -DPACKAGE_STRING="" -DPACKAGE_BUGREPORT="" -DPACKAGE_URL="" -DNDEBUG=1 -DXP_PC=1 -DWIN32=1 -D_CRT_SECURE_NO_WARNINGS=1 -D_CRT_NONSTDC_NO_WARNINGS=1 -DWINNT=1 -D_AMD64_=1 -DFORCE_PR_LOG -D_NSPR_BUILD_ -I../../../dist/include/nspr -I../../../../pr/include -I../../../../pr/include/private "D:/task_170782978489337/nspr/Release/pr/src/io/../../../../pr/src/io/prmwait.c"
prmwait.c
D:/task_170782978489337/nspr/Release/pr/src/io/../../../../pr/src/io/prmwait.c(663): warning C4013: 'InterlockedCompareExchange' undefined; assuming extern returning int

It would be easiest if someone had an installation of the equivalent versions of OS and Compiler/SDK installed, and can investigate by building locally, e.g. trying to find the header that provides the missing functions.

What versions are you using?

The severity field is not set for this bug.
:KaiE, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(kaie)
Severity: N/A → S3
Flags: needinfo?(kaie)

Can you please try the attached revision?

This is a quick shot, which might work or not.
It's based on the header files I saw on a test system given to me, and also matches information from
https://learn.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-interlockedcompareexchange

I tried to execute the NSPR build myself, and I looked at the failed log to see which command were executed.
( https://firefoxci.taskcluster-artifacts.net/UFbvsm5PQBiHNYVUEDmt0Q/0/public/logs/live_backing.log )

It fails on the commend "pushd gyp" from build_gyp.sh - that directory doesn't exist.

I found the clone gyp command in the output, but next failure is "no module named virtualenv".

It seems you have already done some manual preparation for the new build environment, which isn't yet contained in standard scripts?

It would help if you could please run the failing build in the VM given to me, and then let me know, so I can continue to work with the failed environment.

pip install virtualenv
worked, but there's more missing, now it's looking for environment variable VSPATH which isn't set.
I also cannot find a directory name SDK on the system, and directories VC don't contain the files I need, so it seems the test system has an incomplete setup.

ok, I realized that I must apply and merge the patches from the given try build https://treeherder.mozilla.org/jobs?repo=nss-try&revision=7355b70c4c9ee6d2887f16ee59092e49cc7a6faf&searchStr=nspr
which I did

Now it's failing because it doesn't have the TASKCLUSTER_PROXY_URL set

ok, while it probably isn't easy to get the build working interactively, I realize that I could resubmit the old try build, with the patches used at that time, plus my experimental patch.

I found the likely cause.

The build sets an incorrect 64bit define "AMD64" when building 32bit.

cl -Fonow.obj -c -W3 -nologo -GF -Gy -FS -Od -MD -GT -Zi -UNDEBUG -DDEBUG_task_170782997768512 -DPACKAGE_NAME="" -DPACKAGE_TARNAME="" -DPACKAGE_VERSION="" -DPACKAGE_STRING="" -DPACKAGE_BUGREPORT="" -DPACKAGE_URL="" -DDEBUG=1 -DXP_PC=1 -DWIN32=1 -D_CRT_SECURE_NO_WARNINGS=1 -D_CRT_NONSTDC_NO_WARNINGS=1 -D_DEBUG=1 -DWINNT=1 -D_AMD64_=1 -DFORCE_PR_LOG "D:/task_170782997768512/nspr/Debug/config/../../config/now.c"

I think it's necessary to define X86 with -D_X86_=1
which should select the correct symbols when including the windows headers.

It seems the old Windows builder machines use a 32bit Windows OS, can you confirm?

From an old working logfile:
NSPR [1/5] configure ...
../configure --prefix=/z/task_171291681252791/dist/Debug
checking build system type... i686-pc-mingw32
checking host system type... i686-pc-mingw32
checking target system type... i686-pc-mingw32

It seems the new Windows builder machine uses a 64bit Windows OS.

From a failing build:
../configure --prefix=/d/task_170782997768512/dist/Debug
checking build system type... x86_64-pc-msys
checking host system type... x86_64-pc-msys
checking target system type... x86_64-pc-msys

Regardless of build flags, it seems the NSPR configure script always uses the system architecture as the default for the 32/64 bit decision.
It may override the decision and use 64 bit based on parameters.

However, it seems the script doesn't allow building 32bit on a 64bit system.

This results in a bad mix of compiler flags for 64 bit, with include directories (set by the gyp scripts etc.) for a 32 bit build.

Is it possible that you continue to use 32bit Windows build machines?

If not, it will be necessary to work on the NSPR configure script.

I have a patch that changes the configure script.

The disadvantage of my minimal change is that building on a 64bit host would no longer automatically build a 64bit executable - but rather 32bit. It would become necessary to specify a parameter to get a 64bit build. This might be acceptable, it's aligned with our behavior elsewhere.

We have a compiler warning because of the cast to PVOID for the parameters to InterlockedCompareExchange in ntio.c

We have a new linker failure, because link.exe is used from /usr/bin/link.exe, which doesn't understand the parameters.
The build scripts need to be changed to prefer the link.exe from Visual Studio.

Attachment #9396752 - Attachment is obsolete: true

(In reply to Kai Engert (:KaiE:) from comment #17)

We have a new linker failure, because link.exe is used from /usr/bin/link.exe, which doesn't understand the parameters.
The build scripts need to be changed to prefer the link.exe from Visual Studio.

Julien might have tried to fix that by changing the order of directories in PATH, which my latest try build had reverted. I'll restore Julien's reordering and run another new try build.

Assignee: nobody → kaie
Attachment #9396910 - Attachment description: WIP: Bug 1880254 - On Windows, build 32bit on a 64bit host, if USE_64 is unset. → Bug 1880254 - On Windows, build 32bit on a 64bit host, if USE_64 is unset. r=glandium
Status: NEW → ASSIGNED

Not sure if it's necessary to have windows make builds - if it is, I've attached an experimental change that made them work in your upgraded windows environment.

Thanks Kai!

You're welcome. Although I've done the analysis, it seems more work is required to configure NSPR in an acceptable, using a --target parameter, as suggested by glandium in phabricator. I'd appreciate help here, not sure when I could get to that.

That's fair. I'm trying a few things to get nss to pass the right param to nspr's configure (side note: as far as I can tell nspr should be looking at --host, not --target)

(In reply to Julien Cristau [:jcristau] from comment #25)

(side note: as far as I can tell nspr should be looking at --host, not --target)

That ship has sailed. NSPR is like Firefox, it uses --host and --target differently from what autotools normally do.

Comment on attachment 9397087 [details]
WIP: Bug 1880254 - Allow make builds to work

Revision D207691 was moved to bug 1880255. Setting attachment 9397087 [details] to obsolete.

Attachment #9397087 - Attachment is obsolete: true
Attachment #9459552 - Attachment is obsolete: true

https://hg.mozilla.org/projects/nspr/rev/61a5d3497ede6801460a2805d602d10d7cf75739

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 4.37
Blocks: 1975156
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: