Closed Bug 1341222 Opened 3 years ago Closed 2 years ago

--with-system-nspr without --with-system-nss fails to build

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox52 unaffected, firefox-esr52 wontfix, firefox-esr60 wontfix, firefox53 wontfix, firefox54 wontfix, firefox61 wontfix, firefox62 wontfix, firefox63 fixed)

RESOLVED FIXED
mozilla63
Tracking Status
firefox52 --- unaffected
firefox-esr52 --- wontfix
firefox-esr60 --- wontfix
firefox53 --- wontfix
firefox54 --- wontfix
firefox61 --- wontfix
firefox62 --- wontfix
firefox63 --- fixed

People

(Reporter: jbeich, Assigned: glandium)

References

Details

(Keywords: regression)

Attachments

(3 files)

Probably the same issue as in bug 1319405 but can be worked around by passing --with-system-nss as well.

$ echo "ac_add_options --with-system-nspr" >>.mozconfig
$ ./mach build
[...]
Creating config.status
Reticulating splines...
Traceback (most recent call last):
  File "configure.py", line 124, in <module>
    sys.exit(main(sys.argv))
  File "configure.py", line 34, in main
    return config_status(config)
  File "configure.py", line 119, in config_status
    return config_status(args=[], **encode(sanitized_config, encoding))
  File "python/mozbuild/mozbuild/config_status.py", line 147, in config_status
    definitions = list(definitions)
  File "python/mozbuild/mozbuild/frontend/emitter.py", line 194, in emit
    objs = list(self._emit_libs_derived(contexts))
  File "python/mozbuild/mozbuild/frontend/emitter.py", line 228, in _emit_libs_derived
    self._link_libraries(context, obj, variable)
  File "python/mozbuild/mozbuild/frontend/emitter.py", line 341, in _link_libraries
    context)
mozbuild.frontend.reader.SandboxValidationError:
==============================
ERROR PROCESSING MOZBUILD FILE
==============================

The error occurred while processing the following file or one of the files it includes:

    security/nss/cmd/modutil/moz.build

The error occurred when validating the result of the execution. The reported error is:

    USE_LIBS contains "nspr4", which does not match any LIBRARY_NAME in the tree.


*** Fix above errors and then restart with\
               "/usr/local/bin/gmake -f client.mk build"
gmake: *** [client.mk:383: configure] Error 1
gyp_vars and MOZ_FOLD_LIBS in security/moz.build makes it unclear what the correct fix is supposed to be.
There are multiple things that are going wrong here. The patch I applied to Debian is the following:

diff --git a/build/autoconf/nspr-build.m4 b/build/autoconf/nspr-build.m4
index bb4e630d262b..2438d0357bc8 100644
--- a/build/autoconf/nspr-build.m4
+++ b/build/autoconf/nspr-build.m4
@@ -156,8 +156,8 @@ if test -n "$MOZ_SYSTEM_NSPR" -o -n "$NSPR_CFLAGS" -o -n "$NSPR_LIBS"; then
                 ,
                 AC_MSG_ERROR([system NSPR does not support PR_UINT64 or including prtypes.h does not provide it]))
     CFLAGS=$_SAVE_CFLAGS
-    NSPR_INCLUDE_DIR=`echo ${NSPR_CFLAGS} | sed -e 's/.*-I\([^ ]*\).*/\1/'`
-    NSPR_LIB_DIR=`echo ${NSPR_LIBS} | sed -e 's/.*-L\([^ ]*\).*/\1/'`
+    NSPR_INCLUDE_DIR=`echo ${NSPR_CFLAGS} | sed -e 's/.*-I\([[^ ]]*\).*/\1/'`
+    NSPR_LIB_DIR=`echo ${NSPR_LIBS} | sed -e 's/.*-L\([[^ ]]*\).*/\1/'`
 elif test -z "$JS_POSIX_NSPR"; then
     NSPR_INCLUDE_DIR="${DIST}/include/nspr"
     NSPR_CFLAGS="-I${NSPR_INCLUDE_DIR}"
diff --git a/python/mozbuild/mozbuild/frontend/gyp_reader.py b/python/mozbuild/mozbuild/frontend/gyp_reader.py
index 0c1b41e2ecc8..acfe64405aaa 100644
--- a/python/mozbuild/mozbuild/frontend/gyp_reader.py
+++ b/python/mozbuild/mozbuild/frontend/gyp_reader.py
@@ -271,9 +271,10 @@ def process_gyp_result(gyp_result, gyp_dir_attrs, path, config, output,
                   # filtered out by trying to find them in topsrcdir.
                   if include.startswith('/'):
                       resolved = mozpath.abspath(mozpath.join(config.topsrcdir, include[1:]))
-                  else:
+                  elif not include.startswith('%'):
                       resolved = mozpath.abspath(mozpath.join(mozpath.dirname(build_file), include))
-                  if not os.path.exists(resolved):
+                  if not include.startswith('%') and not os.path.exists(resolved):
+                      print(resolved)
                       continue
                 context['LOCAL_INCLUDES'] += [include]
 
diff --git a/security/moz.build b/security/moz.build
index 699d11a8304c..12e3dd894846 100644
--- a/security/moz.build
+++ b/security/moz.build
@@ -45,7 +45,7 @@ else:
             'sqlite',
             'ssl3',
         ]
-        gyp_vars['nspr_libs'] = 'nspr4 plc4 plds4'
+        gyp_vars['nspr_libs'] = 'nspr'
 
     # This disables building some NSS tools.
     gyp_vars['mozilla_client'] = 1
@@ -73,8 +73,8 @@ else:
     # System sqlite here is the in-tree mozsqlite.
     gyp_vars['use_system_sqlite'] = 1
     gyp_vars['sqlite_libs'] = 'sqlite'
-    gyp_vars['nspr_include_dir'] = CONFIG['NSPR_INCLUDE_DIR']
-    gyp_vars['nspr_lib_dir'] = CONFIG['NSPR_LIB_DIR']
+    gyp_vars['nspr_include_dir'] = '%' + CONFIG['NSPR_INCLUDE_DIR']
+    gyp_vars['nspr_lib_dir'] = '%' + CONFIG['NSPR_LIB_DIR']
     # The Python scripts that detect clang need it to be set as CC
     # in the environment, which isn't true here. I don't know that
     # setting that would be harmful, but we already have this information


But I still need to think about it some more whether it's the way we should fix it on mozilla-central.

The nspr-build.m4 thing, along with bug 1329272, also hilights a pattern of bad things in the m4 files.
Mike - can you take this?
Flags: needinfo?(mh+mozilla)
Yeah, sorry, I probably should have moved those bits to moz.configure while touching them but I was already feeling overwhelmed at the scope of work for the NSS gyp build changes.
Assignee: nobody → mh+mozilla
Flags: needinfo?(mh+mozilla)
Note this is a rare combination of options. That is, only --with-system-nspr *without* --with-system-nss fails (afaik).
Summary: --with-system-nspr fails to build → --with-system-nspr without --with-system-nss fails to build
(In reply to Mike Hommey [:glandium] from comment #5)
> Note this is a rare combination of options. That is, only --with-system-nspr
> *without* --with-system-nss fails (afaik).

Rare? mozilla-central very often requires NSS version that hasn't been released.
Product: Core → Firefox Build System
Blocks: 1479787
Attachment #9000146 - Flags: review?(core-build-config-reviews)
Attachment #9000148 - Flags: review?(core-build-config-reviews)
Comment on attachment 9000147 [details] [diff] [review]
Avoid m4 breaking the sed expressions that set NSPR_INCLUDE_DIR and NSPR_LIB_DIR

Review of attachment 9000147 [details] [diff] [review]:
-----------------------------------------------------------------

Clearly this is a well-tested code path.
Attachment #9000147 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 9000146 [details] [diff] [review]
Allow !- and %-prefixed paths in include paths processed by gyp

Review of attachment 9000146 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/frontend/gyp_reader.py
@@ +278,5 @@
> +                    if include.startswith('/'):
> +                        resolved = mozpath.abspath(mozpath.join(config.topsrcdir, include[1:]))
> +                    elif not include.startswith(('!', '%')):
> +                        resolved = mozpath.abspath(mozpath.join(mozpath.dirname(build_file), include))
> +                    if not include.startswith(('!', '%')) and not os.path.exists(resolved):

I really want to be able to say:

elif include.startswith(('!', '%')):
  # Feed it directly to LOCAL_INCLUDES.
  goto LOCAL_INCLUDES;
else:
  resolved = ...
if not os.path.exists(resolved):
  continue
LOCAL_INCLUDES:
  # Fallthrough.
  pass

But I don't see how to do that without significantly restructuring this code.
Attachment #9000146 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 9000148 [details] [diff] [review]
Fix building in-tree NSS against system NSPR

Review of attachment 9000148 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/moz.build
@@ +56,5 @@
>              'smime3',
>              'sqlite',
>              'ssl3',
>          ]
> +        gyp_vars['nspr_libs'] = 'nspr'

Why are we dropping plc4/plds4 here?
Attachment #9000148 - Flags: review?(core-build-config-reviews) → review+
(In reply to Nathan Froyd [:froydnj] from comment #12)
> Comment on attachment 9000148 [details] [diff] [review]
> Fix building in-tree NSS against system NSPR
> 
> Review of attachment 9000148 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: security/moz.build
> @@ +56,5 @@
> >              'smime3',
> >              'sqlite',
> >              'ssl3',
> >          ]
> > +        gyp_vars['nspr_libs'] = 'nspr'
> 
> Why are we dropping plc4/plds4 here?

nspr is a pseudo-library that either adds nspr4 + plc4 + plds4 to USE_LIBS or NSPR_LIBS to OS_LIBS.
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/mozilla-inbound/rev/eb76188a858d
Allow !- and %-prefixed paths in include paths processed by gyp. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0a7377f4f60
Avoid m4 breaking the sed expressions that set NSPR_INCLUDE_DIR and NSPR_LIB_DIR. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/f3c1b09b9477
Fix building in-tree NSS against system NSPR. r=froydnj
https://hg.mozilla.org/mozilla-central/rev/eb76188a858d
https://hg.mozilla.org/mozilla-central/rev/b0a7377f4f60
https://hg.mozilla.org/mozilla-central/rev/f3c1b09b9477
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Given the age of this bug, I'm assuming the fix can ride the trains. Feel free to nominate if I'm misunderstanding, though.
You need to log in before you can comment on or make changes to this bug.