160180 - Unable to build on NetBSD

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Felicia Neff]

When I try to build on NetBSD-1.5.2, I get the following error:

nsNativeCharsetUtils.cpp: In function `static void
nsNativeCharsetConverter::LazyInit()':
nsNativeCharsetUtils.cpp:255: `CODESET' undeclared (first use this function)
nsNativeCharsetUtils.cpp:255: (Each undeclared identifier is reported only once
nsNativeCharsetUtils.cpp:255: for each function it appears in.)
gmake[3]: *** [nsNativeCharsetUtils.o] Error 1

Comment 1

[hacker formerly known as [e-mail]]

Is CODESET defined in any NetBSD system headers?

Over to xpcom. Cc'ing the usual suspects.

Comment 2

[Darin Fisher]

i don't know the first thing about NetBSD, but if i could get access to a NetBSD
system... i'd be more than willing to investigate a solution.

Comment 3

[Felicia Neff]

No, CODESET doesn't appear to be in any of the NetBSD system header files.

If an account on the system I work on would help, one can be made available.
-- Felicia

Comment 4

[Darin Fisher]

sure, that would help.  basically, we just need to figure out if the wchar_t
format is unicode.  if it is, then we don't even need to bother with trying to
call nl_langinfo.  if it is not, then we need to figure out why our autoconf
tests don't seem to cut the mustard (i.e., they seemed to tell us that CODESET
exists, and yet it doesn't!).

Comment 5

[Felicia Neff]

According to the NetBSD developers when asked in wchar_t was unicode:
        no, wchar_t is not UCS4.  you need to handle wchar_t values as opaque
        (we support a lot more things than Unicode, and there's no standard
        that says wchar_t is UCS4).

Comment 6

[Roland Mainz]

Felicia Neff wrote:
> According to the NetBSD developers when asked in wchar_t was unicode:

|wchar_t| was never Unicode - neither UCS2 nor UCS4 - only Linux tries to treat
|wchar_t| as UCS2 (which results in the problem that it will never be able to
support locales like ja_JP.PCK that way (IMHO the inventor of the
xx@@@!!!-"|wchar_t|-is-16bit"-idea should burn in h*ll)).
AFAIK a standard-conformant |wchar_t| is >=32bit and the content is
platform+locale-specific and an application must not make any special assumtions
about the used encoding.

Comment 7

[Darin Fisher]

then NetBSD definitely needs to use iconv, and that requires being able to call
nl_langinfo(CODESET) to determine the filesystem's charset.

Comment 8

[Felicia Neff]

I was able to build previous versions without any problems.  How has this
changed?  It looks like it is CODESET that is the problem.  It is defined for
NetBSD-current but not NetBSD-1.5.

Comment 9

[Darin Fisher]

mozilla recently started requiring native unicode -> filesystem charset
conversion.  on windows this means calling MultiByteToWideChar, etc.  and on
unix systems it either means calling mbtowc or iconv.  on unix if wchar_t is not
unicode, then iconv must be used.  but iconv requires knowledge of the src
charset and the destination charset.  the standard way to learn the filesystem
charset under unix is to call nl_langinfo(CODESET).  if this is not supported,
then we have to guess the charset based on the locale, which currently is not
implemented.  we just fallback on ISO-8859-1.  the compilation error is
confusing since it seems to suggest that our configure test is broken.

Comment 10

[Felicia Neff]

The config.log isn't very helpful.  Does configure test for nl_langinfo with
CODESET?  If you like, I can run some tests for you.

Comment 11

[kill this account]

no, it just test for nl_langinfo
http://lxr.mozilla.org/seamonkey/source/configure.in#1937
1937 AC_CHECK_FUNCS(nl_langinfo strtok_r flockfile)

Comment 12

[Darin Fisher]

well, then we need to do better... and we need to figure out what to do if
CODESET is not defined.  we need to guess the charset or something... or is
ISO-8859-1 OK for NetBSD?

Comment 13

[Felicia Neff]

On NetBSD-1.6, nl_langinfo(CODESET) returns "646", which is ASCII.  I am
guessing that this is true for NetBSD-1.5 as well.

Comment 14

[Felicia Neff]

It has been two months and a couple mozilla releases.  Is there any chance that
anything is going to be done about this problem?

Comment 15

[Adrian Bunk]

*** Bug 167146 has been marked as a duplicate of this bug. ***

Comment 16

[Adrian Bunk]

I marked my bug as a duplicate of your bug. I did try to find whether it  
was already a reported issue when I submitted my bug but I did
oversee this bug.

Comment 17

[Adrian Bunk]

FYI:

gettext ships in codeset.m4 the following autoconf test for nl_langinfo(CODESET):


AC_DEFUN([AM_LANGINFO_CODESET],
[
  AC_CACHE_CHECK([for nl_langinfo and CODESET], am_cv_langinfo_codeset,
    [AC_TRY_LINK([#include <langinfo.h>],
      [char* cs = nl_langinfo(CODESET);],
      am_cv_langinfo_codeset=yes,
      am_cv_langinfo_codeset=no)
    ])
  if test $am_cv_langinfo_codeset = yes; then
    AC_DEFINE(HAVE_LANGINFO_CODESET, 1,
      [Define if you have <langinfo.h> and nl_langinfo(CODESET).])
  fi
])

Comment 18

[kill this account]

I think these are the steps to use this test

add codeset.m4 to <MOZDIR>/build/autoconf/

in configure.in change

  AC_CHECK_FUNCS(nl_langinfo flockfile)

to 

  AC_CHECK_FUNCS(flockfile)
  AM_LANGINFO_CODESET()

and then change all occurances of

  HAVE_NL_LANGINFO

to

  HAVE_LANGINFO_CODESET

Comment 19

[thebeastwitheyesthatstared]

Darin/Doug - what is the opinion on the last two suggested fixes to 
codeset.m4 - did this stop happening, maybe?

I am no developer, and this comment is only to show my interest in seing this 
bug touched - I will gladly test any experimental changes for the NetBSD build.

Are there no PCs at @netscape.com, Darin, where you could install NetBSD? I 
only have an old laptop, very little free space, plus my compiles take a long 
time - not to mention that Mozilla on NetBSD seems slower than the linux build.

Comment 20

[Adrian Bunk]

FYI:
This bug still exists in Mozilla 1.4.1

Comment 21

[Darin Fisher]

sorry guys... this is very low on my priority list.  hopefully someone with an
already existing NetBSD setup can give Brian's suggested fix a try.

Comment 22

[Adrian Bunk]

Attachment: suggested fix (roughly similar to Brian's suggestion)

This patch is a suggested fix against Mozilla 1.6 .

Please doublecheck it - I'm not very familar with the autoconf setup in
Mozilla.

Comment 23

[Christian :Biesinger (don't email me, ping me on IRC)]

this is gpl code and therefore not usable for mozilla. (last file in the patch).

Comment 24

[Christian :Biesinger (don't email me, ping me on IRC)]

sorry, I saw the exception clause only later, so nevermind me.

Comment 25

[Christian :Biesinger (don't email me, ping me on IRC)]

Comment on attachment 139821 [details] [diff] [review]
suggested fix (roughly similar to Brian's suggestion)

hm... why do you have 
+AC_SUBST(HAVE_LANGINFO_CODESET)
in configure.in, and what is the autoconf.mk.in change for?

Comment 26

[Adrian Bunk]

As said, I'm not very familar with the autoconf setup in Mozilla.

AM_LANGINFO_CODESET might define HAVE_LANGINFO_CODESET.

What is the correct way to pass a #define from configure to the C++ files?

Comment 27

[Christian :Biesinger (don't email me, ping me on IRC)]

(In reply to comment #26)
> What is the correct way to pass a #define from configure to the C++ files?

Well, AC_DEFINE as usual - the codeset.m4 file does that already.

AC_SUBST passes stuff to Makefiles, which can access it as @FOO@

Comment 28

[Adrian Bunk]

Attachment: updated patch

Thanks for this explanations.

I've tried a build without the two parts you noted and it works.

Is this updated patch OK, or are there still problems with it?

Comment 29

[Christian :Biesinger (don't email me, ping me on IRC)]

I'd like to defer that question to bsmedberg who knows the build system stuff
better than me...

Comment 30

[Benjamin Smedberg]

Comment on attachment 139821 [details] [diff] [review]
suggested fix (roughly similar to Brian's suggestion)

passing the buck ;)

Comment 31

[cls]

Comment on attachment 139821 [details] [diff] [review]
suggested fix (roughly similar to Brian's suggestion)

Neither this patch (nor the one not marked as obsolete) contain the change to
remove the old check for nl_langinfo.

Comment 32

[Adrian Bunk]

Attachment: second update of the patch

Thanks for your comments.

This version of the patch completely removes the nl_langinfo() check.

It's made against 1.6, but it applies against current 1.7 branch, and I've
checked that it is also correct against 1.7 .

Comment 33

[Asa Dotzler [:asa]]

Comment on attachment 148153 [details] [diff] [review]
second update of the patch

a=asa (on behalf of drivers) for checkin to 1.7

Comment 34

[cls]

The patch has been checked in on the trunk & the 1.7 branch.

Comment 35

[cls]

*** Bug 256069 has been marked as a duplicate of this bug. ***