461652 - Build of xulrunner 1.9.0.3 fails with python/nsdom enabled

Status: RESOLVED FIXED

(Other Applications Graveyard :: PyXPCOM, defect)

Description

[lkcl]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.14) Gecko/20080404 Iceweasel/2.0.0.14 (Debian-2.0.0.14-2)
Build Identifier: 1.9.0.3

make[5]: Entering directory `/mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom'
make[6]: Entering directory `/mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom/src'
rm -f libpydom.so
c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-long-long -g -fno-strict-aliasing -pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fno-reorder-functions  -DPYTHON_SO=\"libpython2.5.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o nsPyDOMISupports.o nsPyTimeout.o     -lpthread   -Wl,-rpath-link,../../../../dist/bin  ../../../../dist/lib/libxpcomglue_s.a -L../../../../dist/bin -lxpcom -L../../../../dist/bin -lxpcom -lxul  -L../../../../dist/bin -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl  -Wl,--version-script -Wl,../../../../build/unix/gnu-ld-scripts/components-version-script -Wl,-Bsymbolic -ldl -lm  -L/usr/lib -lpython2.5 -lutil -L../../../../dist/lib -lpyxpcom  -L../../../../dist/bin -lmozjs  
nsPyContext.o: In function `nsPythonContext::Deserialize(nsIObjectInputStream*, nsScriptObjectHolder&)':
/mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom/src/nsPyContext.cpp:728: undefined reference to `PyMarshal_ReadObjectFromString'
/usr/bin/ld: nsPyContext.o: relocation R_X86_64_PC32 against `PyMarshal_ReadObjectFromString' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value



Reproducible: Always

Steps to Reproduce:
1. apt-get source xulrunner-1.9
2. apt-get build-dep xulrunner-1.9
3. edit debian/mozconfig and change "python/xpcom" to "python".
4. dpkg-buildconfig -rfakeroot -nc
Actual Results:  
failed build.

Expected Results:  
successful build

http://webui.sourcelabs.com/fedora/issues/436581 states that this should be fixed:

"Fixed in xulrunner 1.9.0.2-3"

it's not.

Comment 1

[Benjamin Smedberg]

Not an XPCOM bug, and we don't seem to have a PyXPCOM component. AFAIK this code is not maintained.

Comment 2

[lkcl]

don't know where this bug is supposed to go (so i submitted it for xpcom).

it's a rather obscure but really quite... important feature (nsdom) that opens up *massive* possibilities.  two very big projects - komodo and miro - are using nsdom.  komodo include nsdom but they only produce x86 builds.

if this error is fixed, xulrunner can be used as the back-end for a widget set API that competes with - and is a lot better than - python-qt4 and pygtk2.  see http://pyjd.org for details.  a successful port to webkit has been made.

i'm looking to get ports to python-kde (using python bindings to khtmlpart) and to xulrunner (using python-nsdom).

Comment 3

[lkcl]

research into the various links - esp the fedora one - shows that if you change #include "marshal.h" to #include <marshal.h> and also add a marshal.h into dist/include/system_wrappers.h that this problem happily goes away.

i.e. you get a link.

whether it _works_ or not is another matter, yet to be determined by my beautifully-slow laptop...

Comment 4

[lkcl]

make[7]: Entering directory `/mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom/test/pyxultest'
if test ! -d ../../../../../dist/xpi-stage/pyxultest/defaults/preferences; then /mnt/video/src/xulrunner-1.9.0.3/config/nsinstall -D ../../../../../dist/xpi-stage/pyxultest/defaults/preferences; fi
set -e;   \
        for i in ./pyxultest-prefs.js; do \
          dest=../../../../../dist/xpi-stage/pyxultest/defaults/preferences/`basename $i`; \
          rm -f -f $dest; \
          /usr/bin/python ../../../../../config/Preprocessor.py  -DOSTYPE=\"Linux2.6\" -DOSARCH=Linux -DHAVE_64BIT_OS=1 -DMOZILLA_VERSION=\"1.9.0.3\" -DMOZILLA_VERSION_U=1.9.0.3 -DD_INO=d_ino -DSTDC_HEADERS=1 -DHAVE_ST_BLKSIZE=1 -DHAVE_SIGINFO_T=1 -DHAVE_INT16_T=1 -DHAVE_INT32_T=1 -DHAVE_INT64_T=1 -DHAVE_UINT=1 -DHAVE_UNAME_DOMAINNAME_FIELD=1 -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -DHAVE_VISIBILITY_ATTRIBUTE=1 -DHAVE_DIRENT_H=1 -DHAVE_GETOPT_H=1 -DHAVE_SYS_BITYPES_H=1 -DHAVE_MEMORY_H=1 -DHAVE_UNISTD_H=1 -DHAVE_GNU_LIBC_VERSION_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_MALLOC_H=1 -DHAVE_X11_XKBLIB_H=1 -DHAVE_SYS_STATVFS_H=1 -DHAVE_SYS_STATFS_H=1 -DHAVE_MMINTRIN_H=1 -DHAVE_SYS_CDEFS_H=1 -DHAVE_LIBM=1 -DHAVE_LIBDL=1 -DHAVE_DLADDR=1 -DFUNCPROTO=15 -DHAVE_XSHM=1 -DHAVE_FT_BITMAP_SIZE_Y_PPEM=1 -DHAVE_FT_GLYPHSLOT_EMBOLDEN=1 -DHAVE_FT_LOAD_SFNT_TABLE=1 -DHAVE_FT_SELECT_SIZE=1 -D_REENTRANT=1 -DHAVE_RANDOM=1 -DHAVE_STRERROR=1 -DHAVE_LCHOWN=1 -DHAVE_FCHMOD=1 -DHAVE_SNPRINTF=1 -DHAVE_MEMMOVE=1 -DHAVE_RINT=1 -DHAVE_STAT64=1 -DHAVE_LSTAT64=1 -DHAVE_TRUNCATE64=1 -DHAVE_FLOCKFILE=1 -DHAVE_LOCALTIME_R=1 -DHAVE_STRTOK_R=1 -DHAVE_RES_NINIT=1 -DHAVE_GNU_GET_LIBC_VERSION=1 -DHAVE_LANGINFO_CODESET=1 -DVA_COPY=va_copy -DHAVE_VA_COPY=1 -DHAVE_VA_LIST_AS_ARRAY=1 -DHAVE_I18N_LC_MESSAGES=1 -DMOZ_EMBEDDING_LEVEL_DEFAULT=1 -DMOZ_EMBEDDING_LEVEL_BASIC=1 -DMOZ_EMBEDDING_LEVEL_MINIMAL=1 -DMOZ_XULRUNNER=1 -DMOZ_BUILD_APP=xulrunner -DMOZ_XUL_APP=1 -DMOZ_DEFAULT_TOOLKIT=\"cairo-gtk2\" -DMOZ_WIDGET_GTK2=1 -DMOZ_ENABLE_XREMOTE=1 -DMOZ_ENABLE_STARTUP_NOTIFICATION=1 -DMOZ_X11=1 -DMOZ_DISTRIBUTION_ID=\"org.mozilla\" -DMOZ_ENABLE_GCONF=1 -DMOZ_ENABLE_GNOMEUI=1 -DMOZ_ENABLE_DBUS=1 -DOJI=1 -DIBMBIDI=1 -DMOZ_VIEW_SOURCE=1 -DACCESSIBILITY=1 -DMOZ_XPINSTALL=1 -DMOZ_JSLOADER=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_NO_XPCOM_OBSOLETE=1 -DMOZ_XTF=1 -DMOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DMOZ_MATHML=1 -DMOZ_ENABLE_CANVAS=1 -DMOZ_SVG=1 -DMOZ_SVG_FOREIGNOBJECT=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_PLACES=1 -DMOZ_FEEDS=1 -DMOZ_STORAGE=1 -DMOZ_URL_CLASSIFIER=1 -DMOZ_LOGGING=1 -DSIZEOF_INT_P=8 -DMOZ_MEMORY_SIZEOF_PTR_2POW=3 -DMOZ_MEMORY=1 -DMOZ_MEMORY_LINUX=1 -DHAVE___CXA_DEMANGLE=1 -DHAVE__UNWIND_BACKTRACE=1 -DMOZ_DEFAULT_MOZILLA_FIVE_HOME=\"/usr/lib/xulrunner-1.9\" -DMOZ_USER_DIR=\".mozilla\" -DMOZ_ENABLE_LIBXUL=1 -DHAVE_STDINT_H=1 -DHAVE_INTTYPES_H=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DMOZ_RDF=1 -DMOZ_MORKREADER=1 -DMOZ_DLL_SUFFIX=\".so\" -DXP_UNIX=1 -DUNIX_ASYNC_DNS=1 -DJS_THREADSAFE=1 -DMOZ_ACCESSIBILITY_ATK=1 -DATK_MAJOR_VERSION=1 -DATK_MINOR_VERSION=22 -DATK_REV_VERSION=0 -DMOZILLA_LOCALE_VERSION=\"1.9a1\" -DMOZILLA_REGION_VERSION=\"1.9a1\" -DMOZILLA_SKIN_VERSION=\"1.8\"  -DNDEBUG -DTRIMMED $i > $dest; \
        done
+++ making chrome /mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom/test/pyxultest  => ../../../../../dist/xpi-stage/pyxultest/chrome/pyxultest.jar
+++ updating chrome ../../../../../dist/xpi-stage/pyxultest/chrome/pyxultest.manifest
Traceback (most recent call last):
  File "../../../../../config/Preprocessor.py", line 471, in <module>
    main()
  File "../../../../../config/Preprocessor.py", line 456, in main
    pp.handleCommandLine(None, True)
  File "../../../../../config/Preprocessor.py", line 202, in handleCommandLine
    self.do_include(f)
  File "../../../../../config/Preprocessor.py", line 441, in do_include
    self.handleLine(l)
  File "../../../../../config/Preprocessor.py", line 223, in handleLine
    self.write(aLine)
  File "../../../../../config/Preprocessor.py", line 149, in write
    aLine = f[1](aLine)
  File "../../../../../config/Preprocessor.py", line 403, in filter_substitution
    return self.varsubst.sub(repl, aLine)
  File "../../../../../config/Preprocessor.py", line 401, in repl
    raise Preprocessor.Error(self, 'UNDEFINED_VAR', varname)
__main__.Error: ('/mnt/video/src/xulrunner-1.9.0.3/extensions/python/dom/test/pyxultest/application.ini', 18, 'UNDEFINED_VAR', 'BUILD_ID')
make[7]: *** [libs] Error 1


wtf?? _come_ on, guys.  is _anyone_ building this really strategically important code??

Comment 5

[lkcl]

add these to pyxultest Makefile.in:

....

BUILD_ID = $(shell $(PYTHON) $(topsrcdir)/config/printconfigsetting.py $(LIBXUL_DIST)/bin/platform.ini Build BuildID)

DEFINES += -DBUILD_ID=$(BUILD_ID)

include $(topsrcdir)/config/rules.mk

Comment 6

[lkcl]

https://bugzilla.redhat.com/show_bug.cgi?id=436581

as suspected by michael stone, marshal.h is the key.  i've "avoided" the compile error by specifying #include <marshal.h> rather than #include "marshal.h".  exactly how you go about including /usr/include/{correctpythonversion}/marshal.h inside the system_wrappers/marshal.h i REALLY do not know.

perhaps you would need a marshal.h.in ?

Comment 7

[Takanori MATSUURA]

Do I have to add -I/usr/include/pythonx.y to compiler option?

Comment 8

[Takanori MATSUURA]

Comment #7 is properly set by MOZ_PYTHON_INCLUDES in config/autoconf.mk.
And MOZ_PYTHON_INCLUDES is added to LOCAL_INCLUDES in dom/src/Makefile.

But it is not reflected.

Comment 9

[Todd Whiteman]

@Takanori

Note that Pyxpcom on linux requires a shared Python build (i.e. a Python configured with --enable-shared), and this shared Python will need to be the first Python found on your PATH.

Another tip is to disable gcc visibility, you do this by adding the visibility flag to the configure line:
  ac_cv_visibility_pragma=no ./configure ...

Comment 10

[Martin Stránský [:stransky] (ni? me)]

Attachment: added marshal.h fto system headers

The built fails because system_wrappers are missing marshal.h file.

Comment 11

[Martin Stránský [:stransky] (ni? me)]

I can confirm that bug on Fedora 12 / Python 2.6.

Comment 12

[Todd Whiteman]

I run into the same problem when compiling against the system Python (Ubuntu 9.10).

Added this patch to pyxpcom hg repository (1.9.2):
http://hg.mozilla.org/pyxpcom/rev/2cb5add2914a