295659 - static builds with webdav fail

Status: RESOLVED FIXED

(Core Graveyard :: WebDAV, defect)

Description

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

WebDAV isn't adding itself to the static module list, so builds fail with:

00:12 < beltzner> nsStaticComponents.obj : error LNK2001: unresolved external
                  symbol "struct nsModuleInfo nsWebDAVModules_gModuleInfo"
                  (?nsWebDAVModules_gModuleInfo@@3UnsModuleInfo@@A)

And right now, I can't find the list that it needs to add itself to.  Something
modules/staticmod related? Maybe?

Comment 1

[Mike Beltzner [:beltzner, not reading bugmail]]

Some context & motivation:

I'm trying to build the XPI in a fashion that is consumable by people running a
Thunderbird distribution (release or nightly, whatever) who are interested in
testing. AFAIK, this means doing a static build. So I got the free MS tools and
muddled through the process to get it all working, and then hit this error.

The .mozconfig is the standard thunderbird one along with:
    * ac_add_options --disable-shared
    * ac_add_options --enable-static
    * ac_add_options --enable-extensions=lightning,inspector,venkman

The error, oddly, occurs when building thunderbird.exe (or calendar.exe, if I
try to build calendar with the same ac_add_options):

Creating library thunderbird.lib and object thunderbird.exp
nsStaticComponents.obj : error LNK2001: unresolved external symbol "struct
nsModuleInfo sWebDAVModules_gModuleInfo"
(?nsWebDAVModules_gModuleInfo@@3UnsModuleInfo@@A)
thunderbird.exe : fatal error LNK1120: 1 unresolved externals

Since I'm not even sure that what I need to build a distributable XPI is a
static build, I may be totally barking up the wrong tree; but either way, it
seems odd that thunderbird.exe is trying to touch WebDAVModules.

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

For the lightning xpi, the end result should have 4 native components:
calbasecomps, storagecomps, webdav, and xmlextras (since it's not part of
tbird).  Each of those needs to be a dynamic library.  However, right now each
of those links with xpcom/js dynamically:

vladimir@cyclone[1068]% ldd
~/proj/mozilla-cvs/tbird/dist/xpi-stage/lightning/components/libwebdav.so 
	libxpcom.so => not found
	libxpcom_core.so => not found

Those need to get statically linked into each component.  I've no idea how/if
the build system can handle that.

Comment 3

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

No, I take that back, we ship that stuff as dll's.. so you shouldn't need
--enable-static to produce a shippable lightning.xpi.  I don't think.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

> No, I take that back, we ship that stuff as dll's.. so you shouldn't need
> --enable-static to produce a shippable lightning.xpi.  I don't think.

Agreed, but if you have a thunderbird build environment that mimics the official
nightlies, then you are producing static builds. It would be great to use that
Thunderbird tree and objdir to also produce lightning, rather than need a shared
objdir as well.

Having said that, it's not obvious how you produce the normal lightning.xpi
(with shared components) in the context of a static Thunderbird build; ie
avoiding linking lightning components into the local Thunderbird, which are
missing on official releases. Some investigations yielded

1) xmlextras & webdav are in the call to link thunderbird.exe (using MS VC6).
xmlextras is also in the default extensions list for thunderbird.
2) The other lightning components (storage, calbase) don't get beyond lib,
presumably because of a global static variable. Perhaps this can be fixed with
FORCE_SHARED_LIB dependent on something like MOZ_LIGHTNING.

Nick (CF on irc)

Comment 5

[Andy Willis (abwillis)]

I manually edited the nsStaticComponents.cpp that gets created from the
nsStaticComponents.cpp.in and changed the 2 occurences of nsWebDAVModules to
nsWebDAVService and then the build completed.  I am going to now try changing
the only occureence of nsWebDAVModules in the code
(extensions\webdav\src\makefile.in ) to nsWebDAVService and see if that corrects
the issue.  This worked for me on the 26th (the build I am currently using) but
when I next tried I got this error.  As this was opened on the 26th the code
must have been checked in on the 26th that affected this but looking through
bonzai nothing sticks out as being the cause.

Comment 6

[Andy Willis (abwillis)]

Attachment: change module name

This fixes the build break but I can't test the webdav component.

Comment 7

[Mike Shaver (:shaver emeritus)]

I think Nick has hit the crux of the issue; we need to avoid having webdav
appear in the static-components list when we're building it for our extension.

Benjamin: what would you say to having XPI_PKGNAME trump the --enable-static for
those purposes?  Do you think we'd need to tweak anything else as well?

Comment 8

[Benjamin Smedberg]

Attachment: XPI_NAME trumps static builds, rev. 1

This is an educated-guess patch: it needs testing.

Comment 9

[Mike Beltzner [:beltzner, not reading bugmail]]

Using that patch, and the following .mozconfig:

. $topsrcdir/mail/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../../lightning-static
mk_add_options MOZ_CO_PROJECT=mail

ac_add_options --disable-accessibility
ac_add_options --disable-activex
ac_add_options --enable-static
ac_add_options --disable-shared
ac_add_options --disable-tests
ac_add_options --enable-extensions=lightning,inspector,venkman

I got the following build error:

Building deps for /cygdrive/f/mozilla/mozilla/calendar/lightning/../../db/sqlite
3/src/where.c
where.c
Creating Resource file: module.res
nsinstall: sqlite3_s.lib: No such file or directory
make[6]: *** [libs] Error 3
make[5]: *** [libs] Error 2
make[4]: *** [libs] Error 2
make[3]: *** [libs] Error 2
make[2]: *** [tier_94] Error 2
make[1]: *** [default] Error 2
make: *** [build] Error 2

Is it possible that we now need to explicitly enable storage due to the patch?

Comment 10

[Benjamin Smedberg]

Comment on attachment 185464 [details] [diff] [review]
XPI_NAME trumps static builds, rev. 1

This needs at least an extra test for IS_COMPONENT.

Comment 11

[Andy Willis (abwillis)]

With the patch  https://bugzilla.mozilla.org/attachment.cgi?id=185464
# sh
# Build configuration script
#
# See http://www.mozilla.org/build/unix.html for build instructions.
#

# Options for client.mk.
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj
mk_add_options MOZ_CVS_FLAGS="-q -z 9"
mk_add_options MOZ_CO_PROJECT=suite,browser,mail,composer,calendar,xulrunner
mk_add_options MOZ_CO_MODULE=mozilla/other-licenses/libart_lgpl

# Options for 'configure' (same as command-line options).
ac_add_options --disable-tests
ac_add_options --enable-optimize
ac_add_options --disable-debug
ac_add_options --enable-crypto
ac_add_options --enable-calendar
ac_add_options --with-extensions=all
ac_add_options --enable-application=suite
ac_add_options --disable-shared
ac_add_options --enable-static
ac_add_options --enable-svg-renderer=libart
ac_add_options --enable-svg
ac_add_options --enable-storage


o) : error LNK2029: "_sqlite3_value_type" : unresolved external
E:\cvs\work\mozilla\obj\storage\src\mozStorageValueArray.o(mozStorageValueArray.
o) : error LNK2029: "_sqlite3_column_int" : unresolved external
E:\cvs\work\mozilla\obj\storage\src\mozStorageStatement.o(mozStorageStatement.o)
 : error LNK2029: "_sqlite3_column_int" : unresolved external
E:\cvs\work\mozilla\obj\storage\src\mozStorageStatement.o(mozStorageStatement.o)
 : error LNK2029: "_sqlite3_column_name" : unresolved external


There were 140 errors detected
make.exe[6]: *** [storage_s.dll] Error 1
make.exe[6]: Leaving directory `E:/cvs/work/mozilla/obj/storage/src'
make.exe[5]: *** [libs] Error 2

Comment 12

[Mike Shaver (:shaver emeritus)]

Attachment: check IS_COMPONENT too [checked in]

Added IS_COMPONENT requirement for the trumping.  Not really tested either, but
it's something!

Comment 13

[Simon Paquet [:sipaq]]

Comment on attachment 186023 [details] [diff] [review]
check IS_COMPONENT too [checked in]

With this patch I get the following when doing an optimized static build of
Sunbird on WinXP:

   Bibliothek xmlextras.lib and Objekt xmlextras.exp are created
xmlextrasbase_s.lib(nsXMLHttpRequest.obj) : error LNK2019: unresolved external 
		 symbol "public: int __thiscall nsRect:
:Contains(int,int)const " (?Contains@nsRect@@QBEHHH@Z), referenced in function
"public: int __thiscall nsRect::Contains(struct
 nsPoint const &)const " (?Contains@nsRect@@QBEHABUnsPoint@@@Z)
xmlextrasbase_s.lib(nsXMLHttpRequest.obj) : error LNK2019: unresolved external
symbol "public: void __thiscall nsRect
::Inflate(int,int)" (?Inflate@nsRect@@QAEXHH@Z), referenced in function
"public: void __thiscall nsRect::Inflate(struct nsSize
 const &)" (?Inflate@nsRect@@QAEXABUnsSize@@@Z)
xmlextrasbase_s.lib(nsXMLHttpRequest.obj) : error LNK2019: unresolved external
symbol "public: void __thiscall nsRect
::Deflate(int,int)" (?Deflate@nsRect@@QAEXHH@Z), verwiesen in Funktion "public:
void __thiscall nsRect::Deflate(struct nsSize
 const &)" (?Deflate@nsRect@@QAEXABUnsSize@@@Z)
xmlextras.dll : fatal error LNK1120: 3 unresolved external symbols 
make[8]: *** [xmlextras.dll] Error 96
make[8]: Leaving directory
`/cygdrive/c/mozilla/Sunbird_Win/extensions/xmlextras/build/src'
make[7]: *** [libs] Error 2
make[7]: Leaving directory
`/cygdrive/c/mozilla/Sunbird_Win/extensions/xmlextras/build'
make[6]: *** [libs] Error 2
make[6]: Leaving directory
`/cygdrive/c/mozilla/Sunbird_Win/extensions/xmlextras'
make[5]: *** [libs] Error 2
make[5]: Leaving directory `/cygdrive/c/mozilla/Sunbird_Win/extensions/webdav'
make[4]: *** [libs] Error 2
make[4]: Leaving directory `/cygdrive/c/mozilla/Sunbird_Win/calendar/xpi'
make[3]: *** [libs] Error 2
make[3]: Leaving directory `/cygdrive/c/mozilla/Sunbird_Win/calendar'
make[2]: *** [tier_98] Error 2
make[2]: Leaving directory `/cygdrive/c/mozilla/Sunbird_Win'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/cygdrive/c/mozilla/Sunbird_Win'
make: *** [build] Error 2

Comment 14

[Mike Shaver (:shaver emeritus)]

peterv: can you take a peek at comment 13 and tell us what's up?  I don't see
any references to nsRect in that file, or indeed in all of extensions/xmlextras,
so I'm pretty confused.  Some contentutils thing?

Comment 15

[Peter Van der Beken [:peterv]]

I don't know *why* that happens. nsRect.h probably gets pulled in through
nsGUIEvent.h but that shouldn't be a problem, right?

Comment 16

[Mike Shaver (:shaver emeritus)]

Attachment: Make sure nsRect's symbols are around if needed.

bsmedberg not only suggested this, he convinced me it wasn't all that gross. 
If we had a decently build _s.lib for this stuff, a la the unicode utils, we
could do something that wouldn't require as many lines of comments.

My enable-static sunbird build on OS X seems to be OK with this, at least in
principle.  Could use some more testing from the actually-afflicted, though.

Comment 17

[Benjamin Smedberg]

Comment on attachment 186098 [details] [diff] [review]
Make sure nsRect's symbols are around if needed.

In order for nsRect.cpp to compile and link properly in non-static builds, you
need to add DEFINES += -D_IMPL_NS_GFX to this makefile and any of the xmlextras
makefiles that reference nsRect.h

Comment 18

[Mike Beltzner [:beltzner, not reading bugmail]]

Shaver; is this patch still in need of testing, or have you already gotten
further progress? I know you were playing around a bit more with this today ...

Comment 19

[Mike Shaver (:shaver emeritus)]

No, there are lots of other things wrong with the static-build story, at least
on OS X, so I'm still plugging away.  I think I'm gonna try to get
lightning-as-part-of-static-tbird working, then figure out static sunbird again.

Comment 20

[Mike Shaver (:shaver emeritus)]

Comment on attachment 185442 [details] [diff] [review]
change module name

checked this in; it's obviously correct and necessary.	Thanks, Andy.

Comment 21

[Mike Shaver (:shaver emeritus)]

Attachment: Move the nsRect graft to base/src, define _IMPL_NS_GFX

Moved the nsRect import into base/src, where it's closer to the code that
requires it, and defined the magic _IMPL_NS_GFX.  Needs testing on static
(underway on my machine) and dynamic (please help if you can!) builds both.

Comment 22

[Mike Shaver (:shaver emeritus)]

Comment on attachment 186023 [details] [diff] [review]
check IS_COMPONENT too [checked in]

Committed this patch.

Comment 23

[Christian :Biesinger (don't email me, ping me on IRC)]

so er, why not link to libgfx instead of doing this? a lot of modules are
already doing that...

Comment 24

[Mike Shaver (:shaver emeritus)]

Because an extension that packages xmlextras can't know if it's being installed
into a dynamic or static host app.

Comment 25

[Mike Shaver (:shaver emeritus)]

Comment on attachment 186320 [details] [diff] [review]
Move the nsRect graft to base/src, define _IMPL_NS_GFX

requesting a= to fix lighting and xmlextras-as-XPI-extension-in-static-build
generally

Comment 26

[Mike Shaver (:shaver emeritus)]

Oh my sweet heavens, I think this is FIXED.  If you are not seeing the _exact_
symptom reported in the original description of this bug, please do not reopen
this; we have lots of bug numbers left.