661803 - makefiles: export targets need real dependencies

Status: NEW

(Firefox Build System :: General, defect)

Description

[Joey Armstrong [:joey]]

export targets are using a blind cp to install exported headers.  Real deps should be used to avoid avoid later shells/running nsinstall.

Is there any reason these headers could not be used in place from their checkin directory within the source tree and avoid the installation overhead altogether ?


% gmake -f client.mk configure
cd /mozilla/mozilla-central/obj-ff-dbg
/mozilla/mozilla-central/configure
Adding configure options from /home/joey/.mozconfig:
[snip]

% gmake -C obj-ff-dbg/db/mdb export
gmake: Entering directory `/mozilla/mozilla-central/obj-ff-dbg/db/mdb'
gmake[1]: Entering directory `/mozilla/mozilla-central/obj-ff-dbg/db/mdb/public'
/mozilla/mozilla-central/obj-ff-dbg/config/nsinstall -R -m 644 /mozilla/mozilla-central/db/mdb/public/mdb.h ../../../dist/include

% cd obj-ff-dbg/db/mdb

##
# mdb.h header installed correctly under dist/include
##
% ls -l ../../../dist/include/mdb.h /mozilla/mozilla-central/db/mdb/public/mdb.h
lrwxrwxrwx 1 joey joey     44 2011-06-03 08:57 ../../../dist/include/mdb.h -> /mozilla/mozilla-central/db/mdb/public/mdb.h
-rw-r--r-- 1 joey joey 124939 2011-06-01 14:16 /mozilla/mozilla-central/db/mdb/public/mdb.h

##
# subsequent calls should be a nop but nsinstall mdb.h will be
# invoked on each subsequent attempt.
##
% gmake export | grep nsinstall
/mozilla/mozilla-central/obj-ff-dbg/config/nsinstall -R -m 644 /mozilla/mozilla-central/db/mdb/public/mdb.h ../../../dist/include

% gmake export | grep nsinstall
/mozilla/mozilla-central/obj-ff-dbg/config/nsinstall -R -m 644 /mozilla/mozilla-central/db/mdb/public/mdb.h ../../../dist/include


##
# Header and symlink still exist, timestamps have not changed
##
cd public; ls -l ../../../dist/include/mdb.h /mozilla/mozilla-central/db/mdb/public/mdb.h
lrwxrwxrwx 1 joey joey     44 2011-06-03 08:57 ../../../dist/include/mdb.h -> /mozilla/mozilla-central/db/mdb/public/mdb.h
-rw-r--r-- 1 joey joey 124939 2011-06-01 14:16 /mozilla/mozilla-central/db/mdb/public/mdb.h

Comment 1

[Benjamin Smedberg]

Note that nsinstall does its own dependency checking (only installing the file if it is newer).

In general, what we do is:

nsinstall many.h header.h files.h listed.h here.h <directory>

In which case it is actually faster for nsinstall to do the dep checking than to have make do it, because then make has to run a separate nsinstall command for each header. At least for clobber builds... there might be a depend/clobber tradeoff here somewhere.

Comment 2

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

For a full toplevel depend build we blow away dist/include anyways.  http://mxr.mozilla.org/mozilla-central/source/Makefile.in#91

Comment 3

[(not currently active) Ted Mielczarek]

(In reply to comment #0)
> Is there any reason these headers could not be used in place from their
> checkin directory within the source tree and avoid the installation overhead
> altogether ?

khuey and I have talked about this, there are a few snags:
1) Currently we have a concept of public and private headers, public headers get exported to dist/include, private headers don't. Maybe this distinction is silly and we should just force everyone to include things from srcdir-relative paths? (And only SDK headers would be "externally public"?)
2) We've started moving to EXPORTS_namespaces to put public headers into namespaces, and the headers get installed to dist/include/<namespace> so they can be #included as "<namespace>/header.h". Making that work without installing files seems difficult or impossible.
3) We still have to install some headers because we use a lot of headers that are generated from .idl files via xpidl. This is less of a problem since we already have deps to generate them.

Comment 4

[Joey Armstrong [:joey]]

Ignore if this is a dup, not sure what happened to my last reply.

(In reply to comment #1)
> Note that nsinstall does its own dependency checking (only installing the
> file if it is newer).
> 
> In general, what we do is:
> 
> nsinstall many.h header.h files.h listed.h here.h <directory>
> 
> In which case it is actually faster for nsinstall to do the dep checking
> than to have make do it, because then make has to run a separate nsinstall
> command for each header. At least for clobber builds... there might be a
> depend/clobber tradeoff here somewhere.

It should be possible to add rules that will accumulate a list of modified headers and pass them in bulk to nsinstall rather than forking off a b'zillion individual calls [ yes that would be silly ].

If the makefiles are not allowed to make stale timestamp decisions there is no way to avoid random shell overhead during a nop / dependency builds.