663471 - xpcom/glue and other files are apparently built without PGO

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Justin Lebar (not reading bugmail)]

Attachment: Object files that aren't PGO'ed - output of shell script from comment 0

I ran the following command on a Linux-64 PGO build; in theory it should tell me which object files were built without PGO.

  cat <(find . -name '*.o' | sort) <(find . -name '*.gcda' | sed 's/gcda$/o/' | sort) | sort | uniq -u | grep -iv test 

Full output is attached.  Most of the omissions are expected.  Obviously assembly files aren't PGO'ed, and NSS isn't, which is fine.

I was surprised to see that xpcom/glue and subdirs are apparently not PGO'ed, however.  There are a few other surprising (to me) ones, but I guess most of them (e.g. zlib) aren't PGO'ed because they use a custom build process.

We should check that everything that isn't PGO'ed isn't PGO'ed for a reason.

Comment 1

[(not currently active) Ted Mielczarek]

As a start:
http://mxr.mozilla.org/mozilla-central/search?string=NO_PROFILE_GUIDED_OPTIMIZE

and related:
http://mxr.mozilla.org/mozilla-central/source/config/config.mk#290

zlib should be using our own Makefiles, so I would expect it to get PGOed. Does GCC generate a .gcda file for every object file, even if that code isn't hit during the profiling run?

Comment 2

[Justin Lebar (not reading bugmail)]

> Does GCC generate a .gcda file for every object file, even if that code isn't 
> hit during the profiling run?

Answering this question was actually the reason I ran this command in the first place.  Surely plenty of files in our tree which don't appear in the attachment aren't being hit by our profiling script -- for instance, there's no SVG in there, right? -- but I'll write my own test just in case.

Comment 3

[Justin Lebar (not reading bugmail)]

Just did a simple test, and it appears that gcc does indeed output .gcda files for object files which have no code hit during the profiling run.

Comment 4

[Justin Lebar (not reading bugmail)]

(In reply to comment #0)
>   cat <(find . -name '*.o' | sort) <(find . -name '*.gcda' | sed
> 's/gcda$/o/' | sort) | sort | uniq -u | grep -iv test 

There's a bit of excessive sorting in here.  Meant to take those out...  :)

Comment 5

[Benjamin Smedberg]

It appears to me that:

build/unix is because those are tools, not part of the profiling run
nsLayoutDebugger.cpp contains only #ifdef DEBUG code, and so is not included in any binaries. So is BlockingResourceBase/DeadlockDetector and a few other things.
All of modules/libbz2 is only used by the updater, which is not profiled.
Ditto modules/libmar and bsdiff
modules/libreg/standalone is dead code, we should stop building it
modules/zlib/standalone is either dead or not used in the main product.
We don't use all of modules/zlib/src, because we don't use its file operations, so those objects which only deal with file operations are excluded by the linker.
xpcom/glue and xpcom/glue/standalone (and xpcom/glue/nomozalloc) are static libraries used by embedders and test components but are not included in a standard firefox build (except glandium is going to change some of that).

I think nsHtml5NamedCharactersAccel.o is listed because it is only data and has no functions. The intl files may be similar.

Most of this looks either harmless or intentional.

Comment 6

[Justin Lebar (not reading bugmail)]

Thanks, bsmedberg.

> xpcom/glue and xpcom/glue/standalone (and xpcom/glue/nomozalloc) are static 
> libraries used by embedders and test components but are not included in a 
> standard firefox build (except glandium is going to change some of that).

So until things change, the real xpcom/glue objdir is xpcom/build?

Comment 7

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to comment #6)
> So until things change, the real xpcom/glue objdir is xpcom/build?

If by real you mean "the copy of the code that ends in up Gecko" then yes.

Comment 8

[Justin Lebar (not reading bugmail)]

I filed bug 663810 on removing modules/libreg/standalone.