PGO has very few profile data on incremental builds

RESOLVED FIXED in mozilla9

Status

defect
RESOLVED FIXED
8 years ago
2 years ago

People

(Reporter: glandium, Assigned: glandium)

Tracking

Trunk
mozilla9
All
Linux
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: fixed-in-bs)

Attachments

(2 attachments, 3 obsolete attachments)

We don't rebuild the objects that are already in the objdir, and they are final objects, not ones with instrumentation... (doh)

So we end up running the profile without instrumentation in most object files.

The easy solution would be to clobber before the profile generate pass, but that would be a huge waste of time. I think an easy way to avoid this problem, while keeping fast(ish) builds is to change the object file extension in one of the passes so that we effectively never overwrite object files from the profile generate pass in the profile use pass. That would double disk usage for object files, though.

On the other hand, we could probably skip debugging symbols in the profile generate pass, which would make the object files smaller. (though they might be useful in the case running firefox for the profile crashes)
> The easy solution would be to clobber before the profile generate pass, but 
> that would be a huge waste of time.

Catlee has been working on patches to ccache to support -fprofile-generate and -fprofile-use.  If used that, clobbering wouldn't be so bad.

But of course what you're proposing sounds more clever...
Is this specific to Linux, or would this impact Windows as well?
This is specific to linux.
Assignee: nobody → mh+mozilla
Blocks: 677191
Attachment #551430 - Attachment is obsolete: true
Attachment #551430 - Flags: review?(ted.mielczarek)
Comment on attachment 551432 [details] [diff] [review]
Use a different object file extension for the first PGO build pass with GCC

Review of attachment 551432 [details] [diff] [review]:
-----------------------------------------------------------------

::: config/config.mk
@@ +784,5 @@
>  endif
> +
> +ifdef MOZ_PROFILE_GENERATE
> +ifdef GNU_CC
> +OBJ_SUFFIX := i_o

This could stand a short comment explaining why it's here.

::: config/rules.mk
@@ +916,5 @@
> +ifdef GNU_CC
> +# Force rebuilding libraries and programs in both passes because each
> +# pass uses different object files.
> +$(PROGRAM) $(SHARED_LIBRARY) $(LIBRARY): pgo.relink
> +.PHONY: pgo.relink

If this is phony, just use FORCE like we do elsewhere.
Attachment #551432 - Flags: review?(ted.mielczarek) → review+
Attachment #551432 - Attachment is obsolete: true
I don't know why I thought NO_PROFILE_GUIDED_OPTIMIZE would work in build/unix/elfhack. Fixup:
http://hg.mozilla.org/projects/build-system/rev/66b336cc59b1
Backed out. It looks like Linux x86 doesn't like the i_o extension and fails with:
 /tools/gcc-4.5/lib/gcc/i686-pc-linux-gnu/4.5.2/include/emmintrin.h:32:3: error: #error "SSE2 instruction set not enabled"

http://hg.mozilla.org/projects/build-system/rev/15efac661d6a

:(
Whiteboard: fixed-in-bs
http://hg.mozilla.org/mozilla-central/rev/dfe63797f67a
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla9
Attachment #553397 - Attachment is obsolete: true
Attachment #554313 - Flags: review?(ted.mielczarek) → review+
Attachment #554314 - Flags: review?(ted.mielczarek) → review+
Backed out part 2.
http://hg.mozilla.org/projects/build-system/rev/ac2115d5bf39
Whiteboard: fixed-in-bs
Depends on: 682794
Relanded unchanged (red was due to bug 682794).
http://hg.mozilla.org/projects/build-system/rev/66fb17949bd6
Whiteboard: fixed-in-bs
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.