PGO has very few profile data on incremental builds

RESOLVED FIXED in mozilla9

Status

()

Core
Build Config
RESOLVED FIXED
6 years ago
6 years ago

People

(Reporter: glandium, Assigned: glandium)

Tracking

Trunk
mozilla9
All
Linux
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: fixed-in-bs)

Attachments

(2 attachments, 3 obsolete attachments)

(Assignee)

Description

6 years ago
We don't rebuild the objects that are already in the objdir, and they are final objects, not ones with instrumentation... (doh)

So we end up running the profile without instrumentation in most object files.

The easy solution would be to clobber before the profile generate pass, but that would be a huge waste of time. I think an easy way to avoid this problem, while keeping fast(ish) builds is to change the object file extension in one of the passes so that we effectively never overwrite object files from the profile generate pass in the profile use pass. That would double disk usage for object files, though.

On the other hand, we could probably skip debugging symbols in the profile generate pass, which would make the object files smaller. (though they might be useful in the case running firefox for the profile crashes)
> The easy solution would be to clobber before the profile generate pass, but 
> that would be a huge waste of time.

Catlee has been working on patches to ccache to support -fprofile-generate and -fprofile-use.  If used that, clobbering wouldn't be so bad.

But of course what you're proposing sounds more clever...
Is this specific to Linux, or would this impact Windows as well?
(Assignee)

Comment 3

6 years ago
This is specific to linux.
(Assignee)

Updated

6 years ago
Assignee: nobody → mh+mozilla
(Assignee)

Updated

6 years ago
Blocks: 677191
(Assignee)

Comment 4

6 years ago
Created attachment 551430 [details] [diff] [review]
Use a different object file extension for the first PGO build pass with GCC
Attachment #551430 - Flags: review?(ted.mielczarek)
(Assignee)

Comment 5

6 years ago
Created attachment 551432 [details] [diff] [review]
Use a different object file extension for the first PGO build pass with GCC
Attachment #551432 - Flags: review?(ted.mielczarek)
(Assignee)

Updated

6 years ago
Attachment #551430 - Attachment is obsolete: true
Attachment #551430 - Flags: review?(ted.mielczarek)
Comment on attachment 551432 [details] [diff] [review]
Use a different object file extension for the first PGO build pass with GCC

Review of attachment 551432 [details] [diff] [review]:
-----------------------------------------------------------------

::: config/config.mk
@@ +784,5 @@
>  endif
> +
> +ifdef MOZ_PROFILE_GENERATE
> +ifdef GNU_CC
> +OBJ_SUFFIX := i_o

This could stand a short comment explaining why it's here.

::: config/rules.mk
@@ +916,5 @@
> +ifdef GNU_CC
> +# Force rebuilding libraries and programs in both passes because each
> +# pass uses different object files.
> +$(PROGRAM) $(SHARED_LIBRARY) $(LIBRARY): pgo.relink
> +.PHONY: pgo.relink

If this is phony, just use FORCE like we do elsewhere.
Attachment #551432 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 7

6 years ago
Created attachment 553397 [details] [diff] [review]
Use a different object file extension for the first PGO build pass with GCC

As landed
(Assignee)

Updated

6 years ago
Attachment #551432 - Attachment is obsolete: true
(Assignee)

Comment 8

6 years ago
http://hg.mozilla.org/projects/build-system/rev/dfe63797f67a
Whiteboard: fixed-in-bs
(Assignee)

Comment 9

6 years ago
I don't know why I thought NO_PROFILE_GUIDED_OPTIMIZE would work in build/unix/elfhack. Fixup:
http://hg.mozilla.org/projects/build-system/rev/66b336cc59b1
(Assignee)

Comment 10

6 years ago
Backed out. It looks like Linux x86 doesn't like the i_o extension and fails with:
 /tools/gcc-4.5/lib/gcc/i686-pc-linux-gnu/4.5.2/include/emmintrin.h:32:3: error: #error "SSE2 instruction set not enabled"

http://hg.mozilla.org/projects/build-system/rev/15efac661d6a

:(
(Assignee)

Updated

6 years ago
Whiteboard: fixed-in-bs
http://hg.mozilla.org/mozilla-central/rev/dfe63797f67a
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla9
Ah, this bounced http://hg.mozilla.org/mozilla-central/rev/15efac661d6a
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 13

6 years ago
Created attachment 554313 [details] [diff] [review]
part 1 - Don't use $(OBJ_SUFFIX) before including config.mk
Attachment #554313 - Flags: review?(ted.mielczarek)
(Assignee)

Comment 14

6 years ago
Created attachment 554314 [details] [diff] [review]
part 2 - Use a different object file extension for the first PGO build pass with GCC
Attachment #554314 - Flags: review?(ted.mielczarek)
(Assignee)

Updated

6 years ago
Attachment #553397 - Attachment is obsolete: true
Attachment #554313 - Flags: review?(ted.mielczarek) → review+
Attachment #554314 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 15

6 years ago
http://hg.mozilla.org/projects/build-system/rev/6bedd46abb82
http://hg.mozilla.org/projects/build-system/rev/480f913edf3f
Whiteboard: fixed-in-bs
(Assignee)

Comment 16

6 years ago
Backed out part 2.
http://hg.mozilla.org/projects/build-system/rev/ac2115d5bf39
Whiteboard: fixed-in-bs
(Assignee)

Updated

6 years ago
Depends on: 682794
(Assignee)

Comment 17

6 years ago
Relanded unchanged (red was due to bug 682794).
http://hg.mozilla.org/projects/build-system/rev/66fb17949bd6
Whiteboard: fixed-in-bs
http://hg.mozilla.org/mozilla-central/rev/6bedd46abb82
http://hg.mozilla.org/mozilla-central/rev/480f913edf3f
Backout of second cset:
http://hg.mozilla.org/mozilla-central/rev/ac2115d5bf39
Relanding:
http://hg.mozilla.org/mozilla-central/rev/66fb17949bd6
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.