1443712 - Enable /d2:-cgsummary

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

I was enlightened about the existence of the link.exe /d2:-cgsummary flag in bug 1443590. That will print information about what the PGO build is spending time on, which functions are optimized, etc. Example output is in an attachment in bug 1443590.

The output is a few pages of text and seems useful to have in CI logs. Let's enable it by default.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Bug 1443712 - Add /d2:-cgsummary to PGO link flags;

This will tell us which functions PGO is spending its time in,
which functions are being optimized, and other interesting
statistics. The output is a handful of pages of text in our
current PGO builds and it seems useful for understanding the PGO
black box.

Review commit: https://reviewboard.mozilla.org/r/225680/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/225680/

Comment 2

[(Away)]

Those av1/cdef/aom/vp9 compile times look pretty crazy. I wonder if we should disable PGO in those files since they won't be hit in our current training data anyway. Or maybe those functions are just horribly complex and would take that long to compile even without PGO.

Comment 3

[(Away)]

https://blogs.msdn.microsoft.com/vcblog/2018/01/04/visual-studio-2017-throughput-improvements-and-advice/ seems to suggest that this works outside of PGO builds too. Should we use this flag more generally?

Comment 4

[(not currently active) Ted Mielczarek]

Comment on attachment 8956700 [details]
Bug 1443712 - Add /d2:-cgsummary to PGO link flags;

https://reviewboard.mozilla.org/r/225680/#review231582

Neat!

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d7d8043d20c
Add /d2:-cgsummary to PGO link flags; r=ted

Comment 6

[Gregory Szorc [:gps]]

(In reply to David Major [:dmajor] from comment #3)
> https://blogs.msdn.microsoft.com/vcblog/2018/01/04/visual-studio-2017-
> throughput-improvements-and-advice/ seems to suggest that this works outside
> of PGO builds too. Should we use this flag more generally?

I held off because outside of PGO, where lots of codegen occurs during link, we would have to enable on cl invocations. That would make output very verbose. Or were you thinking about enabling this on all link invocations? A non-PGO link won't perform much codegen, right?

Comment 7

[(Away)]

(In reply to Gregory Szorc [:gps] from comment #6)
> (In reply to David Major [:dmajor] from comment #3)
> > https://blogs.msdn.microsoft.com/vcblog/2018/01/04/visual-studio-2017-
> > throughput-improvements-and-advice/ seems to suggest that this works outside
> > of PGO builds too. Should we use this flag more generally?
> 
> I held off because outside of PGO, where lots of codegen occurs during link,
> we would have to enable on cl invocations. That would make output very
> verbose. Or were you thinking about enabling this on all link invocations? A
> non-PGO link won't perform much codegen, right?

Oh, you're right. For some reason I thought we used LTCG on non-PGO opt builds. (We don't.)

Comment 8

[Gregory Szorc [:gps]]

In case anyone looks at the logs, sees we're using 6.7 cores and is wondering why we're capping /CGTHREADS to 8, it is because 8 is apparently the maximum value for that argument.

Comment 9

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/7d7d8043d20c