Closed Bug 1326329 Opened 7 years ago Closed 4 years ago

Each cl invocation spawns at least 5 processes

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: erahm, Unassigned)

References

(Blocks 1 open bug)

Details

Per bug 1095293, comment 6 it looks we spawn 3 sh.exe processes, 1 python.exe process, and one cl.exe process each time we invoke |cl|. Creating new processes on Windows is known to be slow, it seem like we should just have mozmake call cl directly.
Copy/pasting bug 1095293 comment 9, since it's relevant here:
  sh can probably be skipped with some effort rewriting the commands used to compile such that make calls CreateProcess on its own instead of going through the shell. Python can't be because that's what gets us the dependency files for make to figure out when to rebuild something (although sccache can do this too, and the rust rewrite should have less overhead than python).
I couldn't find any evidence that we'd ever tried this, so I submitted a request to the Visual Studio suggestion box asking for support for generating dependency files like GCC/Clang can:
https://visualstudio.uservoice.com/forums/121579-visual-studio-ide/suggestions/18649837-add-support-for-generating-a-dependency-file-like

Can't hurt, right?
FWIW someone measured the overhead of CreateProcess on Windows here <https://bugs.llvm.org/show_bug.cgi?id=20253#c4> to be around 60ms!  If that or anything around it is to be trusted, doing that 5 times is 300 milli-seconds per invocations...
I don't see any obvious reason we ought to be spawning extraneous shells during compilation. On my machine, a compile invocation looks like:
 c:/build/mozilla-central/debug-mozilla-central/_virtualenv/Scripts/python.exe -m mozbuild.action.cl  cl.exe -Fomozjemalloc_compat.obj -c  -DDEBUG=1 -DTRACING=1 -DMOZ_JEMALLOC4 -DMOZ_REPLACE_JEMALLOC -Ic:/build/mozilla-central/memory/replace/jemalloc -Ic:/build/mozilla-central/debug-mozilla-central/memory/replace/jemalloc -Ic:/build/mozilla-central/debug-mozilla-central/memory/jemalloc/src/include -Ic:/build/mozilla-central/memory/jemalloc/src/include/msvc_compat -Ic:/build/mozilla-central/debug-mozilla-central/dist/include  -Ic:/build/mozilla-central/debug-mozilla-central/dist/include/nspr -Ic:/build/mozilla-central/debug-mozilla-central/dist/include/nss        -MD -FI c:/build/mozilla-central/debug-mozilla-central/mozilla-config.h -DMOZILLA_CLIENT   -TC -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -utf-8 -arch:SSE2 -FS -Gw -wd4244 -wd4267 -we4553  -Zi -Oy-   -Fdgenerated.pdb  c:/build/mozilla-central/memory/build/mozjemalloc_compat.c

I can't see that there are any shell special characters there, so mozmake ought to be able to call CreateProcess with that.
Product: Core → Firefox Build System

I think this is resolved; at the very least, we no longer have cl.py, since clang-cl knows how to write make-style dependencies on its own, and comment 4 indicates that we ought to be calling CreateProcess in any event.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.