799189 - Use mozprocess in cl.py

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

Attachment: Use mozprocess in cl.py, v1

See bug 794490 comment #10.

I'm holding off review until Try comes back.

https://tbpl.mozilla.org/?tree=Try&rev=fae525b15083

Comment 1

[Gregory Szorc [:gps]]

New try is https://tbpl.mozilla.org/?tree=Try&rev=48d84e31bd15 and I can make no sense of the build log.

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

Can we move forward on this?  Just got another confused dev hitting it (or some variant)  in bug 814619

Comment 3

[Gregory Szorc [:gps]]

I forgot why this failed. https://tbpl.mozilla.org/?tree=Try&rev=493da76f6dce

Comment 4

[Gregory Szorc [:gps]]

Attachment: Use mozprocess in cl.py

I'd like to get cl.py invoked as a native pymake command. This might be
a blocker since the last time we landed native cl.py we ran into issues
with output buffering. mozprocess should make things more flexible so we
can address that.

https://tbpl.mozilla.org/?tree=Try&rev=c59d58433b59

Comment 5

[Gregory Szorc [:gps]]

Attachment: Use mozprocess in cl.py

The API of mozprocess changed slightly. This patch appears to be
building fine on my Windows machine.

https://tbpl.mozilla.org/?tree=Try&rev=c25f7cdb647e

Comment 6

[Gregory Szorc [:gps]]

Attachment: Use mozprocess in cl.py

mozprocess swallows newlines. This patch re-adds them and makes the
output look sane again.

https://tbpl.mozilla.org/?tree=Try&rev=c958cacd0d5f

Comment 7

[Michael Shal [:mshal]]

The patch looks fine to me, though there seems to be a small speed penalty when using it. I am trying:

$ touch xpcom/base/*.cpp
$ time ./mach build xpcom/base
(which builds with -j4)

With patch: 15.035s
Without patch: 13.395s

I've run it several times and the numbers are pretty consistent. Do you get similar results? Is this an acceptable trade-off for this bug?

Comment 8

[Gregory Szorc [:gps]]

That slowdown is interesting! I wonder if mozprocess is just that much slower, if this is lack of bytecode caching, etc.

This bug is on the road to making cl.py a native pymake command. I would think that step would make things faster overall. So, I'm willing to tolerate a temporary regression as long as we verify the end state if faster.

Comment 9

[Michael Shal [:mshal]]

Comment on attachment 802054 [details] [diff] [review]
Use mozprocess in cl.py

Ok, hopefully that will negate any losses here.

Comment 10

[Gregory Szorc [:gps]]

https://hg.mozilla.org/projects/build-system/rev/18d33a5c63a0

Comment 11

[Gregory Szorc [:gps]]

https://hg.mozilla.org/mozilla-central/rev/18d33a5c63a0