573306 - stdout and stderr get tangled on Windows (when using automation.py?)

Status: NEW

(Testing :: General, defect)

Description

[Jesse Ruderman]

Steps to reproduce:
1. Save r.xhtml and crashtests.list to your desktop.
2. Load the crashtests.list using runreftest.py.

Result: "XXB" and sometimes "XXA" (going to stdout) get tangled with stack traces (going to stderr).  There will be part of a line of a stack trace, followed by "XXB\n", followed by the rest of the line of the stack trace.

Expected: "XXA" and "XXB" should always appear by themselves on a separate line.

Comment 1

[Jesse Ruderman]

Attachment: r.xhtml

Comment 2

[Jesse Ruderman]

Attachment: crashtests.list

Comment 3

[Jesse Ruderman]

Attachment: example output

Comment 4

[Jesse Ruderman]

For fuzzing it's important for me to be able to process the output from Firefox. But I think I can work around this issue by not printing assertion stacks while fuzzing.

Comment 5

[Benjamin Smedberg]

This is how Windows works: the standard streams are character-buffered. Unless you mean that automation should save stdout/stderr to files separately and then dump them, which seems a bit strange.

Comment 6

[Jesse Ruderman]

What can programs do to make their output deterministic? Not use stderr at all?

Comment 7

[Jesse Ruderman]

(In reply to comment #5)
> This is how Windows works: the standard streams are character-buffered. Unless
> you mean that automation should save stdout/stderr to files separately and then
> dump them, which seems a bit strange.

Allowing them to get tangled is strange too!

Comment 8

[Jesse Ruderman]

Would some carefully placed fflush() calls help?

Comment 9

[Jesse Ruderman]

How should I work around this for fuzzing, if turning off assertion stacks isn't enough? Add fflush calls to Firefox? Add a mode to automation.py where stdout and stderr go to different pipes/files?

Comment 10

[(not currently active) Ted Mielczarek]

We could probably fix this in automation.py by reading stdout and stderr separately, and line-buffering them. That would complicate things a bit, but not terribly, I think. It looks like we probably hit this in bug 740242, where the DOMWINDOW spew and assertion output get intermingled, which is breaking our log parsing.

Also: we're planning on moving all our test harnesses away from automation.py and onto the mozbase modules, so maybe we should just fix this there.