Closed Bug 1582297 Opened 5 years ago Closed 5 years ago

272,000 instances of "pipe error: 109" emitted from ipc/chromium/src/chrome/common/ipc_channel_win.cc during windows10-64 debug testing

Categories

(Core :: IPC, defect, P1)

Product:
Core
Component:
IPC
Platform:
All
Windows
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla72
Tracking Flags:
Tracking Status
firefox72 --- fixed

People

(Reporter: erahm, Assigned: jld)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Bug 1582297 - Suppress IPC "pipe error" messages if the cause was probably the other process exiting.
47 bytes, text/x-phabricator-request
Details | Review

272025 WARNING: pipe error: 109: file ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341

This warning [1] shows up in the following test suites:

 16714 - test-windows10-64/debug-web-platform-tests-e10s-12 wpt12
 14773 - test-windows10-64/debug-web-platform-tests-e10s-17 wpt17
 13785 - test-windows10-64/debug-web-platform-tests-e10s-14 wpt14
 13648 - test-windows10-64/debug-web-platform-tests-e10s-18 wpt18
 13436 - test-windows10-64/debug-web-platform-tests-e10s-15 wpt15
 13330 - test-windows10-64/debug-web-platform-tests-e10s-16 wpt16
 13251 - test-windows10-64/debug-web-platform-tests-e10s-5 wpt5
 13138 - test-windows10-64/debug-web-platform-tests-e10s-8 wpt8
 12151 - test-windows10-64/debug-web-platform-tests-e10s-7 wpt7
 11313 - test-windows10-64/debug-web-platform-tests-e10s-11 wpt11
 11152 - test-windows10-64/debug-web-platform-tests-e10s-10 wpt10
 10518 - test-windows10-64/debug-web-platform-tests-e10s-9 wpt9
 10298 - test-windows10-64/debug-web-platform-tests-e10s-3 wpt3
  9941 - test-windows10-64/debug-web-platform-tests-e10s-6 wpt6
  8808 - test-windows10-64/debug-web-platform-tests-e10s-4 wpt4
  8512 - test-windows10-64/debug-web-platform-tests-e10s-2 wpt2
  8309 - test-windows10-64/debug-test-verify-e10s-1 TV1
  7113 - test-windows10-64/debug-web-platform-tests-e10s-1 wpt1
  6438 - test-windows10-64/debug-web-platform-tests-wdspec-e10s-2 Wd2
  3894 - test-windows10-64/debug-web-platform-tests-wdspec-e10s-1 Wd1
  2923 - test-windows10-64/debug-web-platform-tests-reftests-e10s-4 Wr4
  2682 - test-windows10-64/debug-web-platform-tests-reftests-e10s-1 Wr1
  2546 - test-windows10-64/debug-web-platform-tests-reftests-e10s-3 Wr3
  2506 - test-windows10-64/debug-reftest-e10s-3 R3
  2491 - test-windows10-64/debug-test-verify-wpt-e10s-1 TVw1
  2466 - test-windows10-64/debug-test-verify-wpt-e10s-2 TVw2
  2449 - test-windows10-64/debug-reftest-e10s-2 R2
  2448 - test-windows10-64/debug-mochitest-browser-chrome-e10s-5 bc5
  2177 - test-windows10-64/debug-mochitest-browser-chrome-e10s-6 bc6
  2172 - test-windows10-64/debug-web-platform-tests-reftests-e10s-2 Wr2
  2164 - test-windows10-64/debug-web-platform-tests-reftests-e10s-5 Wr5
  2101 - test-windows10-64/debug-mochitest-browser-chrome-e10s-4 bc4
  2015 - test-windows10-64/debug-mochitest-browser-chrome-e10s-3 bc3
  1986 - test-windows10-64/debug-test-verify-wpt-e10s-3 TVw3
  1974 - test-windows10-64/debug-marionette-e10s Mn
  1921 - test-windows10-64/debug-mochitest-browser-chrome-e10s-1 bc1
  1538 - test-windows10-64/debug-mochitest-e10s-5 5
  1363 - test-windows10-64/debug-mochitest-e10s-3 3
  1235 - test-windows10-64/debug-mochitest-e10s-4 4
  1201 - test-windows10-64/debug-test-verify-gpu-e10s TVg
  1168 - test-windows10-64/debug-mochitest-e10s-2 2
  1062 - test-windows10-64/debug-mochitest-e10s-1 1
  1023 - test-windows10-64/debug-firefox-ui-functional-local-e10s en-US
   735 - test-windows10-64/debug-mochitest-devtools-chrome-e10s-5 dt5
   657 - test-windows10-64/debug-mochitest-devtools-chrome-e10s-1 dt1
   571 - test-windows10-64/debug-mochitest-devtools-chrome-e10s-2 dt2
   557 - test-windows10-64/debug-mochitest-devtools-chrome-e10s-4 dt4
   538 - test-windows10-64/debug-mochitest-media-e10s mda
   498 - test-windows10-64/debug-mochitest-media-spi-e10s mda
   463 - test-windows10-64/debug-mochitest-devtools-chrome-e10s-3 dt3
   313 - test-windows10-64/debug-telemetry-tests-client-e10s c
   275 - test-windows10-64/debug-mochitest-chrome-1proc-1 c1
   156 - test-windows10-64/debug-jsreftest-e10s-1 J1
   149 - test-windows10-64/debug-jsreftest-e10s-2 J2
   148 - test-windows10-64/debug-mochitest-webgl1-core-e10s gl1c
   138 - test-windows10-64/debug-mochitest-chrome-1proc-3 c3
   119 - test-windows10-64/debug-crashtest-e10s C
   115 - test-windows10-64/debug-marionette-gpu-e10s MnG
    92 - test-windows10-64/debug-mochitest-gpu-e10s gpu
    74 - test-windows10-64/debug-mochitest-remote-e10s remote
    47 - test-windows10-64/debug-firefox-ui-functional-remote-e10s en-US
    44 - test-windows10-64/debug-jsreftest-e10s-3 J3
    37 - test-windows10-64/debug-mochitest-webgl2-core-e10s gl2c
    28 - test-windows10-64/debug-mochitest-webgl2-ext-e10s-1 gl2e1
    25 - test-windows10-64/debug-mochitest-webgl2-ext-e10s-3 gl2e3
    23 - test-windows10-64/debug-mochitest-webgl2-ext-e10s-2 gl2e2
    23 - test-windows10-64/debug-mochitest-webgl1-ext-e10s gl1e
    20 - test-windows10-64/debug-mochitest-chrome-1proc-2 c2
    19 - test-windows10-64/debug-mochitest-webgl2-ext-e10s-4 gl2e4
    19 - test-windows10-64/debug-gtest-1proc GTest
     6 - test-windows10-64/debug-mochitest-a11y-1proc a11y
     2 - test-windows10-64/debug-xpcshell-e10s-2 X2
     1 - test-windows10-64/debug-marionette-media-e10s MnM

It shows up in 17949 tests. A few of the most prevalent:

 27199 - [e10s] Shutdown
  1220 - [e10s] file:///Z:/task_1568804856/build/tests/reftest/tests/layout/reftests/bugs/1241631-1.html == file:///Z:/task_1568804856/build/tests/reftest/tests/layout/reftests/bugs/1241631-1-ref.html
   787 - [e10s] /webdriver/tests/new_session/create_alwaysMatch.py
   780 - [e10s] /webdriver/tests/new_session/create_firstMatch.py
   513 - [e10s] /css/motion/offset-path-ray-004.html
   512 - [e10s] /css/motion/offset-path-ray-contain-003.html
   507 - [e10s] /css/motion/offset-path-ray-007.html
   504 - [e10s] /css/motion/offset-path-ray-001.html
   504 - [e10s] /css/motion/offset-path-ray-contain-004.html
   500 - [e10s] /css/motion/offset-path-ray-009.html

[1] https://hg.mozilla.org/mozilla-central/annotate/e71921b729c6/ipc/chromium/src/chrome/common/ipc_channel_win.cc#l341

OS: All → Windows

I'm going to take this, because it should be easy, and the amount of time and aggravation it will save in terms of misfiled bugs (because the IPC message is mistaken as the cause, or especially when it's interleaved with the real error and the result is misparsed by tools) should pay back the cost of doing it in maybe a week or two.

Context, which is probably already in some other bug somewhere: this is a vestige of the old idea that we'd (and I say “we” but this is before my time) have a layer of our own glue on top of the Chromium IPC classes; the signaling that the channel is being intentionally closed is in the mozilla::ipc layer, but before we get there the Chromium code has already yelled about it on stderr. This can happen on Unix, but on Windows it's worse because we get a “broken pipe” error when reading, not only when writing.

Assignee: nobody → jld
Priority: -- → P1

There are two issues here:

  1. These error messages occur even during normal channel shutdown,
    because that's tracked in the mozilla::ipc::MessageChannel layer,
    which the ipc/chromium code can't access.

  2. If we get this kind of error when the channel wasn't intentionally
    closed, it almost certainly means that the other process crashed. In
    that case, having error messages from a different process and a likely
    unrelated subsystem just leads to confusion and misfiled bugs.

(Also complicating things: on Unix a closed channel often, but not
always, results in an end-of-file indication, which already isn't
logged; on Windows it's always a broken pipe error, which causes a much
larger amount of log spam.)

Bonus fix: the error that contains a fd number is clarified to avoid
having it mistaken for an error code.

See Also: → 1428509
Pushed by jedavis@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/512d489d83e1 Suppress IPC "pipe error" messages if the cause was probably the other process exiting. r=froydnj

https://hg.mozilla.org/mozilla-central/rev/512d489d83e1

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox72: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: