Closed Bug 1455206 Opened 6 years ago Closed 6 years ago

Intermittent layout/generic/crashtests/354458-2.html | application terminated with exit code 4 with "fatal runtime error: allocator memory exhausted"

Categories

(Core :: Graphics: WebRender, defect, P5)

Product:
Core
Component:
Graphics: WebRender
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure) 

Filed by: btara [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=174356096&repo=mozilla-central

https://queue.taskcluster.net/v1/task/QTv5Il0VQWKdNgQNc59_9w/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/QTv5Il0VQWKdNgQNc59_9w/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2018-04-18T17:54:45.039Z] 17:54:45     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/generic/crashtests/354458-2.html
[task 2018-04-18T17:54:45.041Z] 17:54:45     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/generic/crashtests/354458-2.html | 1945 / 3538 (54%)
[task 2018-04-18T17:54:45.123Z] 17:54:45     INFO - ++DOMWINDOW == 150 (0x7fdd054c4800) [pid = 1044] [serial = 6224] [outer = 0x7fdd3e510400]
[task 2018-04-18T17:54:46.008Z] 17:54:46     INFO - fatal runtime error: allocator memory exhausted
[task 2018-04-18T17:54:46.211Z] 17:54:46     INFO - Hit MOZ_CRASH(Aborting on channel error.) at /builds/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:2553
[task 2018-04-18T17:54:46.213Z] 17:54:46     INFO - [Child 1044, Chrome_ChildThread] WARNING: pipe error (19): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
[task 2018-04-18T17:54:46.223Z] 17:54:46     INFO - [Child 1044, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
[task 2018-04-18T17:54:46.227Z] 17:54:46     INFO - Hit MOZ_CRASH(Aborting on channel error.) at /builds/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:2553
[task 2018-04-18T17:54:46.228Z] 17:54:46     INFO - #01: mozilla::ipc::ProcessLink::OnChannelError [ipc/glue/MessageLink.cpp:389]
[task 2018-04-18T17:54:46.237Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.238Z] 17:54:46     INFO - #02: event_process_active_single_queue [ipc/chromium/src/third_party/libevent/event.c:1648]
[task 2018-04-18T17:54:46.238Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.239Z] 17:54:46     INFO - #03: event_base_loop [ipc/chromium/src/third_party/libevent/event.c:1745]
[task 2018-04-18T17:54:46.239Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.239Z] 17:54:46     INFO - #04: base::MessagePumpLibevent::Run [ipc/chromium/src/base/message_pump_libevent.cc:381]
[task 2018-04-18T17:54:46.240Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.240Z] 17:54:46     INFO - #05: MessageLoop::RunInternal [ipc/chromium/src/base/message_loop.cc:327]
[task 2018-04-18T17:54:46.241Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.242Z] 17:54:46     INFO - #06: MessageLoop::Run [ipc/chromium/src/base/message_loop.cc:298]
[task 2018-04-18T17:54:46.242Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.243Z] 17:54:46     INFO - #07: base::Thread::ThreadMain [ipc/chromium/src/base/thread.cc:184]
[task 2018-04-18T17:54:46.244Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.245Z] 17:54:46     INFO - #08: ThreadFunc [ipc/chromium/src/base/platform_thread_posix.cc:40]
[task 2018-04-18T17:54:46.245Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.246Z] 17:54:46     INFO - #09: libpthread.so.0 + 0x76ba
[task 2018-04-18T17:54:46.247Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.247Z] 17:54:46     INFO - #01: mozilla::ipc::ProcessLink::OnChannelError [ipc/glue/MessageLink.cpp:389]
[task 2018-04-18T17:54:46.248Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.249Z] 17:54:46     INFO - #02: event_process_active_single_queue [ipc/chromium/src/third_party/libevent/event.c:1648]
[task 2018-04-18T17:54:46.251Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.258Z] 17:54:46     INFO - #03: event_base_loop [ipc/chromium/src/third_party/libevent/event.c:1745]
[task 2018-04-18T17:54:46.259Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.259Z] 17:54:46     INFO - #04: base::MessagePumpLibevent::Run [ipc/chromium/src/base/message_pump_libevent.cc:381]
[task 2018-04-18T17:54:46.260Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.261Z] 17:54:46     INFO - #05: MessageLoop::RunInternal [ipc/chromium/src/base/message_loop.cc:327]
[task 2018-04-18T17:54:46.261Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.262Z] 17:54:46     INFO - #06: MessageLoop::Run [ipc/chromium/src/base/message_loop.cc:298]
[task 2018-04-18T17:54:46.262Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.263Z] 17:54:46     INFO - #07: base::Thread::ThreadMain [ipc/chromium/src/base/thread.cc:184]
[task 2018-04-18T17:54:46.263Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.264Z] 17:54:46     INFO - #08: ThreadFunc [ipc/chromium/src/base/platform_thread_posix.cc:40]
[task 2018-04-18T17:54:46.265Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.265Z] 17:54:46     INFO - #09: libpthread.so.0 + 0x76ba
[task 2018-04-18T17:54:46.266Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.266Z] 17:54:46     INFO - #10: libc.so.6 + 0x10741d
[task 2018-04-18T17:54:46.267Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.268Z] 17:54:46     INFO - #11: ??? (???:???)
[task 2018-04-18T17:54:46.268Z] 17:54:46     INFO - #10: libc.so.6 + 0x10741d
[task 2018-04-18T17:54:46.269Z] 17:54:46     INFO - 
[task 2018-04-18T17:54:46.269Z] 17:54:46     INFO - #11: ??? (???:???)
[task 2018-04-18T17:54:46.339Z] 17:54:46    ERROR - TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/generic/crashtests/354458-2.html | application terminated with exit code 4
[task 2018-04-18T17:54:46.361Z] 17:54:46     INFO - REFTEST INFO | Process mode: e10s
[task 2018-04-18T17:54:46.361Z] 17:54:46  WARNING - leakcheck | refcount logging is off, so leaks can't be detected!
[task 2018-04-18T17:54:46.990Z] 17:54:46    ERROR - Return code: 4
[task 2018-04-18T17:54:46.992Z] 17:54:46    ERROR - No suite end message was emitted by this harness.
[task 2018-04-18T17:54:46.992Z] 17:54:46     INFO - TinderboxPrint: reftest-crashtest<br/>5815/0/16
[task 2018-04-18T17:54:46.993Z] 17:54:46    ERROR - # TBPL FAILURE #
[task 2018-04-18T17:54:46.993Z] 17:54:46  WARNING - setting return code to 2
[task 2018-04-18T17:54:46.994Z] 17:54:46    ERROR - The reftest suite: crashtest ran with return status: FAILURE
[task 2018-04-18T17:54:46.995Z] 17:54:46     INFO - Running post-action listener: _package_coverage_data
[task 2018-04-18T17:54:46.996Z] 17:54:46     INFO - Running post-action listener: _resource_record_post_action
[task 2018-04-18T17:54:46.996Z] 17:54:46     INFO - [mozharness: 2018-04-18 17:54:46.996571Z] Finished run-tests step (success)
[task 2018-04-18T17:54:46.997Z] 17:54:46     INFO - Running post-run listener: _resource_record_post_run
[task 2018-04-18T17:54:47.101Z] 17:54:47     INFO - Validating Perfherder data against /builds/worker/workspace/mozharness/external_tools/performance-artifact-schema.json
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"subtests": [{"name": "cpu_percent", "value": 71.14179487179493}, {"name": "io_write_bytes", "value": 738537472}, {"name": "io.read_bytes", "value": 34226176}, {"name": "io_write_time", "value": 99568}, {"name": "io_read_time", "value": 844}], "extraOptions": ["e10s", "taskcluster-m3.large"], "name": "reftest.crashtest.overall"}, {"subtests": [{"name": "time", "value": 19.16321611404419}, {"name": "cpu_percent", "value": 50.373684210526314}], "name": "reftest.crashtest.install"}, {"subtests": [{"name": "time", "value": 0.0009810924530029297}], "name": "reftest.crashtest.stage-files"}, {"subtests": [{"name": "time", "value": 763.2340288162231}, {"name": "cpu_percent", "value": 71.6726973684211}], "name": "reftest.crashtest.run-tests"}]}
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - Total resource usage - Wall time: 782s; CPU: 72.0%; Read bytes: 34226176; Write bytes: 738537472; Read time: 844; Write time: 99568
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - TinderboxPrint: CPU usage<br/>71.7%
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - TinderboxPrint: I/O read bytes / time<br/>34,226,176 / 844
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - TinderboxPrint: I/O write bytes / time<br/>738,537,472 / 99,568
[task 2018-04-18T17:54:47.109Z] 17:54:47     INFO - TinderboxPrint: CPU idle<br/>430.7 (28.5%)
[task 2018-04-18T17:54:47.110Z] 17:54:47     INFO - TinderboxPrint: CPU system<br/>148.2 (9.8%)
[task 2018-04-18T17:54:47.110Z] 17:54:47     INFO - TinderboxPrint: CPU user<br/>923.7 (61.0%)
[task 2018-04-18T17:54:47.110Z] 17:54:47     INFO - TinderboxPrint: Swap in / out<br/>0 / 0
[task 2018-04-18T17:54:47.112Z] 17:54:47     INFO - install - Wall time: 19s; CPU: 50.0%; Read bytes: 0; Write bytes: 59248640; Read time: 0; Write time: 32880
[task 2018-04-18T17:54:47.113Z] 17:54:47     INFO - stage-files - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2018-04-18T17:54:47.119Z] 17:54:47     INFO - run-tests - Wall time: 763s; CPU: 72.0%; Read bytes: 33947648; Write bytes: 674988032; Read time: 836; Write time: 66252
[task 2018-04-18T17:54:47.302Z] 17:54:47     INFO - Running post-run listener: _upload_blobber_files
[task 2018-04-18T17:54:47.303Z] 17:54:47  WARNING - Blob upload gear skipped. Missing cmdline options.
[task 2018-04-18T17:54:47.303Z] 17:54:47     INFO - Running post-run listener: copy_logs_to_upload_dir
[task 2018-04-18T17:54:47.304Z] 17:54:47     INFO - Copying logs to upload dir...
[task 2018-04-18T17:54:47.304Z] 17:54:47     INFO - mkdir: /builds/worker/workspace/build/upload/logs
[task 2018-04-18T17:54:47.354Z] 17:54:47     INFO - Copying logs to upload dir...
[task 2018-04-18T17:54:47.375Z] 17:54:47  WARNING - returning nonzero exit status 2
[task 2018-04-18T17:54:47.401Z] cleanup
The key part here is "fatal runtime error: allocator memory exhausted".

This is our WebRender platform, hence, version of bug 1442293
Blocks: 1442293
Component: Layout → Graphics: WebRender
Summary: Intermittent layout/generic/crashtests/354458-2.html | application terminated with exit code 4 → Intermittent layout/generic/crashtests/354458-2.html | application terminated with exit code 4 with "fatal runtime error: allocator memory exhausted"
(Perhaps this should be duped to bug 1436384 and we should add this test's name to that bug's summary? That seems to be what we've done on other similar bugs, but also the bug summary field can only hold a limited number of test-names... *shrug*  Anyway, I'm hoping that tying it to bug 1442293 is a good start in tracking this appropriately, for the time being.)
Haven't gotten any of these in over 2 months, closing.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.