Closed
Bug 1233554
Opened 8 years ago
Closed 8 years ago
Linux x64 debug crashtest e10s crash in docker image (tests/reftest/tests/dom/canvas/crashtests/780392-1.html)
Categories
(Core :: Graphics, defect)
Core
Graphics
Tracking
()
RESOLVED
FIXED
| Tracking | Status | |
|---|---|---|
| e10s | + | --- |
People
(Reporter: armenzg, Unassigned)
References
(Blocks 1 open bug)
Details
(Whiteboard: gfx-noted)
Attachments
(2 files)
We're trying to run Linux 64 debug tests inside of docker containers. In the following push [1] we have the e10s crashtest crashing [2]. If you "inspect the task" [3] you will see the .dmp files generated [4] There is also a screenshot of a dialog saying "The application classic guest session has closed unexpectedly" [5] I don't know if the following line in the output is relevant: > [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459 The test in question is this one: https://dxr.mozilla.org/mozilla-central/source/dom/canvas/crashtests/780392-1.html Any thoughts on what could be causing this crash? (STR below) [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3604a422edc3 [2] https://public-artifacts.taskcluster.net/b56UUICuQrOXPU4PzKdv5Q/0/public/logs/live_backing.log [3] https://tools.taskcluster.net/task-inspector/#Hv1NIUCHRneucYyv1gxQrw/0 [4] https://queue.taskcluster.net/v1/task/Hv1NIUCHRneucYyv1gxQrw/runs/0/artifacts/public/test_info//4299947f-0525-b3b7-71e62317-53354004.dmp [5] https://public-artifacts.taskcluster.net/Hv1NIUCHRneucYyv1gxQrw/0/public/test_info//mozilla-test-fail-screenshot_YpFWcm.png STR: #### sudo modprobe v4l2loopback # If you only have /dev/video0 image_name=armenzg/desktop-test:0.5.1 docker pull 'armenzg/desktop-test:0.5.1' # Run docker command docker run -ti \ --name "${container_name}" \ --device=/dev/video1:/dev/video1 \ -e START_VNC='true' \ -e SKIP_RUNNING_SCRIPT='true' \ -e DISPLAY=:0 \ -e MOZILLA_BUILD_URL='https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.tar.bz2' \ -e MOZHARNESS_SCRIPT='mozharness/scripts/desktop_unittest.py' \ -e GECKO_HEAD_REPOSITORY='https://hg.mozilla.org/try/' \ -e MOZHARNESS_URL='https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/mozharness.zip' \ -e NEED_PULSEAUDIO='true' \ -e NEED_WINDOW_MANAGER='true' \ -e MOZHARNESS_CONFIG='mozharness/configs/unittests/linux_unittest.py mozharness/configs/remove_executables.py ' \ -e GECKO_HEAD_REV='3961f20f20a358a53b81231050395e9fc4e415f3' \ ${image_name} \ chmod +x bin/test.sh ~/bin/test.sh cd workspace sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s ##### [1] 17:23:32 INFO - REFTEST TEST-START | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html 17:23:32 INFO - REFTEST TEST-LOAD | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | 251 / 2968 (8%) 17:23:34 INFO - ###!!! [Parent][MessageChannel] Error: (msgtype=0x2A0081,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv 17:23:34 INFO - JavaScript error: resource://app/modules/ContentCrashHandlers.jsm, line 75: TypeError: browser.permanentKey is not a non-null object 17:23:34 INFO - [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459 17:29:04 WARNING - TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application timed out after 330 seconds with no output 17:29:04 INFO - TEST-INFO | screentopng: exit 0 17:29:04 WARNING - TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application terminated with exit code 6 17:29:05 INFO - mozcrash INFO | Downloading symbols from: https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.crashreporter-symbols.zip 17:29:08 INFO - mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/207fc14f-c996-1046-039c6068-78cfe751.dmp 17:29:08 INFO - mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/207fc14f-c996-1046-039c6068-78cfe751.extra 17:29:08 WARNING - PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [unknown top frame] 17:29:08 INFO - Crash dump filename: /tmp/tmpMnGGsS.mozrunner/minidumps/207fc14f-c996-1046-039c6068-78cfe751.dmp 17:29:08 INFO - MINIDUMP_STACKWALK not set, can't process dump. 17:29:08 INFO - mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/526bb3e4-24a2-cef8-5184621d-5c5290ef.dmp 17:29:08 INFO - mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/526bb3e4-24a2-cef8-5184621d-5c5290ef.extra 17:29:08 WARNING - PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [unknown top frame] 17:29:08 INFO - Crash dump filename: /tmp/tmpMnGGsS.mozrunner/minidumps/526bb3e4-24a2-cef8-5184621d-5c5290ef.dmp 17:29:08 INFO - MINIDUMP_STACKWALK not set, can't process dump. 17:29:08 INFO - WARNING | leakcheck | refcount logging is off, so leaks can't be detected! 17:29:08 INFO - REFTEST INFO | runreftest.py | Running tests: end. 17:29:08 ERROR - Return code: 6 17:29:08 ERROR - No tests run or test summary not found
|
||
Comment 1•8 years ago
|
||
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #0) > We're trying to run Linux 64 debug tests inside of docker containers. > In the following push [1] we have the e10s crashtest crashing [2]. > If you "inspect the task" [3] you will see the .dmp files generated [4] What's up with us not getting a stack printed in the log? I see: 17:29:08 INFO - MINIDUMP_STACKWALK not set, can't process dump. Shouldn't mozharness be using tooltool to fetch a minidump_stackwalk binary? > There is also a screenshot of a dialog saying "The application classic guest > session has closed unexpectedly" [5] This is "indicator-session-service" crashing, I don't know if it's related or not. It would be interesting to know if this happens on other test runs. > I don't know if the following line in the output is relevant: > > [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459 This is normal output from an e10s process when one of the processes crashes. It's the other process realizing the pipe it uses for IPC has been closed.
|
|
||
Comment 2•8 years ago
|
||
Here's a stack from that minidump.
|
||
Comment 3•8 years ago
|
||
I already gave Armen some tips to check yesterday. But I had another look at the log and found the following: > 21:40:12 INFO - 'download_minidump_stackwalk': False, This explains why it is happening. The reason for that is the following config file which you make use of: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/remove_executables.py
|
Reporter | |
Comment 4•8 years ago
|
||
I will move the lack of dumping to its own bug and leave this one for crash itself.
Component: General → Graphics
Product: Testing → Core
|
|
||
Comment 5•8 years ago
|
||
Here's annotated disassembly of the crash location, if it helps anyone.
|
|
Reporter | |
Comment 6•8 years ago
|
||
Hello poiru, we're seeing some crashes when running e10s crashtests inside of docker containers (we're moving towards them). ted has a crash dump on comment 5; would you be able to look into this bug and let us know if we need to change anything in the docker image? Thanks!
Flags: needinfo?(birunthan)
|
||
Comment 7•8 years ago
|
||
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #6) > Hello poiru, we're seeing some crashes when running e10s crashtests inside > of docker containers (we're moving towards them). > ted has a crash dump on comment 5; would you be able to look into this bug > and let us know if we need to change anything in the docker image? > > Thanks! I only moved that test from one directory to another (you can use `hg log --follow` to track across renames). The test was originally added in https://hg.mozilla.org/mozilla-central/rev/8a736f13c5f8 You might want to try the author/reviewer of that commit.
Flags: needinfo?(birunthan)
|
|
Reporter | |
Comment 8•8 years ago
|
||
Thanks poiru. My bad. Hi nrc, bas, we're seeing some crashes when running e10s crashtests inside of docker containers (we're moving towards them). ted has a crash dump on comment 5; would you be able to look into this bug and let us know if we need to change anything in the docker image? Thanks!
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
|
||
Updated•8 years ago
|
Whiteboard: gfx-noted
|
|
Reporter | |
Comment 9•8 years ago
|
||
Removing NI as by now too much has changed and we will need new data. If we still hit the issue we will reflag and hope for better 2 way communication.
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
|
||
Updated•8 years ago
|
Blocks: e10s-tests
tracking-e10s:
--- → +
|
|
Reporter | |
Comment 10•8 years ago
|
||
nrc, bas: please let us know if you could have a look at this or if someone else could. This is going to start getting a lot of attention so please help us at least redirect if necessary. Thanks for your help! We want to run Linux64 debug test jobs under docker. We're getting closer to it, however, we now have to figure test issues. In this bug we would like to look at fixing a crash we're seeing in e10s crashtest [1]. Here are the steps to reproduce locally: # First time prep step sudo apt-get install v4l2loopback-dkms # Every time after your host is rebooted sudo modprobe v4l2loopback # This will create a device under /dev/video* NOTE: The value --device is specific to your host. Please check under /dev/video* and pick the one with the highest numeral docker run -ti \ -e GECKO_HEAD_REPOSITORY='https://hg.mozilla.org/try/' \ -e GECKO_HEAD_REV='921c3df79ac83ab115bc054c349f29fc7526a534' \ -e MOZHARNESS_CONFIG='mozharness/configs/unittests/linux_unittest.py mozharness/configs/remove_executables.py ' \ -e MOZHARNESS_SCRIPT='mozharness/scripts/desktop_unittest.py' \ -e MOZHARNESS_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/mozharness.zip' \ -e MOZILLA_BUILD_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2' \ -e NEED_PULSEAUDIO='true' \ -e NEED_WINDOW_MANAGER='true' \ -e START_VNC='true' \ -e SKIP_MOZHARNESS_RUN='true' \ --device /dev/video1:/dev/video1 \ taskcluster/desktop-test:0.5.5 # Now inside of docker ./bin/test.sh # You can now connect to VNC on your host (xtightvncviewer 172.17.0.2) cd workspace sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s --no-run-tests # Modify tests/reftest/tests/testing/crashtest/crashtests.list and sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s --run-tests --binary-path application/firefox/firefox-bin Notice that I'm first running with --no-run-tests to set everything up and then with --run-tests which will only run the tests and not clobber anything. If you close your VNC connection, you will have to call ./bin/test.sh again (maybe just call `x11vnc &` alone). [1] https://public-artifacts.taskcluster.net/OYOkzqk9TmqDtRKxABjh6Q/0/public/logs/live_backing.log
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
|
|
Reporter | |
Updated•8 years ago
|
Summary: Linux x64 debug crashtest e10s crash in docker image → Linux x64 debug crashtest e10s crash in docker image (tests/reftest/tests/dom/canvas/crashtests/780392-1.html)
|
|
Reporter | |
Comment 11•8 years ago
|
||
For reference, this is a push where the crashtest was run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=921c3df79ac8
|
|
Reporter | |
Comment 12•8 years ago
|
||
Alternatively instead of running Mozharness with --run-tests you can do this (which is equivalent): cd ~/workspace/build source /home/worker/workspace/build/venv/bin/activate sudo -E -u worker /home/worker/workspace/build/venv/bin/python -u /home/worker/workspace/build/tests/reftest/runreftest.py --e10s --total-chunks 1 --this-chunk 1 --appname=/home/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.crashreporter-symbols.zip --suite=crashtest -- tests/reftest/tests/testing/crashtest/crashtests.list
|
||
Comment 13•8 years ago
|
||
from the log (https://public-artifacts.taskcluster.net/b6NGMbKLQ7ed_zIlVh4kqw/0/public/logs/live_backing.log): 15:09:12 INFO - REFTEST TEST-START | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html 15:09:12 INFO - REFTEST TEST-LOAD | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | 251 / 2976 (8%) 15:09:12 INFO - ++DOMWINDOW == 62 (0x7f098d7aec00) [pid = 1160] [serial = 713] [outer = 0x7f0991035400] 15:09:12 INFO - --DOMWINDOW == 61 (0x7f098d3a2400) [pid = 1160] [serial = 607] [outer = (nil)] [url = data:text/html,<meta%20charset=UTF-8><body><video%20src=http://localhost:8080/%20controls=true%20loop=true%20autoplay=true%20autobuffer=false></video>] 15:09:12 INFO - --DOMWINDOW == 60 (0x7f098c534400) [pid = 1160] [serial = 606] [outer = (nil)] [url = data:text/html,<meta%20charset=UTF-8><body><video%20src=http://localhost:8080/%20controls=true%20loop=true%20autoplay=true%20autobuffer=false></video>] 15:09:12 INFO - --DOMWINDOW == 59 (0x7f098c535000) [pid = 1160] [serial = 592] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/845093-1.html] 15:09:12 INFO - --DOMWINDOW == 58 (0x7f098c538800) [pid = 1160] [serial = 594] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/845093-2.html] 15:09:12 INFO - --DOMWINDOW == 57 (0x7f098bd7c400) [pid = 1160] [serial = 663] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/bindings/crashtests/832899.html] 15:09:12 INFO - --DOMWINDOW == 56 (0x7f098cd07400) [pid = 1160] [serial = 600] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/849727.html] 15:09:12 INFO - --DOMWINDOW == 55 (0x7f098c541400) [pid = 1160] [serial = 602] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/849732.html] 15:09:12 INFO - --DOMWINDOW == 54 (0x7f098d292400) [pid = 1160] [serial = 677] [outer = 0x7f098d291800] [url = about:blank] 15:09:12 INFO - --DOMWINDOW == 53 (0x7f098d184000) [pid = 1160] [serial = 619] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/898906.html] 15:09:12 INFO - --DOMWINDOW == 52 (0x7f098bd7cc00) [pid = 1160] [serial = 645] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/1157995.html] 15:09:12 INFO - --DOMWINDOW == 51 (0x7f098d28fc00) [pid = 1160] [serial = 647] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/1181619.html] 15:09:12 INFO - --DOMWINDOW == 50 (0x7f098c53bc00) [pid = 1160] [serial = 613] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/864448.html] 15:09:12 INFO - --DOMWINDOW == 49 (0x7f098da05000) [pid = 1160] [serial = 627] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/942979.html] 15:09:12 INFO - --DOMWINDOW == 48 (0x7f098bd75400) [pid = 1160] [serial = 633] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/978646.html] 15:09:13 INFO - --DOMWINDOW == 47 (0x7f098d291c00) [pid = 1160] [serial = 653] [outer = (nil)] [url = http://localhost:43729/1452870464911/1/crashtests/xhr_abortinprogress.html] 15:09:13 INFO - --DOMWINDOW == 46 (0x7f098cd0a000) [pid = 1160] [serial = 655] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/xhr_empty_datauri.html] 15:09:13 INFO - --DOMWINDOW == 45 (0x7f098cd61800) [pid = 1160] [serial = 657] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/xhr_html_nullresponse.html] 15:09:18 INFO - ###!!! [Parent][MessageChannel] Error: (msgtype=0x2A0084,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv 15:09:18 INFO - JavaScript error: resource://app/modules/ContentCrashHandlers.jsm, line 75: TypeError: browser.permanentKey is not a non-null object 15:09:18 INFO - [Parent 1064] WARNING: pipe error (58): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459 15:14:48 WARNING - TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application timed out after 330 seconds with no output 15:14:49 INFO - TEST-INFO | screentopng: exit 0 15:14:50 WARNING - TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application terminated with exit code 6 15:14:50 INFO - mozcrash INFO | Downloading symbols from: https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.crashreporter-symbols.zip 15:15:05 INFO - mozcrash INFO | Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpwQOVaC.mozrunner/minidumps/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp /tmp/tmp54bBuq 15:15:15 INFO - mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp 15:15:15 INFO - mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/2d336a24-e58d-63b7-287b740a-5ec9366b.extra 15:15:15 WARNING - PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [@ sse2_blt] 15:15:15 INFO - Crash dump filename: /tmp/tmpwQOVaC.mozrunner/minidumps/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp 15:15:15 INFO - Operating system: Linux 15:15:15 INFO - 0.0.0 Linux 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 15:15:15 INFO - CPU: amd64 15:15:15 INFO - family 6 model 62 stepping 4 15:15:15 INFO - 1 CPU 15:15:15 INFO - Crash reason: SIGBUS 15:15:15 INFO - Crash address: 0x7f0977a71000 15:15:15 INFO - Thread 0 (crashed) 15:15:15 INFO - 0 libxul.so!sse2_blt [emmintrin.h:531d1f6d1cde : 697 + 0x0] 15:15:15 INFO - rbx = 0x0000000000000000 r12 = 0x00007f098e7f4ac0 15:15:15 INFO - r13 = 0x00007f098e7f4240 r14 = 0x0000000020028888 15:15:15 INFO - r15 = 0x0000000000000000 rip = 0x00007f09ab2d556f 15:15:15 INFO - rsp = 0x00007fff1ee08b58 rbp = 0x00007fff1ee08b60 15:15:15 INFO - Found by: given as instruction pointer in context 15:15:15 INFO - 1 libxul.so!sse2_composite_copy_area [pixman-sse2.c:531d1f6d1cde : 4812 + 0x14] 15:15:15 INFO - rbx = 0x00007fff1ee08c28 r12 = 0x00007f098e7f4ac0 15:15:15 INFO - r13 = 0x00007f098e7f4240 r14 = 0x0000000020028888 15:15:15 INFO - r15 = 0x0000000000000000 rip = 0x00007f09ab2d563c 15:15:15 INFO - rsp = 0x00007fff1ee08b70 rbp = 0x00007fff1ee08bb0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 2 libxul.so!_moz_pixman_image_composite32 [pixman.c:531d1f6d1cde : 707 + 0x5] 15:15:15 INFO - rbx = 0x00007fff1ee08c28 r12 = 0x00007f098e7f4ac0 15:15:15 INFO - r13 = 0x00007f098e7f4240 r14 = 0x0000000020028888 15:15:15 INFO - r15 = 0x0000000000000000 rip = 0x00007f09ab2da968 15:15:15 INFO - rsp = 0x00007fff1ee08bc0 rbp = 0x00007fff1ee08cb0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 3 libxul.so!_clip_and_composite_boxes [cairo-image-surface.c:531d1f6d1cde : 3038 + 0x49] 15:15:15 INFO - rbx = 0x00007f098e72db00 r12 = 0x00007fff1ee09030 15:15:15 INFO - r13 = 0x00007fff1ee08dec r14 = 0x00007fff1ee09338 15:15:15 INFO - r15 = 0x0000000000000001 rip = 0x00007f09ab250ef4 15:15:15 INFO - rsp = 0x00007fff1ee08cc0 rbp = 0x00007fff1ee08d80 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 4 libxul.so!_cairo_image_surface_paint [cairo-image-surface.c:531d1f6d1cde : 3325 + 0x26] 15:15:15 INFO - rbx = 0x00007f098e72db00 r12 = 0x0000000000000001 15:15:15 INFO - r13 = 0x00007fff1ee09338 r14 = 0x0000000000000000 15:15:15 INFO - r15 = 0x0000000000000000 rip = 0x00007f09ab252687 15:15:15 INFO - rsp = 0x00007fff1ee08d90 rbp = 0x00007fff1ee092b0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 5 libxul.so!_cairo_surface_paint [cairo-surface.c:531d1f6d1cde : 2109 + 0xd] 15:15:15 INFO - rbx = 0x00007f098e72db00 r12 = 0x0000000000000001 15:15:15 INFO - r13 = 0x0000000000000000 r14 = 0x00007fff1ee09338 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09ab26cbc0 15:15:15 INFO - rsp = 0x00007fff1ee092c0 rbp = 0x00007fff1ee092e0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 6 libxul.so!_cairo_gstate_fill [cairo-gstate.c:531d1f6d1cde : 1285 + 0x27] 15:15:15 INFO - rbx = 0x00007f098c5531c8 r12 = 0x00007f098c553368 15:15:15 INFO - r13 = 0x0000000000000001 r14 = 0x00007fff1ee09338 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09ab252b3f 15:15:15 INFO - rsp = 0x00007fff1ee092f0 rbp = 0x00007fff1ee09450 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 7 libxul.so!_moz_cairo_fill_preserve [cairo.c:531d1f6d1cde : 2473 + 0xf] 15:15:15 INFO - rbx = 0x00007f098c553000 r12 = 0x00007fff1ee09858 15:15:15 INFO - r13 = 0x00007fff1ee09860 r14 = 0x00007f098da3cc00 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09ab26ff40 15:15:15 INFO - rsp = 0x00007fff1ee09460 rbp = 0x00007fff1ee09470 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 8 libxul.so!_moz_cairo_fill [cairo.c:531d1f6d1cde : 2449 + 0x4] 15:15:15 INFO - rbx = 0x00007f098c553000 r12 = 0x00007fff1ee09858 15:15:15 INFO - r13 = 0x00007fff1ee09860 r14 = 0x00007f098da3cc00 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09ab26ff63 15:15:15 INFO - rsp = 0x00007fff1ee09480 rbp = 0x00007fff1ee09490 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 9 libxul.so!mozilla::gfx::DrawTargetCairo::CopySurfaceInternal(_cairo_surface*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits> const&) [DrawTargetCairo.cpp:531d1f6d1cde : 1067 + 0x8] 15:15:15 INFO - rbx = 0x00007f098cd75ac0 r12 = 0x00007fff1ee09858 15:15:15 INFO - r13 = 0x00007fff1ee09860 r14 = 0x00007f098da3cc00 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a96bc629 15:15:15 INFO - rsp = 0x00007fff1ee094a0 rbp = 0x00007fff1ee09640 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 10 libxul.so!mozilla::gfx::DrawTargetCairo::CopySurface(mozilla::gfx::SourceSurface*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits> const&) [DrawTargetCairo.cpp:531d1f6d1cde : 1093 + 0x10] 15:15:15 INFO - rbx = 0x00007f098cd75ac0 r12 = 0x00007f098da3cc00 15:15:15 INFO - r13 = 0x00007fff1ee09860 r14 = 0x00007fff1ee09858 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a96c7256 15:15:15 INFO - rsp = 0x00007fff1ee09650 rbp = 0x00007fff1ee09810 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 11 libxul.so!mozilla::layers::CopyableCanvasLayer::UpdateTarget(mozilla::gfx::DrawTarget*) [CopyableCanvasLayer.cpp:531d1f6d1cde : 98 + 0x1] 15:15:15 INFO - rbx = 0x00007f098c534800 r12 = 0x00007f098cd75ac0 15:15:15 INFO - r13 = 0x00007f098c518238 r14 = 0x00007f098c534801 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a9745e8c 15:15:15 INFO - rsp = 0x00007fff1ee09820 rbp = 0x00007fff1ee098a0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 12 libxul.so!mozilla::layers::CanvasClient2D::Update(mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::layers::ClientCanvasLayer*) [CanvasClient.cpp:531d1f6d1cde : 112 + 0x4] 15:15:15 INFO - rbx = 0x00007f098c518200 r12 = 0x00007f098c518201 15:15:15 INFO - r13 = 0x00007f098c518238 r14 = 0x00007f098c534801 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a978aad6 15:15:15 INFO - rsp = 0x00007fff1ee098b0 rbp = 0x00007fff1ee09920 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 13 libxul.so!mozilla::layers::ClientCanvasLayer::RenderLayer() [ClientCanvasLayer.cpp:531d1f6d1cde : 133 + 0x5] 15:15:15 INFO - rbx = 0x00007f098c534800 r12 = 0x00007f098c534ae0 15:15:15 INFO - r13 = 0x0000000000000006 r14 = 0x00007fff1ee09968 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a9788de1 15:15:15 INFO - rsp = 0x00007fff1ee09930 rbp = 0x00007fff1ee09950 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 14 libxul.so!mozilla::layers::ClientContainerLayer::RenderLayer() [ClientContainerLayer.h:531d1f6d1cde : 65 + 0x8] 15:15:15 INFO - rbx = 0x00007f098bd72c00 r12 = 0x00007f098c534800 15:15:15 INFO - r13 = 0x0000000000000006 r14 = 0x00007fff1ee09968 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a97885dd 15:15:15 INFO - rsp = 0x00007fff1ee09960 rbp = 0x00007fff1ee09a00 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 15 libxul.so!mozilla::layers::ClientLayerManager::EndTransactionInternal(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) [ClientLayerManager.cpp:531d1f6d1cde : 281 + 0x9] 15:15:15 INFO - rbx = 0x00007f099f5e9ff0 r12 = 0x00007f09aaa44d64 15:15:15 INFO - r13 = 0x00007f098bd72e88 r14 = 0x00007f098bd72c00 15:15:15 INFO - r15 = 0x00007f09a977cf40 rip = 0x00007f09a977f19c 15:15:15 INFO - rsp = 0x00007fff1ee09a10 rbp = 0x00007fff1ee09a90 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 16 libxul.so!mozilla::layers::ClientLayerManager::EndTransaction(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) [ClientLayerManager.cpp:531d1f6d1cde : 324 + 0x10] 15:15:15 INFO - rbx = 0x00007f099f5e9ff0 r12 = 0x0000000000000000 15:15:15 INFO - r13 = 0x00007f09aaa44d64 r14 = 0x00007fff1ee09fb8 15:15:15 INFO - r15 = 0x00007f098ccf6420 rip = 0x00007f09a978f931 15:15:15 INFO - rsp = 0x00007fff1ee09aa0 rbp = 0x00007fff1ee09ac0 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 17 libxul.so!nsDisplayList::PaintRoot(nsDisplayListBuilder*, nsRenderingContext*, unsigned int) [nsDisplayList.cpp:531d1f6d1cde : 1753 + 0x16] 15:15:15 INFO - rbx = 0x00007fff1ee09fb8 r12 = 0x0000000000000000 15:15:15 INFO - r13 = 0x0000000000000000 r14 = 0x00007f098da05800 15:15:15 INFO - r15 = 0x00007f098ccf6420 rip = 0x00007f09aaa8bf79 15:15:15 INFO - rsp = 0x00007fff1ee09ad0 rbp = 0x00007fff1ee09d10 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 18 libxul.so!nsLayoutUtils::PaintFrame(nsRenderingContext*, nsIFrame*, nsRegion const&, unsigned int, unsigned int) [nsLayoutUtils.cpp:531d1f6d1cde : 3400 + 0x4] 15:15:15 INFO - rbx = 0x00007f098cda5920 r12 = 0x0000000000000000 15:15:15 INFO - r13 = 0x00000084dbf300b9 r14 = 0x00007f09942f55e0 15:15:15 INFO - r15 = 0x00007fff1ee09d00 rip = 0x00007f09aaaca3f2 15:15:15 INFO - rsp = 0x00007fff1ee09d20 rbp = 0x00007fff1ee0a740 15:15:15 INFO - Found by: call frame info 15:15:15 INFO - 19 libxul.so!PresShell::Paint(nsView*, nsRegion const&, unsigned int) [nsPresShell.cpp:531d1f6d1cde : 5995 + 0x16] 15:15:15 INFO - rbx = 0x00007f099f5e9ff0 r12 = 0x00007f098da05800 15:15:15 INFO - r13 = 0x00007f098e93f000 r14 = 0x00007f098cda5920 15:15:15 INFO - r15 = 0x0000000000000001 rip = 0x00007f09aaacadfd 15:15:15 INFO - rsp = 0x00007fff1ee0a750 rbp = 0x00007fff1ee0a850 15:15:15 INFO - Found by: call frame info not sure if this will help, the test case is really small, not sure how to bisect it down.
This is possibly related to the stuff Jeff is already trying to resolve (so far without a lot of success.)
Flags: needinfo?(bas) → needinfo?(jmuizelaar)
|
||
Comment 15•8 years ago
|
||
This looks like it might be caused by an unaligned access in the sse2 code. I'll try to investigate further.
|
|
||
Comment 16•8 years ago
|
||
if you would like to push to try or get some help, just ask- I would rather this be more teamwork than dumping bugs over the fence!
|
|
Reporter | |
Comment 17•8 years ago
|
||
ncameron said in an email that it's been too long since he's touched anything in this area. jrmuizel: let me know if there's anything I could try. Thanks for looking into it!
Flags: needinfo?(ncameron)
|
|
Reporter | |
Comment 18•8 years ago
|
||
I will be disabling the test for now as it is the last issue we have for tests on docker.
|
|
Reporter | |
Comment 19•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce1695327604
|
|
Reporter | |
Comment 20•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c74381548cac
|
|
Reporter | |
Comment 21•8 years ago
|
||
I'm now getting a crash somewhere else: > 19:16:26 WARNING - PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/layout/base/crashtests/852293.html | application crashed [@ libc-2.15.so + 0xe79d3] https://tools.taskcluster.net/task-inspector/#OU17rK5GRo-vUgLpK3dj8w/0
|
|
Reporter | |
Comment 22•8 years ago
|
||
jrmuizel and I spoke about this and we're going to try to determine what is the space left, memory and ulimit. We should also compare with the job on the releng host. For now, we will get results in less than an hour for TC: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e065f476de1b
|
|
Reporter | |
Comment 23•8 years ago
|
||
I've triggered another 4-5 jobs as the current run did not hit any crashes. I barely can tell that there is a difference of disk space before to after the test run. From [1] Before test execution [2] After test execution [3] [1] https://public-artifacts.taskcluster.net/Gs-ol4wtScWPi5XdKx7TxQ/0/public/logs/live_backing.log [2] + free -m total used free shared buffers cached Mem: 3748 1166 2582 0 287 378 -/+ buffers/cache: 500 3248 Swap: 0 0 0 + free -mt total used free shared buffers cached Mem: 3748 1167 2581 0 287 378 -/+ buffers/cache: 501 3247 Swap: 0 0 0 Total: 3748 1167 2581 + df -h Filesystem Size Used Avail Use% Mounted on rootfs 394G 11G 364G 3% / none 394G 11G 364G 3% / tmpfs 1.9G 0 1.9G 0% /dev shm 64M 68K 64M 1% /dev/shm /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/resolv.conf /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/hostname /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/hosts tmpfs 1.9G 0 1.9G 0% /proc/kcore tmpfs 1.9G 0 1.9G 0% /proc/latency_stats tmpfs 1.9G 0 1.9G 0% /proc/timer_stats [3] + ulimit unlimited + df -h Filesystem Size Used Avail Use% Mounted on rootfs 394G 11G 363G 3% / none 394G 11G 363G 3% / tmpfs 1.9G 0 1.9G 0% /dev shm 64M 572K 64M 1% /dev/shm /dev/mapper/instance_storage-all 394G 11G 363G 3% /etc/resolv.conf /dev/mapper/instance_storage-all 394G 11G 363G 3% /etc/hostname /dev/mapper/instance_storage-all 394G 11G 363G 3% /etc/hosts tmpfs 1.9G 0 1.9G 0% /proc/kcore tmpfs 1.9G 0 1.9G 0% /proc/latency_stats tmpfs 1.9G 0 1.9G 0% /proc/timer_stats + free -m total used free shared buffers cached Mem: 3748 2353 1395 0 290 1022 -/+ buffers/cache: 1039 2708 Swap: 0 0 0 + free -mt total used free shared buffers cached Mem: 3748 2353 1395 0 290 1022 -/+ buffers/cache: 1039 2708 Swap: 0 0 0 Total: 3748 2353 1395
|
|
Reporter | |
Comment 24•8 years ago
|
||
The crash has *just* dissapeared. Even the new crash: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&exclusion_profile=false&revision=1a61ad3416cf,e065f476de1b&filter-searchStr=tc%20crashtest dustin: what could have changed? Even on a failing push the job has stopped crashing. In those two pushes we have the same build and same m-c code. /me retriggers even in older pushes.
Flags: needinfo?(dustin)
|
|
Reporter | |
Comment 25•8 years ago
|
||
It seems that the new crash is intermittent. The older one was permanent. I will push again *without* disabling the perma issue. I've got a new instance of the new crash [1] This time 852293.html crashes on SIGABRT [2] It seems that we have both SIGBUS and SIGABRT crashes. The script does not give me the disk space and mem because I put the code in the wrong place, however, I have the space before the script run [4] The starting free memory is a little less (46.7% free vs 68.9% free) than the time that it run without a crash (comment 23). > total used free > Total: 3748 1996 1752 vs > total used free > Total: 3748 1167 2581 On the good run, we go from 2581 MB free to 1395 MB after execution (that is 1186 MB less). However, we don't know if at any given moment we peak higher than that. Should we get the output of ps aux? or something similar? [1] https://public-artifacts.taskcluster.net/cpA4yqOrQ9ejNXvbsSjV4w/0/public/logs/live_backing.log [2] 21:24:04 WARNING - PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/layout/base/crashtests/852293.html | application crashed [@ libc-2.15.so + 0xe79d3] 21:24:04 INFO - Crash dump filename: /tmp/tmp7XtF7U.mozrunner/minidumps/7c0b0d25-c6bf-c639-69cad95a-21a6e7e6.dmp 21:24:04 INFO - Operating system: Linux 21:24:04 INFO - 0.0.0 Linux 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 21:24:04 INFO - CPU: amd64 21:24:04 INFO - family 6 model 62 stepping 4 21:24:04 INFO - 1 CPU 21:24:04 INFO - Crash reason: SIGABRT [3] From https://treeherder.mozilla.org/#/jobs?repo=try&author=armenzg@mozilla.com&filter-searchStr=tc crashtest&revision=e065f476de1b,5d322c4c616a https://public-artifacts.taskcluster.net/U4xNtbZBQIe4LgBFKweLCw/0/public/logs/live_backing.log - SIGBUS https://public-artifacts.taskcluster.net/EJwdXoU6RzGTXKPkOTXyJg/0/public/logs/live_backing.log - SIGBUS https://public-artifacts.taskcluster.net/OU17rK5GRo-vUgLpK3dj8w/0/public/logs/live_backing.log - SIGABRT https://public-artifacts.taskcluster.net/cpA4yqOrQ9ejNXvbsSjV4w/0/public/logs/live_backing.log - SIGABRT https://public-artifacts.taskcluster.net/Fhpy8OMmRlWL0BdIvaRmfQ/0/public/logs/live_backing.log - SIGBUS [4] + free -mt total used free shared buffers cached Mem: 3748 1996 1752 0 215 1147 -/+ buffers/cache: 633 3115 Swap: 0 0 0 Total: 3748 1996 1752 + df -h Filesystem Size Used Avail Use% Mounted on rootfs 394G 11G 364G 3% / none 394G 11G 364G 3% / tmpfs 1.9G 0 1.9G 0% /dev shm 64M 68K 64M 1% /dev/shm /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/resolv.conf /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/hostname /dev/mapper/instance_storage-all 394G 11G 364G 3% /etc/hosts tmpfs 1.9G 0 1.9G 0% /proc/kcore tmpfs 1.9G 0 1.9G 0% /proc/latency_stats tmpfs 1.9G 0 1.9G 0% /proc/timer_stats
Flags: needinfo?(dustin)
|
||
Comment 26•8 years ago
|
||
Yes, ps output would be good. I'm not sure exactly what those statistics are showing, since in [4] there are two different numbers listed for cache and buffers. The kernel tries to make memory useful, caching disk blocks and other potentially useful information, and I don't know the details of how it represents that here. At any rate, even if firefox-bin used all 3.75G of RAM, once that process exited, you wouldn't see anything in the output. So I'm not sure this is a good way to chase down this particular issue.
In particular, running out of memory generally ends in SIGTERM/SIGKILL, not SIGBUS or SIGABRT. To verify, I cobbled together
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
void main() {
int pid;
int status;
if ((pid = fork()) == 0) {
while (1) {
malloc(1024*1024);
}
} else {
wait(&status);
printf("exit status %d\n", status);
printf("WIFSIGNALED %d\n", WIFSIGNALED(status));
printf("WEXITSTATUS %d\n", WEXITSTATUS(status));
printf("WTERMSIG %d\n", WTERMSIG(status));
}
}
and sure enough
dustin@dustin-moz-devel ~/p/m-c $ ./tmp
exit status 9
WIFSIGNALED 1
WEXITSTATUS 0
WTERMSIG 9
where 9 is SIGKILL.
SIGBUS, as Jeff pointed out in comment 15, is more often associated with misuse of memory -- usually unaligned access (as an invalid address would end in SIGSEGV). That the failure was in sse2_blt in comment 15 certainly points to that, too! And alignment would generally be intermittent, probably with some binary fractional probability (1/2 of runs fail, 1/4 fail, 1/8 fail, etc.) Basically, I think Jeff is going in the right direction here.
I wonder if a good debugging technique would be to add some assertion about alignment:
diff --git a/gfx/cairo/libpixman/src/pixman-sse2.c b/gfx/cairo/libpixman/src/pixman-sse2.c
--- a/gfx/cairo/libpixman/src/pixman-sse2.c
+++ b/gfx/cairo/libpixman/src/pixman-sse2.c
@@ -4724,16 +4724,18 @@ sse2_blt (pixman_implementation_t *imp,
src_stride *= 4;
dst_stride *= 4;
}
else
{
return FALSE;
}
+ assert(dst_bytes & 0x7 == 0);
+
while (height--)
{
int w;
uint8_t *s = src_bytes;
uint8_t *d = dst_bytes;
src_bytes += src_stride;
dst_bytes += dst_stride;
w = byte_width;
and then see if that assertion fails. It's worth noting that the source uses unaligned operations, so I think only the destination needs to be checked. I'm not sure which NS_ macros are available here, but I'm sure Jeff knows :)|
|
Reporter | |
Comment 27•8 years ago
|
||
It seems that ulimit -a is what we really wanted. Releng host [1] Docker instance [2] For giggles, running docker on my local machine gives different values [3] Did you want ulimit -a to be run *after* the test jobs? or is it fine before it? The clear differences I see are pending signals, open files and max user processes > pending signals (-i) 29669 > open files (-n) 1024 > max user processes (-u) 29669 vs > pending signals (-i) 29509 > open files (-n) 524288 > max user processes (-u) 1048576 Is this information relevant? [1] [cltbld@tst-linux64-ec2-armenzg.test.releng.use1.mozilla.com ~]$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 29669 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 29669 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited [2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=e27e161c88ed&selectedJob=16209769 core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 29509 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 524288 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1048576 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited [3] armenzg@armenzg-thinkpad:~/repos/mozilla-central/testing/taskcluster/tasks/tests$ docker run -ti taskcluster/desktop-test:0.5.7 root@taskcluster-worker:~# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 124671 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 524288 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 524288 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
|
|
||
Comment 28•8 years ago
|
||
It looks like this probably caused by the size of /dev/shm being too small. In docker it defaults to 64MB but we likely need more. See also: https://github.com/jvermillard/docker/commit/77faf17586cecacf79765d47983363962827cb45
Flags: needinfo?(jmuizelaar)
|
|
Reporter | |
Comment 29•8 years ago
|
||
Bug 1245243 cleared this bug. \o/
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•