Closed Bug 1233554 Opened 7 years ago Closed 7 years ago

Linux x64 debug crashtest e10s crash in docker image (tests/reftest/tests/dom/canvas/crashtests/780392-1.html)

Categories

(Core :: Graphics, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Tracking Status
e10s + ---

People

(Reporter: armenzg, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: gfx-noted)

Attachments

(2 files)

We're trying to run Linux 64 debug tests inside of docker containers.
In the following push [1] we have the e10s crashtest crashing [2].
If you "inspect the task" [3] you will see the .dmp files generated [4]
There is also a screenshot of a dialog saying "The application classic guest session has closed unexpectedly" [5]

I don't know if the following line in the output is relevant:
> [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459

The test in question is this one:
https://dxr.mozilla.org/mozilla-central/source/dom/canvas/crashtests/780392-1.html

Any thoughts on what could be causing this crash? (STR below)

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3604a422edc3
[2] https://public-artifacts.taskcluster.net/b56UUICuQrOXPU4PzKdv5Q/0/public/logs/live_backing.log
[3] https://tools.taskcluster.net/task-inspector/#Hv1NIUCHRneucYyv1gxQrw/0
[4] https://queue.taskcluster.net/v1/task/Hv1NIUCHRneucYyv1gxQrw/runs/0/artifacts/public/test_info//4299947f-0525-b3b7-71e62317-53354004.dmp
[5] https://public-artifacts.taskcluster.net/Hv1NIUCHRneucYyv1gxQrw/0/public/test_info//mozilla-test-fail-screenshot_YpFWcm.png

STR:
####
sudo modprobe v4l2loopback # If you only have /dev/video0

image_name=armenzg/desktop-test:0.5.1
docker pull 'armenzg/desktop-test:0.5.1'

# Run docker command
docker run -ti \
  --name "${container_name}" \
  --device=/dev/video1:/dev/video1 \
  -e START_VNC='true' \
  -e SKIP_RUNNING_SCRIPT='true' \
  -e DISPLAY=:0 \
  -e MOZILLA_BUILD_URL='https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.tar.bz2' \
  -e MOZHARNESS_SCRIPT='mozharness/scripts/desktop_unittest.py' \
  -e GECKO_HEAD_REPOSITORY='https://hg.mozilla.org/try/' \
  -e MOZHARNESS_URL='https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/mozharness.zip' \
  -e NEED_PULSEAUDIO='true' \
  -e NEED_WINDOW_MANAGER='true' \
  -e MOZHARNESS_CONFIG='mozharness/configs/unittests/linux_unittest.py mozharness/configs/remove_executables.py
' \
  -e GECKO_HEAD_REV='3961f20f20a358a53b81231050395e9fc4e415f3' \
  ${image_name} \


chmod +x bin/test.sh
~/bin/test.sh
cd workspace

sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s

##### [1]
17:23:32     INFO -  REFTEST TEST-START | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html
17:23:32     INFO -  REFTEST TEST-LOAD | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | 251 / 2968 (8%)
17:23:34     INFO -  ###!!! [Parent][MessageChannel] Error: (msgtype=0x2A0081,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv
17:23:34     INFO -  JavaScript error: resource://app/modules/ContentCrashHandlers.jsm, line 75: TypeError: browser.permanentKey is not a non-null object
17:23:34     INFO -  [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459
17:29:04  WARNING -  TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application timed out after 330 seconds with no output
17:29:04     INFO -  TEST-INFO | screentopng: exit 0
17:29:04  WARNING -  TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application terminated with exit code 6
17:29:05     INFO -  mozcrash INFO | Downloading symbols from: https://queue.taskcluster.net/v1/task/LRLjlqenTQOS7O3yEoURPA/artifacts/public/build/target.crashreporter-symbols.zip
17:29:08     INFO -  mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/207fc14f-c996-1046-039c6068-78cfe751.dmp
17:29:08     INFO -  mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/207fc14f-c996-1046-039c6068-78cfe751.extra
17:29:08  WARNING -  PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [unknown top frame]
17:29:08     INFO -  Crash dump filename: /tmp/tmpMnGGsS.mozrunner/minidumps/207fc14f-c996-1046-039c6068-78cfe751.dmp
17:29:08     INFO -  MINIDUMP_STACKWALK not set, can't process dump.
17:29:08     INFO -  mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/526bb3e4-24a2-cef8-5184621d-5c5290ef.dmp
17:29:08     INFO -  mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/526bb3e4-24a2-cef8-5184621d-5c5290ef.extra
17:29:08  WARNING -  PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [unknown top frame]
17:29:08     INFO -  Crash dump filename: /tmp/tmpMnGGsS.mozrunner/minidumps/526bb3e4-24a2-cef8-5184621d-5c5290ef.dmp
17:29:08     INFO -  MINIDUMP_STACKWALK not set, can't process dump.
17:29:08     INFO -  WARNING | leakcheck | refcount logging is off, so leaks can't be detected!
17:29:08     INFO -  REFTEST INFO | runreftest.py | Running tests: end.
17:29:08    ERROR - Return code: 6
17:29:08    ERROR - No tests run or test summary not found
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #0)
> We're trying to run Linux 64 debug tests inside of docker containers.
> In the following push [1] we have the e10s crashtest crashing [2].
> If you "inspect the task" [3] you will see the .dmp files generated [4]

What's up with us not getting a stack printed in the log? I see:
17:29:08     INFO -  MINIDUMP_STACKWALK not set, can't process dump.

Shouldn't mozharness be using tooltool to fetch a minidump_stackwalk binary?

> There is also a screenshot of a dialog saying "The application classic guest
> session has closed unexpectedly" [5]

This is "indicator-session-service" crashing, I don't know if it's related or not. It would be interesting to know if this happens on other test runs.

> I don't know if the following line in the output is relevant:
> > [Parent 1002] WARNING: pipe error (57): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459

This is normal output from an e10s process when one of the processes crashes. It's the other process realizing the pipe it uses for IPC has been closed.
Attached file stack
Here's a stack from that minidump.
I already gave Armen some tips to check yesterday. But I had another look at the log and found the following:

> 21:40:12     INFO -  'download_minidump_stackwalk': False,

This explains why it is happening. The reason for that is the following config file which you make use of:

https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/remove_executables.py
I will move the lack of dumping to its own bug and leave this one for crash itself.
Component: General → Graphics
Product: Testing → Core
Attached file disassembly
Here's annotated disassembly of the crash location, if it helps anyone.
Hello poiru, we're seeing some crashes when running e10s crashtests inside of docker containers (we're moving towards them).
ted has a crash dump on comment 5; would you be able to look into this bug and let us know if we need to change anything in the docker image?

Thanks!
Flags: needinfo?(birunthan)
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #6)
> Hello poiru, we're seeing some crashes when running e10s crashtests inside
> of docker containers (we're moving towards them).
> ted has a crash dump on comment 5; would you be able to look into this bug
> and let us know if we need to change anything in the docker image?
> 
> Thanks!

I only moved that test from one directory to another (you can use `hg log --follow` to track across renames). The test was originally added in https://hg.mozilla.org/mozilla-central/rev/8a736f13c5f8

You might want to try the author/reviewer of that commit.
Flags: needinfo?(birunthan)
Thanks poiru. My bad.

Hi nrc, bas, we're seeing some crashes when running e10s crashtests inside of docker containers (we're moving towards them).
ted has a crash dump on comment 5; would you be able to look into this bug and let us know if we need to change anything in the docker image?

Thanks!
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
Whiteboard: gfx-noted
Removing NI as by now too much has changed and we will need new data.
If we still hit the issue we will reflag and hope for better 2 way communication.
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
Blocks: e10s-tests
tracking-e10s: --- → +
nrc, bas: please let us know if you could have a look at this or if someone else could. This is going to start getting a lot of attention so please help us at least redirect if necessary. Thanks for your help!

We want to run Linux64 debug test jobs under docker.
We're getting closer to it, however, we now have to figure test issues.
In this bug we would like to look at fixing a crash we're seeing in e10s crashtest [1].




Here are the steps to reproduce locally:
# First time prep step
sudo apt-get install v4l2loopback-dkms
# Every time after your host is rebooted
sudo modprobe v4l2loopback # This will create a device under /dev/video*

NOTE: The value --device is specific to your host. Please check under /dev/video* and pick the one with the highest numeral

docker run -ti \
  -e GECKO_HEAD_REPOSITORY='https://hg.mozilla.org/try/' \
  -e GECKO_HEAD_REV='921c3df79ac83ab115bc054c349f29fc7526a534' \
  -e MOZHARNESS_CONFIG='mozharness/configs/unittests/linux_unittest.py mozharness/configs/remove_executables.py
' \
  -e MOZHARNESS_SCRIPT='mozharness/scripts/desktop_unittest.py' \
  -e MOZHARNESS_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/mozharness.zip' \
  -e MOZILLA_BUILD_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2' \
  -e NEED_PULSEAUDIO='true' \
  -e NEED_WINDOW_MANAGER='true' \
  -e START_VNC='true' \
  -e SKIP_MOZHARNESS_RUN='true' \
  --device /dev/video1:/dev/video1 \
  taskcluster/desktop-test:0.5.5

# Now inside of docker
./bin/test.sh
# You can now connect to VNC on your host (xtightvncviewer 172.17.0.2)
cd workspace
sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s --no-run-tests
# Modify tests/reftest/tests/testing/crashtest/crashtests.list and 
sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=crashtest --total-chunk=1 --this-chunk=1 --e10s --run-tests --binary-path application/firefox/firefox-bin

Notice that I'm first running with --no-run-tests to set everything up and then with --run-tests which will only run the tests and not clobber anything.

If you close your VNC connection, you will have to call ./bin/test.sh again (maybe just call `x11vnc &` alone).

[1] https://public-artifacts.taskcluster.net/OYOkzqk9TmqDtRKxABjh6Q/0/public/logs/live_backing.log
Flags: needinfo?(ncameron)
Flags: needinfo?(bas)
Summary: Linux x64 debug crashtest e10s crash in docker image → Linux x64 debug crashtest e10s crash in docker image (tests/reftest/tests/dom/canvas/crashtests/780392-1.html)
For reference, this is a push where the crashtest was run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=921c3df79ac8
Alternatively instead of running Mozharness with --run-tests you can do this (which is equivalent):
cd ~/workspace/build
source /home/worker/workspace/build/venv/bin/activate
sudo -E -u worker /home/worker/workspace/build/venv/bin/python -u /home/worker/workspace/build/tests/reftest/runreftest.py --e10s --total-chunks 1 --this-chunk 1 --appname=/home/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.crashreporter-symbols.zip --suite=crashtest -- tests/reftest/tests/testing/crashtest/crashtests.list
from the log (https://public-artifacts.taskcluster.net/b6NGMbKLQ7ed_zIlVh4kqw/0/public/logs/live_backing.log):

15:09:12     INFO -  REFTEST TEST-START | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html
15:09:12     INFO -  REFTEST TEST-LOAD | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | 251 / 2976 (8%)
15:09:12     INFO -  ++DOMWINDOW == 62 (0x7f098d7aec00) [pid = 1160] [serial = 713] [outer = 0x7f0991035400]
15:09:12     INFO -  --DOMWINDOW == 61 (0x7f098d3a2400) [pid = 1160] [serial = 607] [outer = (nil)] [url = data:text/html,<meta%20charset=UTF-8><body><video%20src=http://localhost:8080/%20controls=true%20loop=true%20autoplay=true%20autobuffer=false></video>]
15:09:12     INFO -  --DOMWINDOW == 60 (0x7f098c534400) [pid = 1160] [serial = 606] [outer = (nil)] [url = data:text/html,<meta%20charset=UTF-8><body><video%20src=http://localhost:8080/%20controls=true%20loop=true%20autoplay=true%20autobuffer=false></video>]
15:09:12     INFO -  --DOMWINDOW == 59 (0x7f098c535000) [pid = 1160] [serial = 592] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/845093-1.html]
15:09:12     INFO -  --DOMWINDOW == 58 (0x7f098c538800) [pid = 1160] [serial = 594] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/845093-2.html]
15:09:12     INFO -  --DOMWINDOW == 57 (0x7f098bd7c400) [pid = 1160] [serial = 663] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/bindings/crashtests/832899.html]
15:09:12     INFO -  --DOMWINDOW == 56 (0x7f098cd07400) [pid = 1160] [serial = 600] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/849727.html]
15:09:12     INFO -  --DOMWINDOW == 55 (0x7f098c541400) [pid = 1160] [serial = 602] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/849732.html]
15:09:12     INFO -  --DOMWINDOW == 54 (0x7f098d292400) [pid = 1160] [serial = 677] [outer = 0x7f098d291800] [url = about:blank]
15:09:12     INFO -  --DOMWINDOW == 53 (0x7f098d184000) [pid = 1160] [serial = 619] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/898906.html]
15:09:12     INFO -  --DOMWINDOW == 52 (0x7f098bd7cc00) [pid = 1160] [serial = 645] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/1157995.html]
15:09:12     INFO -  --DOMWINDOW == 51 (0x7f098d28fc00) [pid = 1160] [serial = 647] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/1181619.html]
15:09:12     INFO -  --DOMWINDOW == 50 (0x7f098c53bc00) [pid = 1160] [serial = 613] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/864448.html]
15:09:12     INFO -  --DOMWINDOW == 49 (0x7f098da05000) [pid = 1160] [serial = 627] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/942979.html]
15:09:12     INFO -  --DOMWINDOW == 48 (0x7f098bd75400) [pid = 1160] [serial = 633] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/978646.html]
15:09:13     INFO -  --DOMWINDOW == 47 (0x7f098d291c00) [pid = 1160] [serial = 653] [outer = (nil)] [url = http://localhost:43729/1452870464911/1/crashtests/xhr_abortinprogress.html]
15:09:13     INFO -  --DOMWINDOW == 46 (0x7f098cd0a000) [pid = 1160] [serial = 655] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/xhr_empty_datauri.html]
15:09:13     INFO -  --DOMWINDOW == 45 (0x7f098cd61800) [pid = 1160] [serial = 657] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/base/crashtests/xhr_html_nullresponse.html]
15:09:18     INFO -  ###!!! [Parent][MessageChannel] Error: (msgtype=0x2A0084,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv
15:09:18     INFO -  JavaScript error: resource://app/modules/ContentCrashHandlers.jsm, line 75: TypeError: browser.permanentKey is not a non-null object
15:09:18     INFO -  [Parent 1064] WARNING: pipe error (58): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 459
15:14:48  WARNING -  TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application timed out after 330 seconds with no output
15:14:49     INFO -  TEST-INFO | screentopng: exit 0
15:14:50  WARNING -  TEST-UNEXPECTED-FAIL | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application terminated with exit code 6
15:14:50     INFO -  mozcrash INFO | Downloading symbols from: https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.crashreporter-symbols.zip
15:15:05     INFO -  mozcrash INFO | Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpwQOVaC.mozrunner/minidumps/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp /tmp/tmp54bBuq
15:15:15     INFO -  mozcrash INFO | Saved minidump as /home/worker/workspace/build/blobber_upload_dir/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp
15:15:15     INFO -  mozcrash INFO | Saved app info as /home/worker/workspace/build/blobber_upload_dir/2d336a24-e58d-63b7-287b740a-5ec9366b.extra
15:15:15  WARNING -  PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/dom/canvas/crashtests/780392-1.html | application crashed [@ sse2_blt]
15:15:15     INFO -  Crash dump filename: /tmp/tmpwQOVaC.mozrunner/minidumps/2d336a24-e58d-63b7-287b740a-5ec9366b.dmp
15:15:15     INFO -  Operating system: Linux
15:15:15     INFO -                    0.0.0 Linux 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64
15:15:15     INFO -  CPU: amd64
15:15:15     INFO -       family 6 model 62 stepping 4
15:15:15     INFO -       1 CPU
15:15:15     INFO -  Crash reason:  SIGBUS
15:15:15     INFO -  Crash address: 0x7f0977a71000
15:15:15     INFO -  Thread 0 (crashed)
15:15:15     INFO -   0  libxul.so!sse2_blt [emmintrin.h:531d1f6d1cde : 697 + 0x0]
15:15:15     INFO -      rbx = 0x0000000000000000   r12 = 0x00007f098e7f4ac0
15:15:15     INFO -      r13 = 0x00007f098e7f4240   r14 = 0x0000000020028888
15:15:15     INFO -      r15 = 0x0000000000000000   rip = 0x00007f09ab2d556f
15:15:15     INFO -      rsp = 0x00007fff1ee08b58   rbp = 0x00007fff1ee08b60
15:15:15     INFO -      Found by: given as instruction pointer in context
15:15:15     INFO -   1  libxul.so!sse2_composite_copy_area [pixman-sse2.c:531d1f6d1cde : 4812 + 0x14]
15:15:15     INFO -      rbx = 0x00007fff1ee08c28   r12 = 0x00007f098e7f4ac0
15:15:15     INFO -      r13 = 0x00007f098e7f4240   r14 = 0x0000000020028888
15:15:15     INFO -      r15 = 0x0000000000000000   rip = 0x00007f09ab2d563c
15:15:15     INFO -      rsp = 0x00007fff1ee08b70   rbp = 0x00007fff1ee08bb0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   2  libxul.so!_moz_pixman_image_composite32 [pixman.c:531d1f6d1cde : 707 + 0x5]
15:15:15     INFO -      rbx = 0x00007fff1ee08c28   r12 = 0x00007f098e7f4ac0
15:15:15     INFO -      r13 = 0x00007f098e7f4240   r14 = 0x0000000020028888
15:15:15     INFO -      r15 = 0x0000000000000000   rip = 0x00007f09ab2da968
15:15:15     INFO -      rsp = 0x00007fff1ee08bc0   rbp = 0x00007fff1ee08cb0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   3  libxul.so!_clip_and_composite_boxes [cairo-image-surface.c:531d1f6d1cde : 3038 + 0x49]
15:15:15     INFO -      rbx = 0x00007f098e72db00   r12 = 0x00007fff1ee09030
15:15:15     INFO -      r13 = 0x00007fff1ee08dec   r14 = 0x00007fff1ee09338
15:15:15     INFO -      r15 = 0x0000000000000001   rip = 0x00007f09ab250ef4
15:15:15     INFO -      rsp = 0x00007fff1ee08cc0   rbp = 0x00007fff1ee08d80
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   4  libxul.so!_cairo_image_surface_paint [cairo-image-surface.c:531d1f6d1cde : 3325 + 0x26]
15:15:15     INFO -      rbx = 0x00007f098e72db00   r12 = 0x0000000000000001
15:15:15     INFO -      r13 = 0x00007fff1ee09338   r14 = 0x0000000000000000
15:15:15     INFO -      r15 = 0x0000000000000000   rip = 0x00007f09ab252687
15:15:15     INFO -      rsp = 0x00007fff1ee08d90   rbp = 0x00007fff1ee092b0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   5  libxul.so!_cairo_surface_paint [cairo-surface.c:531d1f6d1cde : 2109 + 0xd]
15:15:15     INFO -      rbx = 0x00007f098e72db00   r12 = 0x0000000000000001
15:15:15     INFO -      r13 = 0x0000000000000000   r14 = 0x00007fff1ee09338
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09ab26cbc0
15:15:15     INFO -      rsp = 0x00007fff1ee092c0   rbp = 0x00007fff1ee092e0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   6  libxul.so!_cairo_gstate_fill [cairo-gstate.c:531d1f6d1cde : 1285 + 0x27]
15:15:15     INFO -      rbx = 0x00007f098c5531c8   r12 = 0x00007f098c553368
15:15:15     INFO -      r13 = 0x0000000000000001   r14 = 0x00007fff1ee09338
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09ab252b3f
15:15:15     INFO -      rsp = 0x00007fff1ee092f0   rbp = 0x00007fff1ee09450
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   7  libxul.so!_moz_cairo_fill_preserve [cairo.c:531d1f6d1cde : 2473 + 0xf]
15:15:15     INFO -      rbx = 0x00007f098c553000   r12 = 0x00007fff1ee09858
15:15:15     INFO -      r13 = 0x00007fff1ee09860   r14 = 0x00007f098da3cc00
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09ab26ff40
15:15:15     INFO -      rsp = 0x00007fff1ee09460   rbp = 0x00007fff1ee09470
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   8  libxul.so!_moz_cairo_fill [cairo.c:531d1f6d1cde : 2449 + 0x4]
15:15:15     INFO -      rbx = 0x00007f098c553000   r12 = 0x00007fff1ee09858
15:15:15     INFO -      r13 = 0x00007fff1ee09860   r14 = 0x00007f098da3cc00
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09ab26ff63
15:15:15     INFO -      rsp = 0x00007fff1ee09480   rbp = 0x00007fff1ee09490
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -   9  libxul.so!mozilla::gfx::DrawTargetCairo::CopySurfaceInternal(_cairo_surface*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits> const&) [DrawTargetCairo.cpp:531d1f6d1cde : 1067 + 0x8]
15:15:15     INFO -      rbx = 0x00007f098cd75ac0   r12 = 0x00007fff1ee09858
15:15:15     INFO -      r13 = 0x00007fff1ee09860   r14 = 0x00007f098da3cc00
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a96bc629
15:15:15     INFO -      rsp = 0x00007fff1ee094a0   rbp = 0x00007fff1ee09640
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  10  libxul.so!mozilla::gfx::DrawTargetCairo::CopySurface(mozilla::gfx::SourceSurface*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntPointTyped<mozilla::gfx::UnknownUnits> const&) [DrawTargetCairo.cpp:531d1f6d1cde : 1093 + 0x10]
15:15:15     INFO -      rbx = 0x00007f098cd75ac0   r12 = 0x00007f098da3cc00
15:15:15     INFO -      r13 = 0x00007fff1ee09860   r14 = 0x00007fff1ee09858
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a96c7256
15:15:15     INFO -      rsp = 0x00007fff1ee09650   rbp = 0x00007fff1ee09810
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  11  libxul.so!mozilla::layers::CopyableCanvasLayer::UpdateTarget(mozilla::gfx::DrawTarget*) [CopyableCanvasLayer.cpp:531d1f6d1cde : 98 + 0x1]
15:15:15     INFO -      rbx = 0x00007f098c534800   r12 = 0x00007f098cd75ac0
15:15:15     INFO -      r13 = 0x00007f098c518238   r14 = 0x00007f098c534801
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a9745e8c
15:15:15     INFO -      rsp = 0x00007fff1ee09820   rbp = 0x00007fff1ee098a0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  12  libxul.so!mozilla::layers::CanvasClient2D::Update(mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits>, mozilla::layers::ClientCanvasLayer*) [CanvasClient.cpp:531d1f6d1cde : 112 + 0x4]
15:15:15     INFO -      rbx = 0x00007f098c518200   r12 = 0x00007f098c518201
15:15:15     INFO -      r13 = 0x00007f098c518238   r14 = 0x00007f098c534801
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a978aad6
15:15:15     INFO -      rsp = 0x00007fff1ee098b0   rbp = 0x00007fff1ee09920
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  13  libxul.so!mozilla::layers::ClientCanvasLayer::RenderLayer() [ClientCanvasLayer.cpp:531d1f6d1cde : 133 + 0x5]
15:15:15     INFO -      rbx = 0x00007f098c534800   r12 = 0x00007f098c534ae0
15:15:15     INFO -      r13 = 0x0000000000000006   r14 = 0x00007fff1ee09968
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a9788de1
15:15:15     INFO -      rsp = 0x00007fff1ee09930   rbp = 0x00007fff1ee09950
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  14  libxul.so!mozilla::layers::ClientContainerLayer::RenderLayer() [ClientContainerLayer.h:531d1f6d1cde : 65 + 0x8]
15:15:15     INFO -      rbx = 0x00007f098bd72c00   r12 = 0x00007f098c534800
15:15:15     INFO -      r13 = 0x0000000000000006   r14 = 0x00007fff1ee09968
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a97885dd
15:15:15     INFO -      rsp = 0x00007fff1ee09960   rbp = 0x00007fff1ee09a00
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  15  libxul.so!mozilla::layers::ClientLayerManager::EndTransactionInternal(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) [ClientLayerManager.cpp:531d1f6d1cde : 281 + 0x9]
15:15:15     INFO -      rbx = 0x00007f099f5e9ff0   r12 = 0x00007f09aaa44d64
15:15:15     INFO -      r13 = 0x00007f098bd72e88   r14 = 0x00007f098bd72c00
15:15:15     INFO -      r15 = 0x00007f09a977cf40   rip = 0x00007f09a977f19c
15:15:15     INFO -      rsp = 0x00007fff1ee09a10   rbp = 0x00007fff1ee09a90
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  16  libxul.so!mozilla::layers::ClientLayerManager::EndTransaction(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) [ClientLayerManager.cpp:531d1f6d1cde : 324 + 0x10]
15:15:15     INFO -      rbx = 0x00007f099f5e9ff0   r12 = 0x0000000000000000
15:15:15     INFO -      r13 = 0x00007f09aaa44d64   r14 = 0x00007fff1ee09fb8
15:15:15     INFO -      r15 = 0x00007f098ccf6420   rip = 0x00007f09a978f931
15:15:15     INFO -      rsp = 0x00007fff1ee09aa0   rbp = 0x00007fff1ee09ac0
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  17  libxul.so!nsDisplayList::PaintRoot(nsDisplayListBuilder*, nsRenderingContext*, unsigned int) [nsDisplayList.cpp:531d1f6d1cde : 1753 + 0x16]
15:15:15     INFO -      rbx = 0x00007fff1ee09fb8   r12 = 0x0000000000000000
15:15:15     INFO -      r13 = 0x0000000000000000   r14 = 0x00007f098da05800
15:15:15     INFO -      r15 = 0x00007f098ccf6420   rip = 0x00007f09aaa8bf79
15:15:15     INFO -      rsp = 0x00007fff1ee09ad0   rbp = 0x00007fff1ee09d10
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  18  libxul.so!nsLayoutUtils::PaintFrame(nsRenderingContext*, nsIFrame*, nsRegion const&, unsigned int, unsigned int) [nsLayoutUtils.cpp:531d1f6d1cde : 3400 + 0x4]
15:15:15     INFO -      rbx = 0x00007f098cda5920   r12 = 0x0000000000000000
15:15:15     INFO -      r13 = 0x00000084dbf300b9   r14 = 0x00007f09942f55e0
15:15:15     INFO -      r15 = 0x00007fff1ee09d00   rip = 0x00007f09aaaca3f2
15:15:15     INFO -      rsp = 0x00007fff1ee09d20   rbp = 0x00007fff1ee0a740
15:15:15     INFO -      Found by: call frame info
15:15:15     INFO -  19  libxul.so!PresShell::Paint(nsView*, nsRegion const&, unsigned int) [nsPresShell.cpp:531d1f6d1cde : 5995 + 0x16]
15:15:15     INFO -      rbx = 0x00007f099f5e9ff0   r12 = 0x00007f098da05800
15:15:15     INFO -      r13 = 0x00007f098e93f000   r14 = 0x00007f098cda5920
15:15:15     INFO -      r15 = 0x0000000000000001   rip = 0x00007f09aaacadfd
15:15:15     INFO -      rsp = 0x00007fff1ee0a750   rbp = 0x00007fff1ee0a850
15:15:15     INFO -      Found by: call frame info


not sure if this will help, the test case is really small, not sure how to bisect it down.
This is possibly related to the stuff Jeff is already trying to resolve (so far without a lot of success.)
Flags: needinfo?(bas) → needinfo?(jmuizelaar)
This looks like it might be caused by an unaligned access in the sse2 code. I'll try to investigate further.
if you would like to push to try or get some help, just ask- I would rather this be more teamwork than dumping bugs over the fence!
ncameron said in an email that it's been too long since he's touched anything in this area.

jrmuizel: let me know if there's anything I could try. Thanks for looking into it!
Flags: needinfo?(ncameron)
I will be disabling the test for now as it is the last issue we have for tests on docker.
I'm now getting a crash somewhere else:
> 19:16:26  WARNING -  PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/layout/base/crashtests/852293.html | application crashed [@ libc-2.15.so + 0xe79d3]

https://tools.taskcluster.net/task-inspector/#OU17rK5GRo-vUgLpK3dj8w/0
jrmuizel and I spoke about this and we're going to try to determine what is the space left, memory and ulimit.

We should also compare with the job on the releng host.
For now, we will get results in less than an hour for TC:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e065f476de1b
I've triggered another 4-5 jobs as the current run did not hit any crashes.

I barely can tell that there is a difference of disk space before to after the test run.

From [1]
Before test execution [2]
After test execution [3]

[1] https://public-artifacts.taskcluster.net/Gs-ol4wtScWPi5XdKx7TxQ/0/public/logs/live_backing.log
[2]
+ free -m
             total       used       free     shared    buffers     cached
Mem:          3748       1166       2582          0        287        378
-/+ buffers/cache:        500       3248
Swap:            0          0          0
+ free -mt
             total       used       free     shared    buffers     cached
Mem:          3748       1167       2581          0        287        378
-/+ buffers/cache:        501       3247
Swap:            0          0          0
Total:        3748       1167       2581
+ df -h
Filesystem                        Size  Used Avail Use% Mounted on
rootfs                            394G   11G  364G   3% /
none                              394G   11G  364G   3% /
tmpfs                             1.9G     0  1.9G   0% /dev
shm                                64M   68K   64M   1% /dev/shm
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/resolv.conf
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/hostname
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/hosts
tmpfs                             1.9G     0  1.9G   0% /proc/kcore
tmpfs                             1.9G     0  1.9G   0% /proc/latency_stats
tmpfs                             1.9G     0  1.9G   0% /proc/timer_stats

[3]
+ ulimit
unlimited
+ df -h
Filesystem                        Size  Used Avail Use% Mounted on
rootfs                            394G   11G  363G   3% /
none                              394G   11G  363G   3% /
tmpfs                             1.9G     0  1.9G   0% /dev
shm                                64M  572K   64M   1% /dev/shm
/dev/mapper/instance_storage-all  394G   11G  363G   3% /etc/resolv.conf
/dev/mapper/instance_storage-all  394G   11G  363G   3% /etc/hostname
/dev/mapper/instance_storage-all  394G   11G  363G   3% /etc/hosts
tmpfs                             1.9G     0  1.9G   0% /proc/kcore
tmpfs                             1.9G     0  1.9G   0% /proc/latency_stats
tmpfs                             1.9G     0  1.9G   0% /proc/timer_stats
+ free -m
             total       used       free     shared    buffers     cached
Mem:          3748       2353       1395          0        290       1022
-/+ buffers/cache:       1039       2708
Swap:            0          0          0
+ free -mt
             total       used       free     shared    buffers     cached
Mem:          3748       2353       1395          0        290       1022
-/+ buffers/cache:       1039       2708
Swap:            0          0          0
Total:        3748       2353       1395
The crash has *just* dissapeared. Even the new crash:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&exclusion_profile=false&revision=1a61ad3416cf,e065f476de1b&filter-searchStr=tc%20crashtest

dustin: what could have changed? Even on a failing push the job has stopped crashing. In those two pushes we have the same build and same m-c code.

/me retriggers even in older pushes.
Flags: needinfo?(dustin)
It seems that the new crash is intermittent.
The older one was permanent.
I will push again *without* disabling the perma issue.

I've got a new instance of the new crash [1]
This time 852293.html crashes on SIGABRT [2] It seems that we have both SIGBUS and SIGABRT crashes.


The script does not give me the disk space and mem because I put the code in the wrong place, however, I have the space before the script run [4]
The starting free memory is a little less (46.7% free vs 68.9% free) than the time that it run without a crash (comment 23).
>              total       used       free
> Total:        3748       1996       1752
vs
>              total       used       free
> Total:        3748       1167       2581

On the good run, we go from 2581 MB free to 1395 MB after execution (that is 1186 MB less).
However, we don't know if at any given moment we peak higher than that.

Should we get the output of ps aux? or something similar?


[1] https://public-artifacts.taskcluster.net/cpA4yqOrQ9ejNXvbsSjV4w/0/public/logs/live_backing.log
[2]
21:24:04  WARNING -  PROCESS-CRASH | file:///home/worker/workspace/build/tests/reftest/tests/layout/base/crashtests/852293.html | application crashed [@ libc-2.15.so + 0xe79d3]
21:24:04     INFO -  Crash dump filename: /tmp/tmp7XtF7U.mozrunner/minidumps/7c0b0d25-c6bf-c639-69cad95a-21a6e7e6.dmp
21:24:04     INFO -  Operating system: Linux
21:24:04     INFO -                    0.0.0 Linux 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64
21:24:04     INFO -  CPU: amd64
21:24:04     INFO -       family 6 model 62 stepping 4
21:24:04     INFO -       1 CPU
21:24:04     INFO -  Crash reason:  SIGABRT

[3]
From
https://treeherder.mozilla.org/#/jobs?repo=try&author=armenzg@mozilla.com&filter-searchStr=tc crashtest&revision=e065f476de1b,5d322c4c616a
https://public-artifacts.taskcluster.net/U4xNtbZBQIe4LgBFKweLCw/0/public/logs/live_backing.log - SIGBUS
https://public-artifacts.taskcluster.net/EJwdXoU6RzGTXKPkOTXyJg/0/public/logs/live_backing.log - SIGBUS
https://public-artifacts.taskcluster.net/OU17rK5GRo-vUgLpK3dj8w/0/public/logs/live_backing.log - SIGABRT
https://public-artifacts.taskcluster.net/cpA4yqOrQ9ejNXvbsSjV4w/0/public/logs/live_backing.log - SIGABRT
https://public-artifacts.taskcluster.net/Fhpy8OMmRlWL0BdIvaRmfQ/0/public/logs/live_backing.log - SIGBUS

[4]
+ free -mt
             total       used       free     shared    buffers     cached
Mem:          3748       1996       1752          0        215       1147
-/+ buffers/cache:        633       3115
Swap:            0          0          0
Total:        3748       1996       1752
+ df -h
Filesystem                        Size  Used Avail Use% Mounted on
rootfs                            394G   11G  364G   3% /
none                              394G   11G  364G   3% /
tmpfs                             1.9G     0  1.9G   0% /dev
shm                                64M   68K   64M   1% /dev/shm
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/resolv.conf
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/hostname
/dev/mapper/instance_storage-all  394G   11G  364G   3% /etc/hosts
tmpfs                             1.9G     0  1.9G   0% /proc/kcore
tmpfs                             1.9G     0  1.9G   0% /proc/latency_stats
tmpfs                             1.9G     0  1.9G   0% /proc/timer_stats
Flags: needinfo?(dustin)
Yes, ps output would be good.  I'm not sure exactly what those statistics are showing, since in [4] there are two different numbers listed for cache and buffers.  The kernel tries to make memory useful, caching disk blocks and other potentially useful information, and I don't know the details of how it represents that here.  At any rate, even if firefox-bin used all 3.75G of RAM, once that process exited, you wouldn't see anything in the output.  So I'm not sure this is a good way to chase down this particular issue.

In particular, running out of memory generally ends in SIGTERM/SIGKILL, not SIGBUS or SIGABRT.  To verify, I cobbled together

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

void main() {
    int pid;
    int status;

    if ((pid = fork()) == 0) {
        while (1) {
            malloc(1024*1024);
        }
    } else {
        wait(&status);
        printf("exit status %d\n", status);
        printf("WIFSIGNALED %d\n", WIFSIGNALED(status));
        printf("WEXITSTATUS %d\n", WEXITSTATUS(status));
        printf("WTERMSIG %d\n", WTERMSIG(status));
    }
}

and sure enough

dustin@dustin-moz-devel ~/p/m-c $ ./tmp
exit status 9
WIFSIGNALED 1
WEXITSTATUS 0
WTERMSIG 9

where 9 is SIGKILL.

SIGBUS, as Jeff pointed out in comment 15, is more often associated with misuse of memory -- usually unaligned access (as an invalid address would end in SIGSEGV).  That the failure was in sse2_blt in comment 15 certainly points to that, too!  And alignment would generally be intermittent, probably with some binary fractional probability (1/2 of runs fail, 1/4 fail, 1/8 fail, etc.)  Basically, I think Jeff is going in the right direction here.

I wonder if a good debugging technique would be to add some assertion about alignment:

diff --git a/gfx/cairo/libpixman/src/pixman-sse2.c b/gfx/cairo/libpixman/src/pixman-sse2.c
--- a/gfx/cairo/libpixman/src/pixman-sse2.c
+++ b/gfx/cairo/libpixman/src/pixman-sse2.c
@@ -4724,16 +4724,18 @@ sse2_blt (pixman_implementation_t *imp,
        src_stride *= 4;
        dst_stride *= 4;
     }
     else
     {
        return FALSE;
     }
 
+    assert(dst_bytes & 0x7 == 0);
+
     while (height--)
     {
        int w;
        uint8_t *s = src_bytes;
        uint8_t *d = dst_bytes;
        src_bytes += src_stride;
        dst_bytes += dst_stride;
        w = byte_width;

and then see if that assertion fails.  It's worth noting that the source uses unaligned operations, so I think only the destination needs to be checked.  I'm not sure which NS_ macros are available here, but I'm sure Jeff knows :)
It seems that ulimit -a is what we really wanted.
Releng host [1]
Docker instance [2]
For giggles, running docker on my local machine gives different values [3]

Did you want ulimit -a to be run *after* the test jobs? or is it fine before it?

The clear differences I see are pending signals, open files and max user processes
> pending signals                 (-i) 29669
> open files                      (-n) 1024
> max user processes              (-u) 29669
vs
> pending signals                 (-i) 29509
> open files                      (-n) 524288
> max user processes              (-u) 1048576

Is this information relevant?

[1]
[cltbld@tst-linux64-ec2-armenzg.test.releng.use1.mozilla.com ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 29669
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 29669
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

[2]
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e27e161c88ed&selectedJob=16209769
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 29509
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 524288
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1048576
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

[3]
armenzg@armenzg-thinkpad:~/repos/mozilla-central/testing/taskcluster/tasks/tests$ docker run -ti taskcluster/desktop-test:0.5.7
root@taskcluster-worker:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 124671
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 524288
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 524288
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
It looks like this probably caused by the size of /dev/shm being too small. In docker it defaults to 64MB but we likely need more.

See also:
https://github.com/jvermillard/docker/commit/77faf17586cecacf79765d47983363962827cb45
Flags: needinfo?(jmuizelaar)
Bug 1245243 cleared this bug. \o/
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.