Closed Bug 973696 Opened 10 years ago Closed 10 years ago

Crash when stopping a WebRTC video stream in Firefox OS nightly - mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange(mozilla::CameraControlListener::HardwareState)

Categories

(Core :: WebRTC: Audio/Video, defect)

ARM
Gonk (Firefox OS)
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla30
blocking-b2g 1.4+
Tracking Status
firefox28 --- wontfix
firefox29 --- wontfix
firefox30 --- fixed
b2g-v1.4 --- fixed

People

(Reporter: sole, Assigned: mikeh)

References

Details

(Keywords: crash, regression, reproducible, Whiteboard: [b2g-crash])

Crash Data

Attachments

(1 file, 2 obsolete files)

So Paul just flashed Firefox OS on my Nexus 4 and I tried one of my test cases using the Browser app:
http://people.mozilla.org/~spenades/test_cases/gumhelper/demo/

It starts streaming nicely but when I click on the "stop streaming" button (please ignore the lack of device viewport support for now) all I get is a nice crash!

The logcat output shows some SIGSEGV terrifying stuff, the dump, etc. I'm attaching it too.
Can you include a crash report URL?

See https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks#Getting_crashes_off_the_Device for how to d this.
Component: Gaia → WebRTC: Audio/Video
Flags: needinfo?(sole)
Product: Firefox OS → Core
Version: unspecified → Trunk
Hi Jason

I tried to follow these instructions but I couldn't get any result out of them. Where do I "connect to localhost:6000"? Using the app manager I can only connect to the connected device. I tried "Connect to..." from the Tools -> Web Developer -> Connect to... menu, but I don't see where should I press "Update". Is that a button in the crashes app? The crashes app is just a white screen that says "Crashes".

PS Why are we using this weird way of getting data out of the phone? Isn't there a better way which is easier to setup or something? Should I file a bug for this?
Flags: needinfo?(sole)
I can reproduce. I'm adding the crash info. It's device independent as well.

https://crash-stats.mozilla.com/report/index/e1c0116a-eb7f-4872-b5a4-2f25f2140218

Frame 	Module 	Signature 	Source
0 	libxul.so 	mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange(mozilla::CameraControlListener::HardwareState) 	content/media/webrtc/MediaEngineWebRTCVideo.cpp
1 	libxul.so 	mozilla::CameraControlImpl::OnHardwareStateChange(mozilla::CameraControlListener::HardwareState) 	dom/camera/CameraControlImpl.cpp
2 	libxul.so 	mozilla::nsGonkCameraControl::ReleaseHardwareImpl() 	dom/camera/GonkCameraControl.cpp
3 	libxul.so 	libxul.so@0x9bcc1f 	
4 	libxul.so 	mozilla::CameraControlImpl::ControlMessage::Run() 	dom/camera/CameraControlImpl.cpp
5 	libxul.so 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
6 	libxul.so 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
7 	libxul.so 	mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
8 	libxul.so 	MessageLoop::RunInternal() 	ipc/chromium/src/base/message_loop.cc
9 	libxul.so 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
10 	libxul.so 	nsThread::ThreadFunc(void*) 	xpcom/threads/nsThread.cpp
11 	libnss3.so 	_pt_root 	nsprpub/pr/src/pthreads/ptthread.c
12 	libc.so 	__thread_entry 	bionic/libc/bionic/pthread.c
13 	libc.so 	pthread_create 	bionic/libc/bionic/pthread.c
blocking-b2g: --- → 1.4?
Crash Signature: [@ mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange(mozilla::CameraControlListener::HardwareState)]
Keywords: crash, reproducible
Summary: Crash when stopping a WebRTC video stream in Firefox OS nightly on a Nexus 4 → Crash when stopping a WebRTC video stream in Firefox OS nightly - mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange(mozilla::CameraControlListener::HardwareState)
Severity: normal → critical
Whiteboard: [b2g-crash]
This blocks a basic use case of gUM, so this is a FC blocker for that feature.
Blocks: 923361
Hi Maire,

May I know if you have resource to take a look this crash issue? I agree with Jason this crash bug should be plused for v1.4 as the blocker bug for gUM release. We should fix this one before gUM feature complete
Flags: needinfo?(mreavy)
Mike -- Can you look at this since it appears to involve the Gonk camera code?
Flags: needinfo?(mreavy) → needinfo?(mhabicher)
I'll take a look.
Assignee: nobody → mhabicher
Status: NEW → UNCONFIRMED
Ever confirmed: false
Flags: needinfo?(mhabicher)
Status: UNCONFIRMED → NEW
Ever confirmed: true
Is there something I need to do to access the camera through gUM? When I load that page, I see a "Don't Share / Share" prompt, and if I tap "Share", it says it timed out trying to get the camera. (This is using a Helix.)
Flags: needinfo?(mreavy)
Depending on the order in which I do things, I also see "getUserMedia cannot access the camera", and logcat shows:

02-18 18:23:46.529 18201 18201 E GeckoConsole: Content JS WARN at app://system.gaiamobile.org/shared/js/l10n.js:79 in consoleWarn: [l10n] #perm-video-capture-more-info is undefined.
02-18 18:23:48.719 18201 18201 I GeckoDump: XXX FIXME : Got a mozContentEvent: permission-allow
I see this crash sometimes when clicking on "start streaming" after the error prompts in comment 8 and comment 9:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 24813.25042]
0x40f3126c in ns_if_addref<mozilla::MediaEngineWebRTCVideoSource*> (this=0x43c28ca0, aVSources=0x4455eda0) at ../../../dist/include/nsISupportsUtils.h:59
59	        expr->AddRef();
(gdb) bt
#0  0x40f3126c in ns_if_addref<mozilla::MediaEngineWebRTCVideoSource*> (this=0x43c28ca0, aVSources=0x4455eda0) at ../../../dist/include/nsISupportsUtils.h:59
#1  nsRefPtrHashtable<nsStringHashKey, mozilla::MediaEngineWebRTCVideoSource>::Get (this=0x43c28ca0, aVSources=0x4455eda0) at ../../../dist/include/nsRefPtrHashtable.h:91
#2  mozilla::MediaEngineWebRTC::EnumerateVideoDevices (this=0x43c28ca0, aVSources=0x4455eda0) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/content/media/webrtc/MediaEngineWebRTC.cpp:99
#3  0x40d58538 in GetSources<mozilla::MediaEngineVideoSource> (engine=<value optimized out>, aConstraints=..., aEnumerate=&virtual table offset 8, media_device_name=0x0) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/media/MediaManager.cpp:675
#4  0x40d58706 in mozilla::GetUserMediaDevicesRunnable::Run (this=0x44038e00) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/media/MediaManager.cpp:1072
#5  0x406db9ec in nsThread::ProcessNextEvent (this=0x44a0b400, mayWait=false, result=0x4455ee5f) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:643
#6  0x406aba84 in NS_ProcessNextEvent (thread=0x40397c40, mayWait=false) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/glue/nsThreadUtils.cpp:263
#7  0x40813910 in mozilla::ipc::MessagePumpForNonMainThreads::Run (this=0x43cbd8b0, aDelegate=0x43623280) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/glue/MessagePump.cpp:303
#8  0x408096e8 in MessageLoop::RunInternal (this=0x1000000) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:226
#9  0x40809766 in MessageLoop::RunHandler (this=0x43623280) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:219
#10 MessageLoop::Run (this=0x43623280) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:193
#11 0x406dbf88 in nsThread::ThreadFunc (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:258
#12 0x420fe46c in _pt_root (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/nsprpub/pr/src/pthreads/ptthread.c:212
#13 0x4001a314 in sysconf (name=1151382528) at bionic/libc/unistd/sysconf.c:77
Cannot access memory at address 0x4455f01c
I've tried adding some printf()s in the MediaEngineWebRTCVideoSource ctors and dtors, to track their lifetimes (jesup suggested that the hashtable Get() may be returning a pointer to a dead object) and with those changes, the crash in comment 10 goes away.

I still can't access the camera through gUM() and the GeckoConsole warning in comment 9 remains.
I am finally able to reproduce this issue and have confirmed the STR. gdb bt:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 797.1186]
0x40f3777c in mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange (this=0x44343740, aState=mozilla::CameraControlListener::kHardwareClosed) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/content/media/webrtc/MediaEngineWebRTCVideo.cpp:542
542	    mCameraControl->RemoveListener(this);
(gdb) bt
#0  0x40f3777c in mozilla::MediaEngineWebRTCVideoSource::OnHardwareStateChange (this=0x44343740, aState=mozilla::CameraControlListener::kHardwareClosed) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/content/media/webrtc/MediaEngineWebRTCVideo.cpp:542
#1  0x40dc4526 in mozilla::CameraControlImpl::OnHardwareStateChange (this=0x436237e0, aNewState=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:91
#2  0x40dc876c in mozilla::nsGonkCameraControl::ReleaseHardwareImpl (this=0x436237e0) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/GonkCameraControl.cpp:1362
#3  0x40dc3ba8 in RunImpl (this=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:491
#4  0x40dc4312 in mozilla::CameraControlImpl::ControlMessage::Run (this=0x0) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:281
#5  0x406dfec4 in nsThread::ProcessNextEvent (this=0x40397430, mayWait=false, result=0x469ffe5f) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:643
#6  0x406afffc in NS_ProcessNextEvent (thread=0x0, mayWait=false) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/glue/nsThreadUtils.cpp:263
#7  0x40817b10 in mozilla::ipc::MessagePumpForNonMainThreads::Run (this=0x467ba1c0, aDelegate=0x44151240) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/glue/MessagePump.cpp:303
#8  0x4080d8e8 in MessageLoop::RunInternal (this=0x1000000) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:226
#9  0x4080d966 in MessageLoop::RunHandler (this=0x44151240) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:219
#10 MessageLoop::Run (this=0x44151240) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:193
#11 0x406e0460 in nsThread::ThreadFunc (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:258
#12 0x4217846c in _pt_root (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/nsprpub/pr/src/pthreads/ptthread.c:212
#13 0x40024314 in sysconf (name=1077507120) at bionic/libc/unistd/sysconf.c:77
Cannot access memory at address 0x46a0001c
(gdb) p mCameraControl
$1 = {mRawPtr = 0x0}
(gdb)
OS: Mac OS X → Gonk (Firefox OS)
Hardware: x86 → ARM
Highlights:

  8184:
02-19 12:49:39.719  3050  3079 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=0 --> aDeviceName='back'
02-19 12:49:39.719  3050  3079 I Gecko   : mikeh: mozilla::MediaEngineWebRTCVideoSource::MediaEngineWebRTCVideoSource(int):103 : this=0x4377ee80, mCaptureIndex=0
02-19 12:49:39.719  2364  2364 E QualcommCamera: Qint android::get_camera_info(int, camera_info*): E
02-19 12:49:39.719  3050  3079 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=0 --> aDeviceName='back'
02-19 12:49:39.719  2364  2364 E QualcommCamera: Qint android::get_camera_info(int, camera_info*): E
02-19 12:49:39.719  3050  3079 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=1 --> aDeviceName='front'
02-19 12:49:39.719  3050  3079 I Gecko   : mikeh: mozilla::MediaEngineWebRTCVideoSource::MediaEngineWebRTCVideoSource(int):103 : this=0x4377ff60, mCaptureIndex=1
02-19 12:49:39.729  2364  2364 E QualcommCamera: Qint android::get_camera_info(int, camera_info*): E
02-19 12:49:39.729  2382  2412 E msm7627a.hwcomposer: drawLayerUsingCopybit: invalid handle
02-19 12:49:39.729  3050  3079 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=1 --> aDeviceName='front'

  11355:
02-19 12:50:26.489  3050  3093 I Gecko   : mikeh: virtual mozilla::MediaEngineWebRTCVideoSource::~MediaEngineWebRTCVideoSource():136 : this=0x4377ff60

  11421:
02-19 12:50:32.949  2364  3110 E QualcommCamera: Qint android::get_camera_info(int, camera_info*): E
02-19 12:50:32.949  3050  3233 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=0 --> aDeviceName='back'
02-19 12:50:32.949  2364  3109 E QualcommCamera: Qint android::get_camera_info(int, camera_info*): E
02-19 12:50:32.949  3050  3233 I Gecko   : mikeh: static nsresult mozilla::ICameraControl::GetCameraName(uint32_t, nsCString&):67 : aDeviceNum=1 --> aDeviceName='front'
Attachment #8378408 - Flags: feedback?(rjesup)
Backtrace causing deletion:

Breakpoint 1, ~MediaEngineWebRTCVideoSource (this=0x43e15880, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/content/media/webrtc/MediaEngineWebRTC.h:136
136	  ~MediaEngineWebRTCVideoSource() { printf_stderr("mikeh: %s:%d : this=%p\n", __func__, __LINE__, this); Shutdown(); }
(gdb) bt
#0  ~MediaEngineWebRTCVideoSource (this=0x43e15880, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/content/media/webrtc/MediaEngineWebRTC.h:136
#1  0x408b61b8 in mozilla::dom::workers::WorkerTask::Release (this=0x43e15894) at ../../dist/include/mozilla/dom/workers/Workers.h:195
#2  0x40dc3fea in ~nsRefPtr (this=0x438227ec, start=0, count=1) at ../../dist/include/nsAutoPtr.h:900
#3  nsTArrayElementTraits<nsRefPtr<mozilla::CameraControlListener> >::Destruct (this=0x438227ec, start=0, count=1) at ../../dist/include/nsTArray.h:536
#4  nsTArray_Impl<nsRefPtr<mozilla::CameraControlListener>, nsTArrayInfallibleAllocator>::DestructRange (this=0x438227ec, start=0, count=1) at ../../dist/include/nsTArray.h:1585
#5  nsTArray_Impl<nsRefPtr<mozilla::CameraControlListener>, nsTArrayInfallibleAllocator>::RemoveElementsAt (this=0x438227ec, start=0, count=1) at ../../dist/include/nsTArray.h:1302
#6  0x40dc4066 in nsTArray_Impl<nsRefPtr<mozilla::CameraControlListener>, nsTArrayInfallibleAllocator>::Clear (this=0x43e15880, __in_chrg=<value optimized out>) at ../../dist/include/nsTArray.h:1313
#7  ~nsTArray_Impl (this=0x43e15880, __in_chrg=<value optimized out>) at ../../dist/include/nsTArray.h:764
#8  ~nsTArray (this=0x43e15880, __in_chrg=<value optimized out>) at ../../dist/include/nsTArray.h:1659
#9  0x40dc4560 in ~CameraControlImpl (this=0x438227e0, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:50
#10 0x40dc944c in ~nsGonkCameraControl (this=0x438227e0, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/GonkCameraControl.cpp:216
#11 0x40dc945c in ~nsGonkCameraControl (this=0x43e15880, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/GonkCameraControl.cpp:216
#12 0x40dc3bf0 in mozilla::ICameraControl::Release (this=0x444ea44c, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/ICameraControl.h:77
#13 ~nsRefPtr (this=0x444ea44c, __in_chrg=<value optimized out>) at ../../dist/include/nsAutoPtr.h:900
#14 0x40dc3c4a in ~ControlMessage (this=0x444ea440, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:271
#15 0x40dc3c5c in ~ControlMessage (this=0x43e15880, __in_chrg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/dom/camera/CameraControlImpl.cpp:271
#16 0x406aff9e in nsRunnable::Release (this=0x444ea440) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/glue/nsThreadUtils.cpp:32
#17 0x406a7dbe in ~nsCOMPtr_base (this=0x46bffe30, __in_chrg=<value optimized out>) at ../../../dist/include/nsCOMPtr.h:433
#18 ~nsCOMPtr (this=0x46bffe30, __in_chrg=<value optimized out>) at ../../../dist/include/nsCOMPtr.h:476
#19 0x406dfed8 in ~nsCOMPtr (this=0x446dd890, mayWait=true, result=0x46bffe5f) at ../../dist/include/nsCOMPtr.h:476
#20 nsThread::ProcessNextEvent (this=0x446dd890, mayWait=true, result=0x46bffe5f) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:648
#21 0x406afffc in NS_ProcessNextEvent (thread=0x43e15880, mayWait=true) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/glue/nsThreadUtils.cpp:263
#22 0x40817b56 in mozilla::ipc::MessagePumpForNonMainThreads::Run (this=0x438c3d30, aDelegate=0x44464400) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/glue/MessagePump.cpp:332
#23 0x4080d8e8 in MessageLoop::RunInternal (this=0x1000001) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:226
#24 0x4080d966 in MessageLoop::RunHandler (this=0x44464400) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:219
#25 MessageLoop::Run (this=0x44464400) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/ipc/chromium/src/base/message_loop.cc:193
#26 0x406e0460 in nsThread::ThreadFunc (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/xpcom/threads/nsThread.cpp:258
#27 0x4217846c in _pt_root (arg=<value optimized out>) at /home/mikeh/dev/mozilla/m-c/b2g-inbound/nsprpub/pr/src/pthreads/ptthread.c:212
#28 0x4010c314 in sysconf (name=1148049552) at bionic/libc/unistd/sysconf.c:77
#29 0x00000000 in ?? ()
Okay, I believe I understand what's going on--bug 909542 introduced the CameraControlListener class[1] that is a threadsafe ref-counted class using NS_INLINE_DECL_THREADSAFE_REFCOUNTING. Unfortunately, the  MediaEngineWebRTCVideoSource class[2] is also ref-counted, implemented by NS_DECL_THREADSAFE_ISUPPORTS.

In other words, MEWRTCVideoSource has two independent reference counts.

1. http://dxr.mozilla.org/mozilla-central/source/dom/camera/CameraControlListener.h
2. http://dxr.mozilla.org/mozilla-central/source/content/media/webrtc/MediaEngineWebRTC.h#157
Flags: needinfo?(mreavy)
That sounds like it would be the problem!  Good find.
Attachment #8378408 - Flags: feedback?(rjesup)
try-server push: https://tbpl.mozilla.org/?tree=Try&rev=b0b86f12f827
Attachment #8377265 - Attachment is obsolete: true
Attachment #8378408 - Attachment is obsolete: true
Attachment #8378529 - Flags: review?(rjesup)
Attachment #8378529 - Flags: feedback?(sole)
Status: NEW → ASSIGNED
Depends on: 909542
Comment on attachment 8378529 [details] [diff] [review]
Fix CameraControl lifetime regressions in WebRTC, v1

Review of attachment 8378529 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.  Thanks for adding the comments too
Attachment #8378529 - Flags: review?(rjesup) → review+
https://hg.mozilla.org/mozilla-central/rev/01fa011daae6
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla30
blocking-b2g: 1.4? → 1.4+
Comment on attachment 8378529 [details] [diff] [review]
Fix CameraControl lifetime regressions in WebRTC, v1

Removing my request for feedback because FIXED and because I don't know how to update my nightly build yet O:-) Thanks!
Attachment #8378529 - Flags: feedback?(sole)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: