Closed Bug 1148311 Opened 10 years ago Closed 10 years ago

[Bluetooth]bluedroid crash after receiving event AVRC_EVT_APP_SETTING_CHANGE

Categories

(Firefox OS Graveyard :: Bluetooth, defect)

Product:
Firefox OS Graveyard
Component:
Bluetooth
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(blocking-b2g:2.2+, firefox38 wontfix, firefox39 wontfix, firefox40 fixed, b2g-v2.2 fixed, b2g-master fixed)

Status:
RESOLVED FIXED
Milestone:
2.2 S9 (3apr)
Project Flags:
blocking-b2g 2.2+
Tracking Flags:
Tracking Status
firefox38 --- wontfix
firefox39 --- wontfix
firefox40 --- fixed
b2g-v2.2 --- fixed
b2g-master --- fixed

People

(Reporter: twen, Assigned: shawnjohnjr)

References

Details

(Keywords: crash, Whiteboard: [b2g-crash])

Crash Data

Attachments

(4 files)

logcat
398.02 KB, text/x-vhdl
Details
crash report
344.31 KB, application/octet-stream
Details
backtrace.txt
218.14 KB, text/plain
Details
Bug 1148311 - Handle AVRCP app settings notification response
2.79 KB, patch
tzimmermann
: review+
bajaj
: approval-mozilla-b2g37+
Details | Diff | Splinter Review
Attached file logcat —
[Description]: When pairing and connecting to device, flame shows successful connection but the BT device lost connection and shows connection failed. Then, flame crashed. [Testing Steps]: 1. Go to Settings -> turn on Bluetooth 2. Tap search for devices 3. Tap to connect car kit device 4. After the device shows connected to phone/media audio -> go to homescreen 5. Tap on Music app 6. Play music [Expected Result]: Music can be played on BT device and correct music information shown on screen. [Actual Result]: Connection lost, no music or information on BT device and flame crashed. Attached adb logcat file, hope it can be useful.
Only failed on master. Nominate blocking for 3.0.
blocking-b2g: --- → 3.0?
(In reply to Teri Wen [:twen] from comment #1) > Only failed on master. Nominate blocking for 3.0. Is it 100% reproducible? Can you provide minidump?
Flags: needinfo?(twen)
Attached file crash report —
Attached crash report. Let me know if you need more info. Build info: Gaia-Rev 525c341254e08f07f90da57a4d1cd5971a3cc668 Gecko-Rev https://hg.mozilla.org/mozilla-central/rev/59554288b4eb Build-ID 20150326160206 Version 39.0a1 Device-Name flame FW-Release 4.4.2 FW-Incremental eng.cltbld.20150326.193247 FW-Date Thu Mar 26 19:32:58 EDT 2015 Bootloader L1TC000118D0
Flags: needinfo?(twen)
Connection lost is 100% reproducible, crash may or may not happen.
Crash reason: SIGSEGV Crash address: 0x1e Thread 0 (crashed) 0 libxul.so!mozilla::dom::bluetooth::BluetoothProfileController::NotifyCompletion(nsAString_internal const&) [nsTArray.h : 942 + 0x0] r4 = 0x00000000 r5 = 0xaffef200 r6 = 0x00000000 r7 = 0xb27ad640 r8 = 0xabf93ee0 r9 = 0xae2a638c r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8e80 lr = 0xb56e3f35 pc = 0xb56e0a6e Found by: given as instruction pointer in context 1 libxul.so!mozilla::dom::bluetooth::BluetoothA2dpManager::Disconnect(mozilla::dom::bluetooth::BluetoothProfileController*) [BluetoothA2dpManager.cpp:59554288b4eb : 564 + 0x3] r4 = 0x00000000 r5 = 0xaffef200 r6 = 0x00000000 r7 = 0xb27ad640 r8 = 0xabf93ee0 r9 = 0xae2a638c r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8f00 pc = 0xb56e3f35 Found by: call frame info 2 libxul.so!mozilla::dom::bluetooth::BluetoothService::StopBluetooth(bool) [BluetoothService.cpp:59554288b4eb : 429 + 0x7] r4 = 0x00000000 r5 = 0xaffef200 r6 = 0xaf806f24 r7 = 0xb27ad640 r8 = 0xabf93ee0 r9 = 0xae2a638c r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8f20 pc = 0xb56e1f11 Found by: call frame info 3 libxul.so!mozilla::dom::bluetooth::BluetoothService::HandleSettingsChanged(nsISupports*) [BluetoothService.cpp:59554288b4eb : 577 + 0x5] r4 = 0x00000000 r5 = 0xb6791720 r6 = 0xbe8c8f54 r7 = 0xb27ad640 r8 = 0xabf93ee0 r9 = 0xae2a638c r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8f40 pc = 0xb56e20e9 Found by: call frame info 4 libxul.so!nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) [nsObserverList.cpp:59554288b4eb : 100 + 0x7] r4 = 0xbe8c8f84 r5 = 0x0000000f r6 = 0xb56e2129 r7 = 0xae25e510 r8 = 0xabf93ee0 r9 = 0xae2a638c r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8f80 pc = 0xb4c6d08f Found by: call frame info 5 libxul.so!nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) [nsObserverService.cpp:59554288b4eb : 329 + 0x9] r4 = 0xabf93ee0 r5 = 0xae2a638c r6 = 0xae25e510 r7 = 0xb6a6fcb0 r8 = 0x00000003 r9 = 0xbe8c9138 r10 = 0x00000091 fp = 0xb6791720 sp = 0xbe8c8fa8 pc = 0xb4c6d0ef
Hi Teri, From the backtrace, it shows the crash happened after you turn off bluetooth. Did you try to turn off bluetooth at that time?
Flags: needinfo?(twen)
Ok, bluedroid got stack corruption. D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_RC_OPEN_EVT flags 0 W/bt-btif ( 2377): blacklist absolute volume for 90:03:b7 D/EventHub( 209): No input device configuration file found for device 'AVRCP'. W/EventHub( 209): Unable to disable kernel key repeat for /dev/input/event9: Function not implemented I/EventHub( 209): New device: id=10, fd=162, path='/dev/input/event9', name='AVRCP', classes=0x80000001, configuration='', keyLayout='/system/usr/keylayout/AVRCP.kl', keyCharacterMap='/system/usr/keychars/Generic.kcm', builtinKeyboard=false, usingSuspendBlockIoctl=true, usingClockIoctl=true I/InputReader( 209): Device added: id=10, name='AVRCP', sources=0x00000101 E/bt-avp ( 2377): opcode 0 E/bt-avp ( 2377): opcode 0 D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0 E/bt-avp ( 2377): opcode 0 D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0 E/bt-avp ( 2377): opcode 0 D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0 E/bt-avp ( 2377): opcode 0 D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0 E/bt-avp ( 2377): avrc_is_valid_player_attrib_value() found not matching attrib(x6d)-value(xa6) pair! E/bt-avp ( 2377): bad player app seeting attribute or value E/bt-btif ( 2377): send_metamsg_rsp: failed to build metamsg response. status: 0x01 F/libc ( 2377): stack corruption detected F/libc ( 2377): Fatal signal 6 (SIGABRT) at 0x00000949 (code=-6), thread 2377 (bluetoothd) I/DEBUG ( 200): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** I/DEBUG ( 200): Build fingerprint: 'qcom/flame/flame:4.4.2/KOT49H/eng.cltbld.20150326.193247:eng/test-keys' I/DEBUG ( 200): Revision: '0' I/DEBUG ( 200): pid: 2377, tid: 2377, name: bluetoothd >>> /system/bin/bluetoothd <<< I/DEBUG ( 200): signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr -------- I/DEBUG ( 200): Abort message: 'stack corruption detected' I/DEBUG ( 200): r0 00000000 r1 00000949 r2 00000006 r3 00000000 I/DEBUG ( 200): r4 00000006 r5 00000002 r6 00000949 r7 0000010c I/DEBUG ( 200): r8 00000008 r9 b6fbebdb sl b6fc1858 fp becdba2c I/DEBUG ( 200): ip b6f81394 sp becdb458 lr b6f4613d pc b6f550e8 cpsr 000f0010 I/DEBUG ( 200): d0 64656c696166203a d1 6c697562206f7420 I/DEBUG ( 200): d2 736d6174656d2064 d3 6e6f707365722067 I/DEBUG ( 200): d4 3a3462223d676154 d5 66393a36373a3662 I/DEBUG ( 200): d6 3e2263323a64643a d7 202020202020200a I/DEBUG ( 200): d8 0000000000000000 d9 0000000000000000 I/DEBUG ( 200): d10 0000000000000000 d11 0000000000000000 I/DEBUG ( 200): d12 0000000000000000 d13 0000000000000000 I/DEBUG ( 200): d14 0000000000000000 d15 0000000000000000 I/DEBUG ( 200): d16 6f63206b63617473 d17 6e6f697470757272 I/DEBUG ( 200): d18 00000014b6f43fc3 d19 b6f42f2500200000 I/DEBUG ( 200): d20 0000000000000000 d21 b6ece01000000000 I/DEBUG ( 200): d22 b6fbebdb00000008 d23 becdba2cb6fc1858 I/DEBUG ( 200): d24 0000000000000000 d25 0000000000000000 I/DEBUG ( 200): d26 0000000000000000 d27 0000000000000000 I/DEBUG ( 200): d28 0000000000000000 d29 0000000000000000 I/DEBUG ( 200): d30 0000000000000000 d31 0000000000000000 I/DEBUG ( 200): scr 00000010 I/DEBUG ( 200): I/DEBUG ( 200): backtrace: I/DEBUG ( 200): #00 pc 000220e8 /system/lib/libc.so (tgkill+12) I/DEBUG ( 200): #01 pc 00013139 /system/lib/libc.so (pthread_kill+48) I/DEBUG ( 200): #02 pc 0001334d /system/lib/libc.so (raise+10) I/DEBUG ( 200): #03 pc 00012083 /system/lib/libc.so I/DEBUG ( 200): #04 pc 0002199c /system/lib/libc.so (abort+4) I/DEBUG ( 200): #05 pc 00012b69 /system/lib/libc.so I/DEBUG ( 200): #06 pc 00012053 /system/lib/libc.so (__stack_chk_fail+6) I/DEBUG ( 200): #07 pc 0000451b /system/bin/bluetoothd I/DEBUG ( 200): #08 pc b1e68644 <unknown> I/DEBUG ( 200): I/DEBUG ( 200): stack: I/DEBUG ( 200): becdb418 00000001 I/DEBUG ( 200): becdb41c b7b40660 [heap] I/DEBUG ( 200): becdb420 b6ecc05c I/DEBUG ( 200): becdb424 becdb3e4 [stack] I/DEBUG ( 200): becdb428 b6df39c8 /system/lib/hw/bluetooth.default.so I/DEBUG ( 200): becdb42c 00000000 I/DEBUG ( 200): becdb430 b6ecaf64 I/DEBUG ( 200): becdb434 3f8487c5 I/DEBUG ( 200): becdb438 484bf97d
Flags: needinfo?(twen)
sigh~~ Having checked branch caf/b2g_kk_3.5, the current bluedroid branch supports AVRCP 1.5. However, since SDP records mentioned the phone supports "player application settings" had already enabled, so the car kit is trying to fetch attributes for "player application settings". Since Gecko is not aware of this feature had been enabled. Following the log, I could see Gecko did not call any Player application settings api and it causes the value of player settings attributes not initialized, and stack corruption detected. But even bluedroid crashes, b2g should be alive anyway. Based on Comment 5, it looks like we shall check sBluetoothHfpInterface/sBluetoothA2dpInterface are not null. Two actions need to take, 1) Find feasible way to disable player setting in stack 2) Fix b2g crash even bluetoothd crashes when disabling bluetooth. commit 5626eaf8d2997a0e3e20ad49705910eda91b22e4 Author: Rohit Singh <singhr@codeaurora.org> Date: Thu Jul 18 19:05:24 2013 +0530 Bluetooth: Changes to support AVRCP 1.5 feature Changes in stack for AVRCP 1.5. - SDP and capability changes. - Changes done to Register Callback functions with L2CAP layer for Browsing channel on a particular PSM. L2CAP through callback sends indication to AVCT layer of connection, disconnection, data and congestion indication. - Changes made in AVCT and AVRC layer to support AVRCP browsing response - Changes made in BTA layer to handle browsing command - BTIF changes for Browsing and address player command. Added Address player and available player change notification. - During DUT initiated disconnect,changes made to disconnect L2CAP Browsing channel. - Seprate Queue added in browsing channel to handle L2CAP congestion. Change-Id: I54a052349f672141f24f6e4d5db0f6f929a9c66d (cherry picked from commit 86086a18f03a0bd7752ee1db6aa0fd8200087bc2) (cherry picked from commit 68dc0d817e1d2bab37b64fdf398539bf78ddf5d5) (cherry picked from commit ec20b2c8b2ac90a8edf39ad3e82012f61aadfc35) commit 394688b4760e85c8cc96616fc8e5d7f21584f6db Author: Ayan Ghosh <abghosh@codeaurora.org> Date: Thu Aug 29 12:34:55 2013 +0530 libHardware changes to support Avrcp1.5 Provide support for Avrcp1.5 so that remote CT is able to browse the list of available media players on DUT and select the media player from the available list of players, from remote. Change-Id: Ib804efc3a42d14f302d0d1eec7ab34c9f13b5c7d (cherry picked from commit b665c0ce1ff6f8561f1147d2525985319e64f59a) (cherry picked from commit bd87a5d8c09a8c7e92ba3910a62c482f167b04fe) (cherry picked from commit c549c3190adeeadaea5af0cb9d816c16594f3f31)
(In reply to Shawn Huang [:shawnjohnjr] from comment #9) > Two actions need to take, 1) Find feasible way to disable player setting in > stack 2) Fix b2g crash even bluetoothd crashes when disabling bluetooth. Application settings is part of AVRCP 1.3. And even Android 5.1 is still missing this implementation. See: http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/btif_rc.c#1064 branch b2g_kk_3.5: https://www.codeaurora.org/cgit/quic/la/platform/external/bluetooth/bluedroid/tree/btif/src/btif_rc.c?h=b2g_kk_3.5&id=5626eaf8d2997a0e3e20ad49705910eda91b22e4#n1181 So I guess I cannot easily find ways to ignore player application settings now. :( It seems that the missing AVRCP 1.3 player application settings feature committed together with AVRCP 1.5...
Summary: [Bluetooth]Can't connect to car kit device → [Bluetooth]bluedroid crash after receiving event AVRC_EVT_APP_SETTING_CHANGE
I enabled full log for bluedroid, this just happened after avrcp connected, and car kit sent PDU to register player settings notification. 03-27 20:30:50.870 D/bt-btif ( 1568): Received vendor command.code,PDU and label: 3, 49,5 03-27 20:30:50.870 I/bt-btif ( 1568): handle_rc_metamsg_cmd:New register notification received.event_id:AVRC_EVT_APP_SETTING_CHANGE,label:0x5,code:3 03-27 20:30:50.870 I/bt-btif ( 1568): handle_rc_metamsg_cmd: Passing received metamsg command to app. pdu: AVRC_PDU_REGISTER_NOTIFICATION 03-27 20:30:50.870 I/bt-btif ( 1568): btif_rc_upstreams_evt pdu: AVRC_PDU_REGISTER_NOTIFICATION handle: 0x1 ctype:3 label:5 03-27 20:30:50.870 I/bt-btif ( 1568): HAL bt_rc_callbacks->register_notification_cb 03-27 20:30:50.870 D/bt-btif ( 1568): ## register_notification_rsp ## 03-27 20:30:50.870 I/bt-btif ( 1568): ## register_notification_rsp ## event_id:AVRC_EVT_APP_SETTING_CHANGE 03-27 20:30:50.870 I/bt-btif ( 1568): +send_metamsg_rsp: rc_handle: 1, label: 5, code: 0x03, pdu: AVRC_PDU_REGISTER_NOTIFICATION 03-27 20:30:50.870 E/bt-avp ( 1568): avrc_is_valid_player_attrib_value() found not matching attrib(x0)-value(xa) pair! 03-27 20:30:50.870 E/bt-avp ( 1568): bad player app seeting attribute or value 03-27 20:30:50.870 E/bt-btif ( 1568): send_metamsg_rsp: failed to build metamsg response. status: 0x01 03-27 20:30:50.870 F/libc ( 1568): stack corruption detected 03-27 20:30:50.870 F/libc ( 1568): Fatal signal 6 (SIGABRT) at 0x00000620 (code=-6), thread 1568 (bluetoothd)
hmm... I just found bluetoothd tries to call |register_notification_rsp| with event_id:AVRC_EVT_APP_SETTING_CHANGE.
Ok, I see, this looks like a case I never expect it would happen :( #0 mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp (this=0xae0c6e70, aEvent=mozilla::dom::bluetooth::AVRCP_EVENT_APP_SETTINGS_CHANGED, aType=mozilla::dom::bluetooth::AVRCP_NTF_INTERIM, aParam=..., aRes=0x0) at ../../../../../../../code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/bluedroid/BluetoothDaemonAvrcpInterface.cpp:1067 #1 0xb574a144 in mozilla::dom::bluetooth::BluetoothA2dpManager::UpdateRegisterNotification (this=<optimized out>, aEvent=<optimized out>, aParam=<optimized out>) at /code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/bluedroid/BluetoothA2dpManager.cpp:930 #2 0xb574c7c6 in mozilla::dom::bluetooth::BluetoothNotificationRunnable2<mozilla::dom::bluetooth::BluetoothDaemonAvrcpModule::NotificationHandlerWrapper, void, int, int, int, int>::Run ( this=<optimized out>) at ../../../../../../../code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/BluetoothInterfaceHelpers.h:433
(In reply to Teri Wen [:twen] from comment #4) > Connection lost is 100% reproducible, crash may or may not happen. Clarification: It depends on STR. If you let car kit UI switch to media player page, car kit sends PLAYER_APP_SETTING_CHANGE event to ask for registration event, bluedroid crashes.
Assignee: nobody → shuang
Crash Signature: [@ mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp ]
Keywords: crash
Whiteboard: [b2g-crash]
Crash Signature: [@ mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp ] → [@ libxul.so!mozilla::dom::bluetooth::BluetoothProfileController::NotifyCompletion(nsAString_internal const&) [nsTArray.h : 942 + 0x0] ]
Correct Crash Signature, Comment 13 is just for clarification using gdb not real crash signature.
Based on Comment 9, Comment 10. We have to deal with player app settings, at least for this case, we don't want to crash bluedroid (from codearoura variant), even though AOSP bluedroid did not implement this feature, but AOSP HAL indeed define the interfaces. Equalizer ON/OFF status/Scan ON/OFF status skipped because the current music player don't support this feature: https://www.codeaurora.org/cgit/quic/la/platform/packages/apps/Bluetooth/tree/src/com/android/bluetooth/a2dp/Avrcp.java?h=kk_rb5#n3089
blocking-b2g: 3.0? → 2.2?
Change blocking flag to 2.2? because incomplete RegisterNotificationRespose can cause bluedroid crash (non AOSP version from codearoura), this should also apply to 'Orion' device.
I opened follow up bug (bug 1149131) for feature implementation. But now I don't want to introduce this in v2.2. This feature is not expected and not in v2.2 road-map.
Comment on attachment 8585466 [details] [diff] [review] Bug 1148311 - Handle AVRCP app settings notification response Review of attachment 8585466 [details] [diff] [review]: ----------------------------------------------------------------- The code looks good. As far as I understand the comments, the bug happens on CAF code, but not AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least contains the required interfaces. Right? Does the fix work on AOSP as well?
Attachment #8585466 - Flags: review?(tzimmermann) → review+
(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #20) > Comment on attachment 8585466 [details] [diff] [review] > Bug 1148311 - Handle AVRCP app settings notification response > > Review of attachment 8585466 [details] [diff] [review]: > ----------------------------------------------------------------- > > The code looks good. > > As far as I understand the comments, the bug happens on CAF code, but not > AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least > contains the required interfaces. Right? Does the fix work on AOSP as well? http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/btif_rc.c#1067 When bluedroid received app_settings related notification registration from the remote device (car kit), it just simply immediately rejects, so I think Gecko won't have chance to get the request. But you're right, we should double confirm again. I will test on various combinations (flame-kk (CAF), Nexus-5(AOSP), Orion(CAF L)).
blocking-b2g: 2.2? → 2.2+
(In reply to Shawn Huang [:shawnjohnjr] from comment #21) > (In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #20) > > Comment on attachment 8585466 [details] [diff] [review] > > Bug 1148311 - Handle AVRCP app settings notification response > > > > Review of attachment 8585466 [details] [diff] [review]: > > ----------------------------------------------------------------- > > > > The code looks good. > > > > As far as I understand the comments, the bug happens on CAF code, but not > > AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least > > contains the required interfaces. Right? Does the fix work on AOSP as well? > > http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/ > btif_rc.c#1067 > When bluedroid received app_settings related notification registration from > the remote device (car kit), it just simply immediately rejects, so I think > Gecko won't have chance to get the request. But you're right, we should > double confirm again. I will test on various combinations (flame-kk (CAF), > Nexus-5(AOSP), Orion(CAF L)). I've tested Flame-kk/Nexus-5. As expected, Nexus-5 won't be registered app settings notification.
https://hg.mozilla.org/mozilla-central/rev/7d0fbe8f67c2
Status: NEW → RESOLVED
Closed: 10 years ago
status-firefox40: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 2.2 S9 (3apr)
Please nominate this patch for b2g37 approval when you get a chance.
status-b2g-v2.2: --- → affected
status-b2g-master: --- → fixed
status-firefox38: --- → wontfix
status-firefox39: --- → wontfix
Flags: needinfo?(shuang)
Comment on attachment 8585466 [details] [diff] [review] Bug 1148311 - Handle AVRCP app settings notification response NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings. [Approval Request Comment] Bug caused by (feature/regressing bug #): CAF bluedroid enables app settings features User impact if declined: Using CAF bluedroid, system could crash Testing completed: I tested the patch with a car kit which supports application settings using Nexus 5/flame-kk Risk to taking this patch (and alternatives if risky): This patch provides initialized application settings value to avoid bluedroid internal crash. String or UUID changes made by this patch:None
Flags: needinfo?(shuang)
Attachment #8585466 - Flags: approval-mozilla-b2g37?
Attachment #8585466 - Flags: approval-mozilla-b2g37? → approval-mozilla-b2g37+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: