Closed Bug 1148311 Opened 9 years ago Closed 9 years ago

[Bluetooth]bluedroid crash after receiving event AVRC_EVT_APP_SETTING_CHANGE

Categories

(Firefox OS Graveyard :: Bluetooth, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(blocking-b2g:2.2+, firefox38 wontfix, firefox39 wontfix, firefox40 fixed, b2g-v2.2 fixed, b2g-master fixed)

RESOLVED FIXED
2.2 S9 (3apr)
blocking-b2g 2.2+
Tracking Status
firefox38 --- wontfix
firefox39 --- wontfix
firefox40 --- fixed
b2g-v2.2 --- fixed
b2g-master --- fixed

People

(Reporter: twen, Assigned: shawnjohnjr)

References

Details

(Keywords: crash, Whiteboard: [b2g-crash])

Crash Data

Attachments

(4 files)

Attached file logcat
[Description]:
When pairing and connecting to device, flame shows successful connection but the BT device lost connection and shows connection failed. Then, flame crashed. 

[Testing Steps]: 
1. Go to Settings -> turn on Bluetooth
2. Tap search for devices
3. Tap to connect car kit device
4. After the device shows connected to phone/media audio -> go to homescreen
5. Tap on Music app
6. Play music

[Expected Result]: 
Music can be played on BT device and correct music information shown on screen.

[Actual Result]: 
Connection lost, no music or information on BT device and flame crashed. 

Attached adb logcat file, hope it can be useful.
Only failed on master. Nominate blocking for 3.0.
blocking-b2g: --- → 3.0?
(In reply to Teri Wen [:twen] from comment #1)
> Only failed on master. Nominate blocking for 3.0.

Is it 100% reproducible? Can you provide minidump?
Flags: needinfo?(twen)
Attached file crash report
Attached crash report. Let me know if you need more info.

Build info:
Gaia-Rev        525c341254e08f07f90da57a4d1cd5971a3cc668
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/59554288b4eb
Build-ID        20150326160206
Version         39.0a1
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20150326.193247
FW-Date         Thu Mar 26 19:32:58 EDT 2015
Bootloader      L1TC000118D0
Flags: needinfo?(twen)
Connection lost is 100% reproducible, crash may or may not happen.
Crash reason:  SIGSEGV
Crash address: 0x1e

Thread 0 (crashed)
 0  libxul.so!mozilla::dom::bluetooth::BluetoothProfileController::NotifyCompletion(nsAString_internal const&) [nsTArray.h : 942 + 0x0]
     r4 = 0x00000000    r5 = 0xaffef200    r6 = 0x00000000    r7 = 0xb27ad640
     r8 = 0xabf93ee0    r9 = 0xae2a638c   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8e80    lr = 0xb56e3f35    pc = 0xb56e0a6e
    Found by: given as instruction pointer in context
 1  libxul.so!mozilla::dom::bluetooth::BluetoothA2dpManager::Disconnect(mozilla::dom::bluetooth::BluetoothProfileController*) [BluetoothA2dpManager.cpp:59554288b4eb : 564 + 0x3]
     r4 = 0x00000000    r5 = 0xaffef200    r6 = 0x00000000    r7 = 0xb27ad640
     r8 = 0xabf93ee0    r9 = 0xae2a638c   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8f00    pc = 0xb56e3f35
    Found by: call frame info
 2  libxul.so!mozilla::dom::bluetooth::BluetoothService::StopBluetooth(bool) [BluetoothService.cpp:59554288b4eb : 429 + 0x7]
     r4 = 0x00000000    r5 = 0xaffef200    r6 = 0xaf806f24    r7 = 0xb27ad640
     r8 = 0xabf93ee0    r9 = 0xae2a638c   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8f20    pc = 0xb56e1f11
    Found by: call frame info
 3  libxul.so!mozilla::dom::bluetooth::BluetoothService::HandleSettingsChanged(nsISupports*) [BluetoothService.cpp:59554288b4eb : 577 + 0x5]
     r4 = 0x00000000    r5 = 0xb6791720    r6 = 0xbe8c8f54    r7 = 0xb27ad640
     r8 = 0xabf93ee0    r9 = 0xae2a638c   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8f40    pc = 0xb56e20e9
    Found by: call frame info
 4  libxul.so!nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) [nsObserverList.cpp:59554288b4eb : 100 + 0x7]
     r4 = 0xbe8c8f84    r5 = 0x0000000f    r6 = 0xb56e2129    r7 = 0xae25e510
     r8 = 0xabf93ee0    r9 = 0xae2a638c   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8f80    pc = 0xb4c6d08f
    Found by: call frame info
 5  libxul.so!nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) [nsObserverService.cpp:59554288b4eb : 329 + 0x9]
     r4 = 0xabf93ee0    r5 = 0xae2a638c    r6 = 0xae25e510    r7 = 0xb6a6fcb0
     r8 = 0x00000003    r9 = 0xbe8c9138   r10 = 0x00000091    fp = 0xb6791720
     sp = 0xbe8c8fa8    pc = 0xb4c6d0ef
Hi Teri,
From the backtrace, it shows the crash happened after you turn off bluetooth. Did you try to turn off bluetooth at that time?
Flags: needinfo?(twen)
Ok, bluedroid got stack corruption.

D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_RC_OPEN_EVT flags 0
W/bt-btif ( 2377): blacklist absolute volume for 90:03:b7
D/EventHub(  209): No input device configuration file found for device 'AVRCP'.
W/EventHub(  209): Unable to disable kernel key repeat for /dev/input/event9: Function not implemented
I/EventHub(  209): New device: id=10, fd=162, path='/dev/input/event9', name='AVRCP', classes=0x80000001, configuration='', keyLayout='/system/usr/keylayout/AVRCP.kl', keyCharacterMap='/system/usr/keychars/Generic.kcm', builtinKeyboard=false, usingSuspendBlockIoctl=true, usingClockIoctl=true
I/InputReader(  209): Device added: id=10, name='AVRCP', sources=0x00000101
E/bt-avp  ( 2377): opcode 0
E/bt-avp  ( 2377): opcode 0
D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0
E/bt-avp  ( 2377): opcode 0
D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0
E/bt-avp  ( 2377): opcode 0
D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0
E/bt-avp  ( 2377): opcode 0
D/bt-btif ( 2377): btif_av_state_opened_handler event:BTA_AV_META_MSG_EVT flags 0
E/bt-avp  ( 2377): avrc_is_valid_player_attrib_value() found not matching attrib(x6d)-value(xa6) pair!
E/bt-avp  ( 2377): bad player app seeting attribute or value
E/bt-btif ( 2377): send_metamsg_rsp: failed to build metamsg response. status: 0x01
F/libc    ( 2377): stack corruption detected
F/libc    ( 2377): Fatal signal 6 (SIGABRT) at 0x00000949 (code=-6), thread 2377 (bluetoothd)
I/DEBUG   (  200): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   (  200): Build fingerprint: 'qcom/flame/flame:4.4.2/KOT49H/eng.cltbld.20150326.193247:eng/test-keys'
I/DEBUG   (  200): Revision: '0'
I/DEBUG   (  200): pid: 2377, tid: 2377, name: bluetoothd  >>> /system/bin/bluetoothd <<<
I/DEBUG   (  200): signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
I/DEBUG   (  200): Abort message: 'stack corruption detected'
I/DEBUG   (  200):     r0 00000000  r1 00000949  r2 00000006  r3 00000000
I/DEBUG   (  200):     r4 00000006  r5 00000002  r6 00000949  r7 0000010c
I/DEBUG   (  200):     r8 00000008  r9 b6fbebdb  sl b6fc1858  fp becdba2c
I/DEBUG   (  200):     ip b6f81394  sp becdb458  lr b6f4613d  pc b6f550e8  cpsr 000f0010
I/DEBUG   (  200):     d0  64656c696166203a  d1  6c697562206f7420
I/DEBUG   (  200):     d2  736d6174656d2064  d3  6e6f707365722067
I/DEBUG   (  200):     d4  3a3462223d676154  d5  66393a36373a3662
I/DEBUG   (  200):     d6  3e2263323a64643a  d7  202020202020200a
I/DEBUG   (  200):     d8  0000000000000000  d9  0000000000000000
I/DEBUG   (  200):     d10 0000000000000000  d11 0000000000000000
I/DEBUG   (  200):     d12 0000000000000000  d13 0000000000000000
I/DEBUG   (  200):     d14 0000000000000000  d15 0000000000000000
I/DEBUG   (  200):     d16 6f63206b63617473  d17 6e6f697470757272
I/DEBUG   (  200):     d18 00000014b6f43fc3  d19 b6f42f2500200000
I/DEBUG   (  200):     d20 0000000000000000  d21 b6ece01000000000
I/DEBUG   (  200):     d22 b6fbebdb00000008  d23 becdba2cb6fc1858
I/DEBUG   (  200):     d24 0000000000000000  d25 0000000000000000
I/DEBUG   (  200):     d26 0000000000000000  d27 0000000000000000
I/DEBUG   (  200):     d28 0000000000000000  d29 0000000000000000
I/DEBUG   (  200):     d30 0000000000000000  d31 0000000000000000
I/DEBUG   (  200):     scr 00000010
I/DEBUG   (  200): 
I/DEBUG   (  200): backtrace:
I/DEBUG   (  200):     #00  pc 000220e8  /system/lib/libc.so (tgkill+12)
I/DEBUG   (  200):     #01  pc 00013139  /system/lib/libc.so (pthread_kill+48)
I/DEBUG   (  200):     #02  pc 0001334d  /system/lib/libc.so (raise+10)
I/DEBUG   (  200):     #03  pc 00012083  /system/lib/libc.so
I/DEBUG   (  200):     #04  pc 0002199c  /system/lib/libc.so (abort+4)
I/DEBUG   (  200):     #05  pc 00012b69  /system/lib/libc.so
I/DEBUG   (  200):     #06  pc 00012053  /system/lib/libc.so (__stack_chk_fail+6)
I/DEBUG   (  200):     #07  pc 0000451b  /system/bin/bluetoothd
I/DEBUG   (  200):     #08  pc b1e68644  <unknown>
I/DEBUG   (  200): 
I/DEBUG   (  200): stack:
I/DEBUG   (  200):          becdb418  00000001  
I/DEBUG   (  200):          becdb41c  b7b40660  [heap]
I/DEBUG   (  200):          becdb420  b6ecc05c  
I/DEBUG   (  200):          becdb424  becdb3e4  [stack]
I/DEBUG   (  200):          becdb428  b6df39c8  /system/lib/hw/bluetooth.default.so
I/DEBUG   (  200):          becdb42c  00000000  
I/DEBUG   (  200):          becdb430  b6ecaf64  
I/DEBUG   (  200):          becdb434  3f8487c5  
I/DEBUG   (  200):          becdb438  484bf97d
sigh~~

Having checked branch caf/b2g_kk_3.5, the current bluedroid branch supports AVRCP 1.5.
However, since SDP records mentioned the phone supports "player application settings" had already enabled, so the car kit is trying to fetch attributes for "player application settings". Since Gecko is not aware of this feature had been enabled.  Following the log, I could see Gecko did not call any Player application settings api and it causes the value of player settings attributes not initialized, and stack corruption detected.

But even bluedroid crashes, b2g should be alive anyway. Based on Comment 5, it looks like we shall check sBluetoothHfpInterface/sBluetoothA2dpInterface are not null.

Two actions need to take, 1) Find feasible way to disable player setting in stack 2) Fix b2g crash even bluetoothd crashes when disabling bluetooth. 

commit 5626eaf8d2997a0e3e20ad49705910eda91b22e4
Author: Rohit Singh <singhr@codeaurora.org>
Date:   Thu Jul 18 19:05:24 2013 +0530

    Bluetooth: Changes to support AVRCP 1.5 feature
    
    Changes in stack for AVRCP 1.5.
    - SDP and capability changes.
    - Changes done to Register Callback functions with L2CAP
      layer for Browsing channel on a particular PSM.
      L2CAP through callback sends indication to AVCT layer
      of connection, disconnection, data and congestion
      indication.
    - Changes made in AVCT and AVRC layer to support AVRCP
      browsing response
    - Changes made in BTA layer to handle browsing command
    - BTIF changes for Browsing and address player command.
      Added  Address player and available player change
      notification.
    - During DUT initiated disconnect,changes made to
      disconnect L2CAP Browsing channel.
    - Seprate Queue added in browsing channel to handle
      L2CAP congestion.
    
    Change-Id: I54a052349f672141f24f6e4d5db0f6f929a9c66d
    (cherry picked from commit 86086a18f03a0bd7752ee1db6aa0fd8200087bc2)
    (cherry picked from commit 68dc0d817e1d2bab37b64fdf398539bf78ddf5d5)
    (cherry picked from commit ec20b2c8b2ac90a8edf39ad3e82012f61aadfc35)

commit 394688b4760e85c8cc96616fc8e5d7f21584f6db
Author: Ayan Ghosh <abghosh@codeaurora.org>
Date:   Thu Aug 29 12:34:55 2013 +0530

    libHardware changes to support Avrcp1.5
    
    Provide support for Avrcp1.5 so that remote CT is able to
    browse the list of available media players on DUT and select the
    media player from the available list of players, from remote.
    
    Change-Id: Ib804efc3a42d14f302d0d1eec7ab34c9f13b5c7d
    (cherry picked from commit b665c0ce1ff6f8561f1147d2525985319e64f59a)
    (cherry picked from commit bd87a5d8c09a8c7e92ba3910a62c482f167b04fe)
    (cherry picked from commit c549c3190adeeadaea5af0cb9d816c16594f3f31)
(In reply to Shawn Huang [:shawnjohnjr] from comment #9) 
> Two actions need to take, 1) Find feasible way to disable player setting in
> stack 2) Fix b2g crash even bluetoothd crashes when disabling bluetooth. 

Application settings is part of AVRCP 1.3. And even Android 5.1 is still missing this implementation.
See: http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/btif_rc.c#1064

branch b2g_kk_3.5:
https://www.codeaurora.org/cgit/quic/la/platform/external/bluetooth/bluedroid/tree/btif/src/btif_rc.c?h=b2g_kk_3.5&id=5626eaf8d2997a0e3e20ad49705910eda91b22e4#n1181

So I guess I cannot easily find ways to ignore player application settings now. :(
It seems that the missing AVRCP 1.3 player application settings feature committed together with AVRCP 1.5...
Summary: [Bluetooth]Can't connect to car kit device → [Bluetooth]bluedroid crash after receiving event AVRC_EVT_APP_SETTING_CHANGE
I enabled full log for bluedroid, this just happened after avrcp connected, and car kit sent PDU to register player settings notification.

03-27 20:30:50.870 D/bt-btif ( 1568): Received vendor command.code,PDU and label: 3, 49,5
03-27 20:30:50.870 I/bt-btif ( 1568): handle_rc_metamsg_cmd:New register notification received.event_id:AVRC_EVT_APP_SETTING_CHANGE,label:0x5,code:3
03-27 20:30:50.870 I/bt-btif ( 1568): handle_rc_metamsg_cmd: Passing received metamsg command to app. pdu: AVRC_PDU_REGISTER_NOTIFICATION
03-27 20:30:50.870 I/bt-btif ( 1568): btif_rc_upstreams_evt pdu: AVRC_PDU_REGISTER_NOTIFICATION handle: 0x1 ctype:3 label:5
03-27 20:30:50.870 I/bt-btif ( 1568): HAL bt_rc_callbacks->register_notification_cb
03-27 20:30:50.870 D/bt-btif ( 1568): ## register_notification_rsp ##
03-27 20:30:50.870 I/bt-btif ( 1568): ## register_notification_rsp ## event_id:AVRC_EVT_APP_SETTING_CHANGE
03-27 20:30:50.870 I/bt-btif ( 1568): +send_metamsg_rsp: rc_handle: 1, label: 5, code: 0x03, pdu: AVRC_PDU_REGISTER_NOTIFICATION
03-27 20:30:50.870 E/bt-avp  ( 1568): avrc_is_valid_player_attrib_value() found not matching attrib(x0)-value(xa) pair!
03-27 20:30:50.870 E/bt-avp  ( 1568): bad player app seeting attribute or value
03-27 20:30:50.870 E/bt-btif ( 1568): send_metamsg_rsp: failed to build metamsg response. status: 0x01
03-27 20:30:50.870 F/libc    ( 1568): stack corruption detected
03-27 20:30:50.870 F/libc    ( 1568): Fatal signal 6 (SIGABRT) at 0x00000620 (code=-6), thread 1568 (bluetoothd)
hmm... I just found bluetoothd tries to call |register_notification_rsp| with event_id:AVRC_EVT_APP_SETTING_CHANGE.
Ok, I see, this looks like a case I never expect it would happen :(

#0  mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp (this=0xae0c6e70, aEvent=mozilla::dom::bluetooth::AVRCP_EVENT_APP_SETTINGS_CHANGED, 
    aType=mozilla::dom::bluetooth::AVRCP_NTF_INTERIM, aParam=..., aRes=0x0) at ../../../../../../../code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/bluedroid/BluetoothDaemonAvrcpInterface.cpp:1067
#1  0xb574a144 in mozilla::dom::bluetooth::BluetoothA2dpManager::UpdateRegisterNotification (this=<optimized out>, aEvent=<optimized out>, aParam=<optimized out>)
    at /code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/bluedroid/BluetoothA2dpManager.cpp:930
#2  0xb574c7c6 in mozilla::dom::bluetooth::BluetoothNotificationRunnable2<mozilla::dom::bluetooth::BluetoothDaemonAvrcpModule::NotificationHandlerWrapper, void, int, int, int, int>::Run (
    this=<optimized out>) at ../../../../../../../code/b2g37_v2_2/mozilla-b2g37_v2_2/dom/bluetooth/BluetoothInterfaceHelpers.h:433
(In reply to Teri Wen [:twen] from comment #4)
> Connection lost is 100% reproducible, crash may or may not happen.

Clarification:
It depends on STR. If you let car kit UI switch to media player page, car kit sends PLAYER_APP_SETTING_CHANGE event to ask for registration event, bluedroid crashes.
Assignee: nobody → shuang
Crash Signature: [@ mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp ]
Keywords: crash
Whiteboard: [b2g-crash]
Crash Signature: [@ mozilla::dom::bluetooth::BluetoothDaemonAvrcpInterface::RegisterNotificationRsp ] → [@ libxul.so!mozilla::dom::bluetooth::BluetoothProfileController::NotifyCompletion(nsAString_internal const&) [nsTArray.h : 942 + 0x0] ]
Correct Crash Signature, Comment 13 is just for clarification using gdb not real crash signature.
Based on Comment 9, Comment 10. We have to deal with player app settings, at least for this case, we don't want to crash bluedroid (from codearoura variant), even though AOSP bluedroid did not implement this feature, but AOSP HAL indeed define the interfaces. 

Equalizer ON/OFF status/Scan ON/OFF status skipped because the current music player don't support this feature: https://www.codeaurora.org/cgit/quic/la/platform/packages/apps/Bluetooth/tree/src/com/android/bluetooth/a2dp/Avrcp.java?h=kk_rb5#n3089
blocking-b2g: 3.0? → 2.2?
Change blocking flag to 2.2? because incomplete RegisterNotificationRespose can cause bluedroid crash (non AOSP version from codearoura), this should also apply to 'Orion' device.
I opened follow up bug (bug 1149131) for feature implementation. But now I don't want to introduce this in v2.2. This feature is not expected and not in v2.2 road-map.
Comment on attachment 8585466 [details] [diff] [review]
Bug 1148311 - Handle AVRCP app settings notification response

Review of attachment 8585466 [details] [diff] [review]:
-----------------------------------------------------------------

The code looks good.

As far as I understand the comments, the bug happens on CAF code, but not AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least contains the required interfaces. Right? Does the fix work on AOSP as well?
Attachment #8585466 - Flags: review?(tzimmermann) → review+
(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #20)
> Comment on attachment 8585466 [details] [diff] [review]
> Bug 1148311 - Handle AVRCP app settings notification response
> 
> Review of attachment 8585466 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> The code looks good.
> 
> As far as I understand the comments, the bug happens on CAF code, but not
> AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least
> contains the required interfaces. Right? Does the fix work on AOSP as well?

http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/btif_rc.c#1067
When bluedroid received app_settings related notification registration from the remote device (car kit), it just simply immediately rejects, so I think Gecko won't have chance to get the request. But you're right, we should double confirm again. I will test on various combinations (flame-kk (CAF), Nexus-5(AOSP), Orion(CAF L)).
blocking-b2g: 2.2? → 2.2+
(In reply to Shawn Huang [:shawnjohnjr] from comment #21)
> (In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #20)
> > Comment on attachment 8585466 [details] [diff] [review]
> > Bug 1148311 - Handle AVRCP app settings notification response
> > 
> > Review of attachment 8585466 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > The code looks good.
> > 
> > As far as I understand the comments, the bug happens on CAF code, but not
> > AOSP. The required feature is only in CAF, but not AOSP. But AOSP at least
> > contains the required interfaces. Right? Does the fix work on AOSP as well?
> 
> http://androidxref.com/5.1.0_r1/xref/external/bluetooth/bluedroid/btif/src/
> btif_rc.c#1067
> When bluedroid received app_settings related notification registration from
> the remote device (car kit), it just simply immediately rejects, so I think
> Gecko won't have chance to get the request. But you're right, we should
> double confirm again. I will test on various combinations (flame-kk (CAF),
> Nexus-5(AOSP), Orion(CAF L)).

I've tested Flame-kk/Nexus-5. As expected, Nexus-5 won't be registered app settings notification.
https://hg.mozilla.org/mozilla-central/rev/7d0fbe8f67c2
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2.2 S9 (3apr)
Please nominate this patch for b2g37 approval when you get a chance.
Flags: needinfo?(shuang)
Comment on attachment 8585466 [details] [diff] [review]
Bug 1148311 - Handle AVRCP app settings notification response

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): CAF bluedroid enables app settings features
User impact if declined: Using CAF bluedroid, system could crash
Testing completed: I tested the patch with a car kit which supports application settings using Nexus 5/flame-kk
Risk to taking this patch (and alternatives if risky): This patch provides initialized application settings value to avoid bluedroid internal crash.
String or UUID changes made by this patch:None
Flags: needinfo?(shuang)
Attachment #8585466 - Flags: approval-mozilla-b2g37?
Attachment #8585466 - Flags: approval-mozilla-b2g37? → approval-mozilla-b2g37+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: