[Nexus 5] fanotify EOVERFLOW error for 4g up video file size in GonkDiskSpaceWatcher().

RESOLVED FIXED in Firefox 41, Firefox OS v2.2

Status

()

--
critical
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: hcheng, Assigned: jerry, NeedInfo)

Tracking

({crash, verifyme})

unspecified
mozilla41
ARM
Gonk (Firefox OS)
crash, verifyme
Points:
---

Firefox Tracking Flags

(blocking-b2g:2.2+, firefox39 wontfix, firefox40 wontfix, firefox41 fixed, b2g-v2.0 affected, b2g-v2.0M affected, b2g-v2.1 affected, b2g-v2.1S affected, b2g-v2.2 fixed, b2g-master fixed)

Details

(Whiteboard: [v2.2-nexus-5-l][b2g-crash], crash signature)

Attachments

(4 attachments)

Comment hidden (empty)
* STR:
1. restart phone
2. unlock phone, and launch Camera at Homescreen
3. start record with flashlight on
4. keep recording more than 30 minutes

* Actual result:
After step 4, the phone is really hot, and then b2g crashes. After reboot, when I try to launch video and audio app, the b2g would crash. However, gallery can be launched.

* Crash Report:
https://crash-stats.mozilla.com/report/index/0bebbb46-999d-42c0-8dd0-25bdf2150522

Below is the list. Its content seems similar.
root@hammerhead:/data/b2g/mozilla/Crash Reports/submitted # ll
-rw-r----- root     root           49 2015-05-21 04:41 bp-0bebbb46-999d-42c0-8dd0-25bdf2150522.txt
-rw-r----- root     root           49 2015-05-21 05:10 bp-1243542b-e726-40b2-b429-a12ba2150522.txt
-rw-r----- root     root           49 2015-05-21 05:17 bp-36c9b0b5-c321-4f50-8240-8486b2150522.txt
-rw-r----- root     root           49 2015-05-21 05:19 bp-57f12ce4-ec0a-4173-8b0e-6e0252150522.txt
-rw-r----- root     root           49 2015-05-21 05:18 bp-6875bb3f-55d2-476d-97d3-b4ee52150522.txt
-rw-r----- root     root           49 2015-05-21 05:20 bp-90298975-0ec6-46f0-a74e-25bb12150522.txt
-rw-r----- root     root           49 2015-05-21 04:48 bp-cba1de2b-1bf0-4698-b4fa-a12bc2150522.txt
-rw-r----- root     root           49 2015-05-21 04:41 bp-d643f430-1a20-4823-9db1-c32402150522.txt
-rw-r----- root     root           49 2015-05-21 05:20 bp-fe3a292f-6d31-4acd-8b13-af6432150522.txt


* Test Env:
Build ID               20150521002508
Gaia Revision          bc42fbc12d622bffd7e8afcb8d56f8a1d9773c60
Gaia Date              2015-05-20 22:32:56
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/6e4eaf59efda
Gecko Version          37.0
Device Name            hammerhead
Firmware(Release)      5.1
Firmware(Incremental)  eng.cltbld.20150521.040628
Firmware Date          Thu May 21 04:06:43 EDT 2015
Bootloader             HHZ12f
Severity: normal → critical
blocking-b2g: --- → 2.2?
Summary: [Nexus 5] After recording a long video (more than 30 mins), b2g crashes. and launching video/music results in crashes. → [Nexus 5] After recording a long video (more than 30 mins), b2g crashes. Then, launching video/music results in crashes.
Whiteboard: [v2.2-nexus-5-l]
Component: MTP/UMS → GonkIntegration
Whiteboard: [v2.2-nexus-5-l] → [v2.2-nexus-5-l][b2g-crash]
@Josh, please help here.
Flags: needinfo?(jocheng)
Let me try this on flame-kk
Keywords: crash
@Eric, we might also need to test this without flashlight on. Could you ask Marigold for help?
Hi Peter, Could you also check this on FLAME-KK with master, from my test on v2.2, video recording is not even working, I need to create a new bug after I confirm that again. Thanks.
Flags: needinfo?(pbylenga)
Here is the kernel log.
The VID_0001.3gp is the video with 30min recording content.

<36>[   50.217784] type=1400 audit(1432285370.500:46): avc: denied { search } for pid=531 comm=53747265616D5472616E73202331 name="/" dev="fuse" ino=0 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=dir
<36>[   50.218458] type=1400 audit(1432285370.500:47): avc: denied { read } for pid=531 comm=53747265616D5472616E73202331 name="0" dev="fuse" ino=1 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=dir
<36>[   50.218740] type=1400 audit(1432285370.500:48): avc: denied { open } for pid=531 comm=53747265616D5472616E73202331 name="0" dev="fuse" ino=1 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=dir
<36>[   50.223230] type=1400 audit(1432285370.500:49): avc: denied { getattr } for pid=531 comm=53747265616D5472616E73202331 path="/mnt/shell/emulated/0/DCIM/100MZLLA/VID_0001.3gp" dev="fuse" ino=5 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=file
<36>[   50.288781] type=1400 audit(1432285370.570:50): avc: denied { read } for pid=3115 comm=426C6F62204F70656E6572 name="VID_0001.3gp" dev="fuse" ino=5 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=file
<36>[   50.289212] type=1400 audit(1432285370.570:51): avc: denied { open } for pid=3115 comm=426C6F62204F70656E6572 name="VID_0001.3gp" dev="fuse" ino=5 scontext=u:r:init:s0 tcontext=u:object_r:fuse:s0 tclass=file
<36>[   50.385785] type=1400 audit(1432285370.670:52): avc: denied { transfer } for pid=732 comm="Binder_1" scontext=u:r:mediaserver:s0 tcontext=u:r:init:s0 tclass=binder
<36>[   50.386797] type=1400 audit(1432285370.670:53): avc: denied { call } for pid=3118 comm=4D65646961506C7E6261636B202332 scontext=u:r:init:s0 tcontext=u:r:init:s0 tclass=binder
<36>[   50.387200] type=1400 audit(1432285370.670:54): avc: denied { transfer } for pid=3118 comm=4D65646961506C7E6261636B202332 scontext=u:r:init:s0 tcontext=u:r:init:s0 tclass=binder
(Assignee)

Updated

4 years ago
Crash Signature: @ mozilla::hal_impl::GonkDiskSpaceWatcher::OnFileCanReadWithoutBlocking
Created attachment 8609546 [details] [diff] [review]
fix fanotify 4g size limit. v1

Add O_LARGEFILE to support 4g up size for 32bit os.
(Assignee)

Updated

4 years ago
Component: GonkIntegration → Hardware Abstraction Layer (HAL)
Product: Firefox OS → Core
(Assignee)

Updated

4 years ago
Assignee: nobody → hshih
Status: NEW → ASSIGNED
Summary: [Nexus 5] After recording a long video (more than 30 mins), b2g crashes. Then, launching video/music results in crashes. → [Nexus 5] fanotify EOVERFLOW error for 4g up video file size in GonkDiskSpaceWatcher().
Comment on attachment 8609546 [details] [diff] [review]
fix fanotify 4g size limit. v1

When device keeps recording more than 30 minutes, the video file will reach 4G size.
We don't handle the EOVERFLOW error in GonkDiskSpaceWatcher::OnFileCanReadWithoutBlocking(), then we get crash at [1].

This patch adds O_LARGEFILE flag to support 4G up file size
[1]
https://hg.mozilla.org/mozilla-central/annotate/3e737d30f842/hal/gonk/GonkDiskSpaceWatcher.cpp#l261
Attachment #8609546 - Flags: review?(fabrice)
Attachment #8609546 - Flags: review?(bent.mozilla)
(Assignee)

Updated

4 years ago
Attachment #8609546 - Flags: review?(bent.mozilla)
Attachment #8609546 - Flags: review?(fabrice) → review+
Comment on attachment 8609546 [details] [diff] [review]
fix fanotify 4g size limit. v1

[Approval Request Comment]
Bug caused by (feature/regressing bug #):
Video recording size exceed 4G.
User impact if declined:
When we have a large video file, B2G always crash when we click video app. 
Testing completed:
Test manually at m-c and v2.2 with 4G file. I don't get crash in video app.
Risk to taking this patch (and alternatives if risky):
Low. This patch try to support 4g up file at 32bit system.
String or UUID changes made by this patch:
None.
Attachment #8609546 - Flags: approval-mozilla-b2g37?
Should we also have this patch in b2g34 and b2g32? I don't know how to delete the 4G up file without video app unless I use adb command to kill it. If I try to open video app, b2g will crash.
* NEXUS-5-L V2.2
** Reproducible on NEXUS-5-L, flash is not a factor here. 
1. restart phone
2. unlock phone, and launch Camera at Homescreen
4. keep recording more than 30 minutes

* FLAME-KK V2.2, V3.0
** Not reproducible on FLAME-KK, but I am trying to record for longer time to make the clip larger in file size, resolution & bitrate of flame is lower than NEXUS

Video clip by Flame-KK V2.2 - res:1280x720, bitrate: 38.8Kbps, file size 1.9GB for 45 min recording.
Video clip by NEXUS-5-L V2.2 - res:1920x1080, bitrate: 16.6 Mbps,
Flags: needinfo?(pbylenga)
I think you should try to record for 90min. Thus, we can get a 4g up video file.
Flags: needinfo?(echang)
Created attachment 8609746 [details]
2015-05-23-20-56-43.png

FLAME pops a warning says that my storage is full when the file is around 3.8GB(60min), but actually there is still 40% room left in my SD card, I need a gaia or gecko tweak to make it more than 4GB.
Flags: needinfo?(echang)

Comment 17

4 years ago
Waiting patch land on master for b2g37 approval
Flags: needinfo?(jocheng)
Josh, is this v2.2 blocking?
Flags: needinfo?(jocheng)

Comment 19

3 years ago
(In reply to Jerry Shih[:jerry] (UTC+8) from comment #18)
> Josh, is this v2.2 blocking?

Hi Jerry,
I make this as 2.2+ since this will lead to device crash which is not acceptable.
Thanks
blocking-b2g: 2.2? → 2.2+
Flags: needinfo?(jocheng)

Updated

3 years ago
status-b2g-v2.0: --- → affected
status-b2g-v2.0M: --- → affected
status-b2g-v2.1: --- → affected
status-b2g-v2.1S: --- → affected
status-b2g-v2.2: --- → affected
status-b2g-master: --- → affected
Created attachment 8610084 [details]
v2.2.mp4

I tried to do regression window-wanted on N5-L v2.2, but I found that this bug also exists on the latest nightly build which is newer than the oldest b2g-inboud build. Below is my test result using the nightly build:

Results: After recording more than 30 minutes, the device crashes. After rebooting it, when launch video and audio app, the device would crash but gallery can be launched.

See attachment: v2.2.mp4
Reproduce rate: 5/5

Device: Nexus 5 v2.2(Fail)
Build ID               20150224162516
Gaia Revision          ca64f2fe145909f31af266b1730874051ba76c78
Gaia Date              2015-02-24 22:06:53
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/16804008c29f
Gecko Version          37.0
Device Name            hammerhead
Firmware(Release)      5.0
Firmware(Incremental)  eng.cltbld.20150224.201149
Firmware Date          Tue Feb 24 20:12:04 EST 2015
Bootloader             HHZ12f
QA Whiteboard: [MGSEI-Triage+]
Comment hidden (obsolete)
https://hg.mozilla.org/mozilla-central/rev/847a0f04a73c
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
status-firefox41: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla41

Updated

3 years ago
status-b2g-master: affected → fixed

Updated

3 years ago
Attachment #8609546 - Flags: approval-mozilla-b2g37? → approval-mozilla-b2g37+
status-firefox39: --- → wontfix
status-firefox40: --- → wontfix
Keywords: verifyme
Depends on: 1174626
Currently, there is no crash now but got some message mentioned at bug 1174626.
Still try to figure out the definition of this msg, so let's wait until bug 1174626 is resolved.
It's limited in media/libstagefright/MPEG4Writer.cpp

static const int64_t kMax32BitFileSize = 0x00ffffffffLL; // 2^32-1 : max FAT32
                                                         // filesystem file size
                                                         // used by most SD cards

NI Munro to see if we can do a dynamic file size limit based on the storage file system.
Flags: needinfo?(mchiang)
We need to detect the storage type (internal or external SD card) and set the max file size accordingly.
I will try to implement this mechanism.
Flags: needinfo?(mchiang)
Current design limits the maximum recording clip to 4GB, so it will stop recording after around 30 minutes. What's the expected behavior? Remove the limitation and only stop the recording when the whole storage is full? Or increase the maximum size to a larger number like 8GB?
Flags: needinfo?(hcheng)
(In reply to Munro Chiang [:mchiang] from comment #28)
> Current design limits the maximum recording clip to 4GB, so it will stop
> recording after around 30 minutes. What's the expected behavior? Remove the
> limitation and only stop the recording when the whole storage is full? Or
> increase the maximum size to a larger number like 8GB?

Currently, the message is "You have run out of media storage space" which might let user misunderstand the actual meaning. Either changing the notification message or changing the behavior to stop the recording when storage is full are both acceptable for me.

Besides, how did we define 4GB as the maximum size? Does it fit to a newer device?

I think we need UX to define clear behavior.
@Harly, do you know who can help here?
Flags: needinfo?(hcheng) → needinfo?(hhsu)

Comment 30

3 years ago
Yes, I do think the message is misleading to the user, and probably show something like "You have reached the maximum file size for a video clip" would make more sense. I am wondering is the 4GB limitation because of sdcard file system uses FAT32? Also needinfo Rob as he might have more insight to the camera stuff.
Flags: needinfo?(hhsu) → needinfo?(rmacdonald)
4GB file size is limitation of FAT32.
If the file is stored at ext4 partition, like nexus 5 and Aries, the max file size 16 TiB.
4GB file size is limitation of FAT32.
If the file is stored at ext4 partition, like the internal storage of nexus 5 and Aries, the max file size can reach 16 TiB.
If there is a way for camera app to query current file system, then camera app can set the max file size to the whole free space when ext4 partition is used.

Or we can keep current design and just modify the misleading message. However, just be noticed that 4GB can only afford 30 minutes 1080p recording or 10 minutes 4K recording.
Duplicate of this bug: 1174626
Create bug 1184484 to demand more information (file system type) from devicestorage API.
Excluding this bug in our queries because we don't have a Nexus 5.
QA Whiteboard: [MGSEI-Triage+] → [MGSEI-Triage+][QAExclude]
Flags: needinfo?(jmercado)
Flags: needinfo?(jmercado)
You need to log in before you can comment on or make changes to this bug.