1177038 - Video - not all files are present in video list when "Device Storage" low/full

Status: RESOLVED DUPLICATE of bug 1042603

(Firefox OS Graveyard :: Gaia::Video, defect)

Description

[aaron ai]

In mozilla v2.2:
with the memory is low/full - Launch Video app, Not all files are present in list or there are no files at all (though they exist). If you free memory they become visible in list.
LOG:
"MediaDB: " "QuotaExceededError" "at MediaDB/openRequest.onsuccess/media.db.onerror (app://88dc40eb-f129-4ef1-9c7a-37849a58a8a1/shared/js/mediadb.js:498:0)" mediadb.js:498:0

"MediaDB: unexpected error in insertRecord:" DOMError { name: "QuotaExceededError", message: "The current transaction exceeded it
…" } "for file:" "/sdcard/New folder (2)/h264_qcif_30fps_128k_aac128k_176x144 - Copy - Copy - Copy - Copy (4) - Copy - Copy - Copy - Copy - Copy -Copy.3g2" " at processQueue/storeRecord/request.onerror (app://88dc40eb-f129-4ef1-9c7a-37849a58a8a1/shared/js/mediadb.js:1952:0)"

CODE(mediadb.js):
var openRequest = indexedDB.open(this.dbname,dbVersion);
// This should never happen for Gaia apps
openRequest.onerror = function(e) {
  console.error('MediaDB():', openRequest.error.name);
};

Comment 1

[Evelyn Hung]

Could you describe more on "memory is low/full" condition? 

(1)How do you go into this situation?
(2)How low is it? Please use |adb b2g-info| command and attached the log.

Comment 2

[aaron ai]

Question 1:
This situation is:
1.	You could open ‘Setting’ -> ‘Storage’ -> ‘Media Storage’, you could see “Internal Storage” , its storage space is 1.57G;
2.	Now, store video files to “Internal Storage”, we could copy hundreds of video to storage, makes the storage space only have a few or full;
3.	You could see “Space left” only have below 200K(may be 200k, 100k,10K or 0K);
4.	In this case above, Open Video app, user do not see any video files in video list;
If delete some video files, make the “Space left” have 50M,100M or more, when open Video app, could see all the video files in video list.

Question 2:
Below is the b2g-info log:
1.	Before store:
C:\Users\cninaaai>adb shell b2g-info
This program needs to run as the root user in order to query pids.
              |    megabytes    |
NAME PID PPID CPU(s) NICE USS PSS RSS SWAP VSIZE OOM_ADJ USER

System memory info:
            Total 848.0 MB
        SwapTotal 424.0 MB
     Used - cache 263.0 MB
  B2G procs (PSS)   0.0 MB
    Non-B2G procs 263.0 MB
     Free + cache 585.0 MB
             Free 399.7 MB
            Cache 185.3 MB
         SwapFree 424.0 MB

zRAM memory info:
                  Total 424.0 MB
             Used total   0.0 MB
          Original size   0.0 MB
        Compressed size   0.0 MB
  Compressed efficiency  100.0 %
      Num of Read/Write      2/1

Low-memory killer parameters:

  notify_trigger 9216 KB

  oom_adj min_free
        0  4096 KB
       58  5120 KB
      117  6144 KB
      352  7168 KB
      470  8192 KB
      588 10240 KB

2.	After store:
C:\Users\cninaaai>adb shell b2g-info
This program needs to run as the root user in order to query pids.
              |    megabytes    |
NAME PID PPID CPU(s) NICE USS PSS RSS SWAP VSIZE OOM_ADJ USER

System memory info:
            Total 848.0 MB
        SwapTotal 424.0 MB
     Used - cache 252.7 MB
  B2G procs (PSS)   0.0 MB
    Non-B2G procs 252.7 MB
     Free + cache 595.3 MB
             Free  30.3 MB
            Cache 565.0 MB
         SwapFree 410.4 MB

zRAM memory info:
                  Total 424.0 MB
             Used total   4.1 MB
          Original size  10.4 MB
        Compressed size   2.7 MB
  Compressed efficiency   74.4 %
      Num of Read/Write 809/3504

Low-memory killer parameters:

  notify_trigger 9216 KB

  oom_adj min_free
        0  4096 KB
       58  5120 KB
      117  6144 KB
      352  7168 KB
      470  8192 KB
      588 10240 KB

Comment 3

[Evelyn Hung]

(In reply to aaron ai from comment #2)
> Question 1:
> This situation is:
> 1.	You could open ‘Setting’ -> ‘Storage’ -> ‘Media Storage’, you could see
> “Internal Storage” , its storage space is 1.57G;
> 2.	Now, store video files to “Internal Storage”, we could copy hundreds of
> video to storage, makes the storage space only have a few or full;
> 3.	You could see “Space left” only have below 200K(may be 200k, 100k,10K or
> 0K);
> 4.	In this case above, Open Video app, user do not see any video files in
> video list;
> If delete some video files, make the “Space left” have 50M,100M or more,
> when open Video app, could see all the video files in video list.

I guess you were saying "device storage" is full, not "memory". It's totally different. So my question is, how do you do step #2 (store video files to “Internal Storage”)? We have a storage hard limit set but the check happens "on-device writing operation", not the case of mount by USB and copy data into storage.

Comment 4

[aaron ai]

(In reply to Evelyn Hung [:evelyn] from comment #3)
> (In reply to aaron ai from comment #2)
> > Question 1:
> > This situation is:
> > 1.	You could open ‘Setting’ -> ‘Storage’ -> ‘Media Storage’, you could see
> > “Internal Storage” , its storage space is 1.57G;
> > 2.	Now, store video files to “Internal Storage”, we could copy hundreds of
> > video to storage, makes the storage space only have a few or full;
> > 3.	You could see “Space left” only have below 200K(may be 200k, 100k,10K or
> > 0K);
> > 4.	In this case above, Open Video app, user do not see any video files in
> > video list;
> > If delete some video files, make the “Space left” have 50M,100M or more,
> > when open Video app, could see all the video files in video list.
> 
> I guess you were saying "device storage" is full, not "memory". It's totally
> different. So my question is, how do you do step #2 (store video files to
> “Internal Storage”)? We have a storage hard limit set but the check happens
> "on-device writing operation", not the case of mount by USB and copy data
> into storage.

Comment 5

[aaron ai]

(In reply to aaron ai from comment #4)
> (In reply to Evelyn Hung [:evelyn] from comment #3)
> > (In reply to aaron ai from comment #2)
> > > Question 1:
> > > This situation is:
> > > 1.	You could open ‘Setting’ -> ‘Storage’ -> ‘Media Storage’, you could see
> > > “Internal Storage” , its storage space is 1.57G;
> > > 2.	Now, store video files to “Internal Storage”, we could copy hundreds of
> > > video to storage, makes the storage space only have a few or full;
> > > 3.	You could see “Space left” only have below 200K(may be 200k, 100k,10K or
> > > 0K);
> > > 4.	In this case above, Open Video app, user do not see any video files in
> > > video list;
> > > If delete some video files, make the “Space left” have 50M,100M or more,
> > > when open Video app, could see all the video files in video list.
> > 
> > I guess you were saying "device storage" is full, not "memory". It's totally
> > different. So my question is, how do you do step #2 (store video files to
> > “Internal Storage”)? We have a storage hard limit set but the check happens
> > "on-device writing operation", not the case of mount by USB and copy data
> > into storage.

yes, it is "device storage", I have changed the title " Video - not all files are present in video list when "Device Storage" low/full".
when user copy data into storage by USB, how to handle this case? if copy too much files to storage, whether could guide users to store files to SD Card or other processing?

Comment 6

[Evelyn Hung]

> > > I guess you were saying "device storage" is full, not "memory". It's totally
> > > different. So my question is, how do you do step #2 (store video files to
> > > “Internal Storage”)? We have a storage hard limit set but the check happens
> > > "on-device writing operation", not the case of mount by USB and copy data
> > > into storage.
> 
> yes, it is "device storage", I have changed the title " Video - not all
> files are present in video list when "Device Storage" low/full".
> when user copy data into storage by USB, how to handle this case? if copy
> too much files to storage, whether could guide users to store files to SD
> Card or other processing?

When storage is mounted by USB, device doesn't have control on file writing, so this is not a correct direction to solve the problem.

Clarify the problem I observed: user data ran out all data space so application can't write anything into its Indexed DB, and then application can't work as normal.

Here are questions come to my mind:

1. to avoid user data ran out space of Indexed DB, maybe we should reserve them into different partitions. I'm not sure how to configure these partitions, this should be done in Gonk or lower layer. Ping jinyoon for comments.

2. IndexedDB accessing failure shouldn't affect UI displaying video list. It seems that we didn't handle this case well. Hema, could you find someone from Media team to identify if this is something about error handling?

Comment 8

[Jinyoon]

I think we have two problems as Evelyn said in comment 6, 

1) When device storage for user data is full, operation for indexed DB is incorrect.(STR from STC)
   It depends on the low layer implementation (disk partition).
   Storage full case in user data area will be not affected to system area including indexed DB file 
   in commercial devices because system partition and user data partition is separated normally.
   
   But it can be happened only on your reference device which uses same partition for system&user data.

2) potential problem in FxOS : bug 1042503
   Even if storage partition is separated, bug 1042503 is still remained when system area becomes full.
   Although it is expected result as designed - at least OS is not crashed, end-user maybe cannot understand OS's behavior against his or her intention or expectation. 

I think it is better to reopen and keep bug 1042503 and make this bug as duplication.

Comment 9

[Jinyoon]

(In reply to Jinyoon from comment #8)
Just correction of bug number (bug 1042503 --> bug 1042603)

> I think we have two problems as Evelyn said in comment 6, 
> 
> 1) When device storage for user data is full, operation for indexed DB is
> incorrect.(STR from STC)
>    It depends on the low layer implementation (disk partition).
>    Storage full case in user data area will be not affected to system area
> including indexed DB file 
>    in commercial devices because system partition and user data partition is
> separated normally.
>    
>    But it can be happened only on your reference device which uses same
> partition for system&user data.
> 
> 2) potential problem in FxOS : bug 1042603
>    Even if storage partition is separated, bug 1042603 is still remained
> when system area becomes full.
>    Although it is expected result as designed - at least OS is not crashed,
> end-user maybe cannot understand OS's behavior against his or her intention
> or expectation. 
> 
> I think it is better to reopen and keep bug 1042603 and make this bug as
> duplication.

Comment 10

[Evelyn Hung]

Okay, agree that even if separating to different partitions, we will face this full storage problem. Since bug 1042603 described more detailed on expectation in its comment 6, keeping that one is a better idea. Thanks.