1016157 - [tarako]When "USB Storage" enabled , speed very slow while PC copy file to phone

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Performance, defect, P1)

Description

[yiwen.liu]

1. enable "USB Storage", and copy a 1.1GB file to phone, it consume 17 minutes
2. not  enable "USB Storage", use "adb push file /sdcard/", only 5-6 minutes
3. same 6821 phone, same sd card, flash android 2.3.5 system FW, in "USB Storage" mode about 4-5 minutes .

Is there some extra security/content check slow down the speed when access SD card on FFOS?

Need help

Comment 1

[Dave Hylands [:dhylands]]

FxOS doesn't participate in that transfer at all.

The speed of transfer is purely a function of the flash speed and/or the kernel driver. FxOS doesn't even get to see the data being transferred until the phone is unplugged from the PC.

Comment 2

[Dave Hylands [:dhylands]]

When the phone is running as FxOS, does the kernel have access to the full RAM of the phone? I believe that we'll often truncate the memory to say 512Mb if there was say 1Gb of RAM on the phone.

The amount of available RAM could very well impact the performance (since the kernel will cache the blocks and then lazy write then out afterwards).

Comment 3

[yiwen.liu]

dhylands:

Thanks for replied.

Yes, compare the "USB Storage" and "adb push", the meminfo and process are different. 
"adb push" higher the Cached and Writeback and lower the Buffers, with extra process "flush-179:0".
while "USB Storage" lower the Cached and Writeback and higher the Buffers, no process "flush-179:0"

/*adb push*/
MemTotal:         100396 kB
MemFree:            2616 kB
Buffers:            1212 kB
Cached:            35700 kB
SwapCached:          268 kB
Active:            35172 kB
Inactive:          40208 kB
Active(anon):      24028 kB
Inactive(anon):    24192 kB
Active(file):      11144 kB
Inactive(file):    16016 kB
Unevictable:         332 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         100396 kB
LowFree:            2616 kB
SwapTotal:         65532 kB
SwapFree:          53464 kB
Dirty:              2236 kB
Writeback:          7040 kB
AnonPages:         46876 kB
Mapped:            15432 kB
Shmem:              1228 kB
Slab:               7048 kB
SReclaimable:       2912 kB
SUnreclaim:         4136 kB
KernelStack:        1784 kB
PageTables:         1396 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      115728 kB
Committed_AS:     258352 kB
VmallocTotal:     385024 kB
VmallocUsed:        6624 kB
VmallocChunk:     375812 kB

/*USB Storage*/

MemTotal:         100396 kB
MemFree:            2892 kB
Buffers:           14612 kB
Cached:            15956 kB
SwapCached:         3016 kB
Active:            29596 kB
Inactive:          41452 kB
Active(anon):      23532 kB
Inactive(anon):    26760 kB
Active(file):       6064 kB
Inactive(file):    14692 kB
Unevictable:         332 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         100396 kB
LowFree:            2892 kB
SwapTotal:         65532 kB
SwapFree:          41320 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         46632 kB
Mapped:            12684 kB
Shmem:              1288 kB
Slab:               5340 kB
SReclaimable:       1140 kB
SUnreclaim:         4200 kB
KernelStack:        1856 kB
PageTables:         1532 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      115728 kB
Committed_AS:     311332 kB
VmallocTotal:     385024 kB
VmallocUsed:        7644 kB
VmallocChunk:     374788 kB

Comment 4

[yiwen.liu]

But when flash andorid FW, in "USB Storage" mode ,notice that also had a "flush-179:0" process and Writeback always work. Why FFOS's "USB Storage" has not the process, I think the flush process gain the speed.

Comment 5

[yiwen.liu]

Actually， the process flush-179:0 exist in ps , but not schedule, always sleep.

Comment 6

[thomas tsai]

Hi Yiwen: If you copy some smaller (<100MB or <10MB) files from PC to tarako, is the ratio ("pc copy" to "adb push") similar to big files (1 GB) copy?

adb push will use the device kernel file system calls to write the transferring data to local storage, 
but in usb mass storage mode, PC directly accesses the low level file format of local storage (sdcard) and many scsi commands and raw data will transfer from PC to device's UMS to write data.
These two modes are quite different, so you may need to find out different latencies of the call flows in the full copy path.

Comment 7

[yiwen.liu]

<10MB is so fast, so I try a 88MB file.

adb push: 23s
pc copy:  82s

Comment 8

[yiwen.liu]

thomas:

I notice the WRITEBACK system, when "Idle" or "adb push" mode, the follow show:
shell@android:/sys/kernel/debug/bdi/179:0 $ cat stats                          
BdiWriteback:            0 kB
BdiReclaimable:          4 kB
BdiDirtyThresh:      15020 kB
DirtyThresh:         15020 kB
BackgroundThresh:     3752 kB
b_dirty:                 1
b_io:                    0
b_more_io:               0
bdi_list:                1
state:                  10

And when set to "USB Storage" mode and enable share, the follow show:
shell@android:/sys/kernel/debug/bdi/179:0 $ cat stats                          
BdiWriteback:            0 kB
BdiReclaimable:          0 kB
BdiDirtyThresh:          0 kB
DirtyThresh:             0 kB
BackgroundThresh:        0 kB
b_dirty:                 0
b_io:                    0
b_more_io:               0
bdi_list:                1
state:                  10

Seems something close the WRITEBACK.

And I try in Dolphin(1.4), the Idle and "adb push" and "usb storage" all ok, WRITEBACK not clear to 0.

I don't know if this infomation useful.

Comment 9

[thomas tsai]

(In reply to yiwen.liu from comment #0)
> 1. enable "USB Storage", and copy a 1.1GB file to phone, it consume 17
> minutes
> 2. not  enable "USB Storage", use "adb push file /sdcard/", only 5-6 minutes
> 3. same 6821 phone, same sd card, flash android 2.3.5 system FW, in "USB
> Storage" mode about 4-5 minutes .
> 
> Is there some extra security/content check slow down the speed when access
> SD card on FFOS?
> 
> Need help

Is it possible to use kernel 3.x in android to test UMS?
And what are the CPU loads in the 3 different test cases?

Comment 10

[yiwen.liu]

thomas:

We found something now.

In VOLD , google write 0 to /proc/sys/vm/dirty_ratio when share USB storage for improve responsiveness. We now give a default value 20 to dirty_ratio , and WriteBack works, 1.1GB file in "PC copy" only 6 minutes.

Comment 11

[James Zhang (Spreadtrum)]

I have fixed this issue on my side.

But kaizhen should modify v1.3t tarako manifest, use sprdb2g_gonk4.0_6821 branch.
<project path="system/vold" name="platform/system/vold" revision="sprdb2g_gonk4.0_6821"/>


commit dfa8014a99240c2c47fc4506373f0dfe3643fb88
Author: zhangrong <[e-mail]>
Date:   Mon Feb 18 13:43:50 2013 +0800

    Bug#315488 cherry-pick bug#27926<usb transfer rate can't reach the high-speed transmission>
    
    [root cause  ]
    [changes     ]
    [side effects]
    [reviewers   ]
    [self test   ] yes
    
    Change-Id: I676409d4ad3a0d1c468d1380e40260a4677ebdd4

Comment 12

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Update vold to fix UMS is too slow

Partner has fixed this in their local build, but I think we should also have this fix in Tarako manifest.

Michael, can you review this patch? Thanks!

Comment 13

[James Zhang (Spreadtrum)]

Kaizhen, you can also modify it on master branch.
And I have merged your pull request.

v1.3t
https://github.com/mozilla-b2g/b2g-manifest/commit/548045bcf6bfda4bcc44c7decbdaa76d7b31970a

Comment 14

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Update vold to fix UMS is too slow

Michael, can you review this patch for master branch? Thanks!

Comment 15

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to James Zhang from comment #13)
> Kaizhen, you can also modify it on master branch.
> And I have merged your pull request.
> 
> v1.3t
> https://github.com/mozilla-b2g/b2g-manifest/commit/
> 548045bcf6bfda4bcc44c7decbdaa76d7b31970a

James, I sent PR to master branch. Let's wait Michael review and get it landed.

Comment 16

[Aki Sasaki (not active)]

Backed out due to bug 1017073.
https://github.com/mozilla-b2g/b2g-manifest/commit/dc458aa79e9b12c64005e3b4b73dc8c0638d9843

Comment 17

[bhavana bajaj [:bajaj]]

James/Kai, Can we please watch the tree while this lands again, we had busted builds today and had to back this out per comment #16

Comment 18

[Joe Cheng [:jcheng] (please needinfo)]

triage: partner fixed it on their side, let's take this as well .thanks

Comment 19

[James Zhang (Spreadtrum)]

(In reply to bhavana bajaj [:bajaj] from comment #17)
> James/Kai, Can we please watch the tree while this lands again, we had
> busted builds today and had to back this out per comment #16

I don't know why, maybe you haven't repo sync all project.
It's OK on my hudson build.

Comment 20

[James Zhang (Spreadtrum)]

I think it's OK now.

Comment 21

[James Zhang (Spreadtrum)]

See bug 1017623.

Comment 22

[Kai-Zhen Li [:seinlin][:kli]]

Aki, 

Now branch "sprdb2g_gonk4.0_6821" of "platform/system/vold" is on git.m.o.
I think we need to re-land the patch has been backed out in v1.3t manifest to fix this bug.
https://github.com/mozilla-b2g/b2g-manifest/commit/dc458aa79e9b12c64005e3b4b73dc8c0638d9843

Comment 23

[Aki Sasaki (not active)]

Have you fixed the root cause of bug 1017073 ?
If not, when we change the manifests back, Tarako builds will be broken again.

Comment 24

[Kai-Zhen Li [:seinlin][:kli]]

The root cause of bug 1017073 is the branch "sprdb2g_gonk4.0_6821" of "platform/system/vold" is missing in that build. In that build, commit 2cfc9271e4403844eaa3deec62d3339412c5ec5f of vold is used. 

In the latest source of vold libfs_mgr is not used.

So the root cause of bug 1017073 was fixed.

Comment 25

[Aki Sasaki (not active)]

Ok, we should be able to back out https://github.com/mozilla-b2g/b2g-manifest/commit/dc458aa79e9b12c64005e3b4b73dc8c0638d9843 then.  Do you have access, or do you need me to?

Comment 26

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Aki Sasaki [:aki] from comment #25)
> Ok, we should be able to back out
> https://github.com/mozilla-b2g/b2g-manifest/commit/
> dc458aa79e9b12c64005e3b4b73dc8c0638d9843 then.  Do you have access, or do
> you need me to?

Yes, need your help as I do not have access to mozilla-b2g. Thanks!

Comment 27

[Aki Sasaki (not active)]

Backed out.
The results should show up on https://tbpl.mozilla.org/?tree=Mozilla-B2g28-v1.3t soon.