Closed Bug 960914 Opened 10 years ago Closed 6 years ago

[OPEN C_1.3]Issue of b2gperf testing

Categories

(Firefox OS Graveyard :: General, defect, P2)

ARM
Gonk (Firefox OS)
defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: chen.cong, Unassigned, NeedInfo)

Details

(Keywords: perf, Whiteboard: [c=automation p= s= u=] [NPOTB])

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36

Steps to reproduce:

     Preconditions:
     PC OS: ubuntu 12.04
     Mobile phone SW:B02
     Python version:2.7.2
     gaiatest version:0.13

    test procedure:
     1.zte@zte-NV59C:~/gaia/tests/python/gaia-ui-tests/gaiatest$ b2gperf homescreen music contacts video gallery settings messages email
     2.check the console and summary



Actual results:

Issue describtion:
     1.while running Running B2GPerfLaunchTest,the cosole show the AppLaunchError: Error launching app and ExceededThresholdError: Exceeded failure threshold for gathering results...
     2.while running Running B2GPerfLaunchMusicTest,the cosole show IncorrectCountError: Incorrect number of music files. Expected 0 but found 1695
     3.while running Running B2GPerfLaunchVideoTest,the cosole show IncorrectCountError: Incorrect number of video files. Expected 0 but found 1708
     4.while running Running B2GPerfLaunchGalleryTest,the cosole show IncorrectCountError: Incorrect number of picture files. Expected 0 but found 278
     5.I am not sure it is a issue of testscript or phone SW,please check it.
blocking-b2g: --- → 1.3?
OS: All → Gonk (Firefox OS)
Hardware: All → ARM
Hey Dave, I don't know which component to put this in, but I think you'll know ;)
Flags: needinfo?(dave.hunt)
Can you try updating to the latest b2gperf version, which is 0.14 and requires gaiatest 0.21.2.
Flags: needinfo?(dave.hunt)
Component: Gaia::TestAgent → General
Can we get actual proof of end-user impact on this?
Clearing nomination until we get proof of user impact. Infrastructure issues generally aren't going to block.
blocking-b2g: 1.3? → ---
See comment 2 and tell us if that solves it. Thanks.
Flags: needinfo?(chen.cong)
Whiteboard: [c=automation p= s= u=]
Whiteboard: [c=automation p= s= u=] → [c=automation p= s= u=][NPOTB]
QA, please confirm whether you also see the errors reported in comment 0 when running b2gperf. If so please also try the suggestion in comment 2.
Keywords: perf, qawanted
Priority: -- → P2
Whiteboard: [c=automation p= s= u=][NPOTB] → [c=automation p= s= u=] [NPOTB]
(In reply to Mike Lee [:mlee] from comment #6)
> QA, please confirm whether you also see the errors reported in comment 0
> when running b2gperf. If so please also try the suggestion in comment 2.

I don't think QA is committed to doing QA Wanted requests for infrastructure problems. These need to be handled by the relevant automation team maintaining the framework.
Keywords: qawanted
Jonathan,

Per Jason's comment 7 can you have an automation engineer confirm this issue?

Thanks,
Mike
Flags: needinfo?(jgriffin)
This bug is going in circles.  :)  I don't know what phone 'SW:B02' is, but I don't think we have one.  

We really need the original reporter to follow the steps in comment #2:

> Can you try updating to the latest b2gperf version, which is 0.14 and
> requires gaiatest 0.21.2.

It would also be helpful to have a logcat captured during the run.
Flags: needinfo?(jgriffin)
I had updataed gaiatest to 0.21.2
but there are still err.
and I run homescreen music contacts video gallery settings messages email,8 app.
but there is not result for video\gallery \music\homescreen 
Another Attachment is the log for this time .
There are two problems here:

1 - We aren't successfully launching the 'homescreen' app.  We don't run startup tests of the homescreen app internally, because the logic we use to run these tests leaves the homescreen running all the time, and therefore we can't successfully do a cold launch on it.  You should remove homescreen from the list of apps you're testing, or we'd need to add logic to handle it.

2 - All of the tests that use media files are dying due to insufficient permissions.  The tests use device storage to pre-populate music, photos, and videos, and they do this using navigator.getDeviceStorage(), which is returning 'null' here.  The tests do this pre-population before launching the app, so this is done in the context of the system app.  In a default B2G build, the system app has storage permissions; does the system app not have permissions for storage in the implementation you're testing?
(In reply to Jonathan Griffin (:jgriffin) from comment #12)
> There are two problems here:

1 - We aren't successfully launching the
> 'homescreen' app.  We don't run startup tests of the homescreen app
> internally, because the logic we use to run these tests leaves the
> homescreen running all the time, and therefore we can't successfully do a
> cold launch on it.  You should remove homescreen from the list of apps
> you're testing, or we'd need to add logic to handle it.

2 - All of the
> tests that use media files are dying due to insufficient permissions.  The
> tests use device storage to pre-populate music, photos, and videos, and they
> do this using navigator.getDeviceStorage(), which is returning 'null' here. 
> The tests do this pre-population before launching the app, so this is done
> in the context of the system app.  In a default B2G build, the system app
> has storage permissions; does the system app not have permissions for
> storage in the implementation you're testing?

I used another SD card,there is no "JavascriptException: TypeError: storage is null".
but still errs, and can't run correctly.
please see the new attactment.
Attached file b2gperf 20140221.txt
There are errors like "mozdevice.devicemanager.DMError: Error creating directory: read only file system" in the log for Bug 960911 too, but I don't know where they come from. I don't know about the other issues either.
There are two general problems here; one, we fail when launching the Music app...is the Music app on your phone present, and is it called Music?

Second, we seem to be encountering various errors when trying to pre-populate media files of various kinds.  It could be that the default location (see e.g., https://github.com/mozilla/b2gpopulate/blob/master/b2gpopulate/b2gpopulate.py#L315) is not valid on this phone.
Firefox OS is not being worked on
Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: