Closed
Bug 1030459
Opened 10 years ago
Closed 10 years ago
gaiatest test_endurance_camera_photo.py broken in FFOS 2.0
Categories
(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)
Tracking
(blocking-b2g:2.0+, b2g-v2.0 fixed, b2g-v2.1 fixed)
RESOLVED
FIXED
blocking-b2g | 2.0+ |
People
(Reporter: tkundu, Unassigned)
References
Details
(Keywords: regression)
Attachments
(1 file)
3.33 KB,
text/plain
|
Details |
We are seeing following error for gaiatest failure : raise JavascriptException(message=message, status=status, stacktrace=stacktrace) JavascriptException: JavascriptException: TypeError: apps is undefined stacktrace: execute_async_script @gaia_test.py, line 44 inline javascript, line 109 src: " let installedApps = apps.installedApps;" TEST-UNEXPECTED-FAIL | test_endurance_camera_photo.py It seems like that this testcase is broken in FFOS 2.0. It is working fine v1.5
Reporter | ||
Updated•10 years ago
|
blocking-b2g: --- → 2.0?
Comment 1•10 years ago
|
||
Justindarc or diego, please check this test failure. Thanks Hema
Flags: needinfo?(jdarcangelo)
Flags: needinfo?(dmarcos)
Comment 2•10 years ago
|
||
Blocking because of test failure (works fine in previous version)
blocking-b2g: 2.0? → 2.0+
Comment 3•10 years ago
|
||
We don't maintain the python integration tests and the message is not very descriptive. We now have integration tests using marionette js that run on device. https://github.com/mozilla-b2g/gaia/tree/master/apps/camera#integration-tests The failing tests can be easily replicated in JS. I'm happy to help migrate it so we can help maintain it. We have no bandwidth to keep two identical test suites in sync.
Flags: needinfo?(dmarcos)
Comment 4•10 years ago
|
||
(In reply to Diego Marcos [:dmarcos] from comment #3) > We don't maintain the python integration tests and the message is not very > descriptive. We now have integration tests using marionette js that run on > device. If they do, they aren't running in a CI environment. The earliest plan to have MarionetteJS on device in a CI environment is planned in Q3 for only the most critical tests on the phone to be ran on a per commit basis. There's no plans to support a CI-based solution here for MarionetteJS for all tests that currently exist in the tree as of right now.
Comment 5•10 years ago
|
||
Anyways - the problem here is in Gaia UI Tests, not developer-driven integration tests, so I'm moving this over to UI Tests & adding needinfo on Rob to take a look.
Component: Gaia::Camera → Gaia::UI Tests
Flags: needinfo?(rwood)
Comment 6•10 years ago
|
||
The problem here is that the get_permission/set_permission need to be executed from the system frame. The user is running without --restart flag (which the tests are designed to run with) which causes the test to execute set_permission from the homescreen frame.
Comment 8•10 years ago
|
||
From Moz's perspective, we can't block on this. CAF is welcome to take advantage of the tests & framework written, but Moz is not liable for fixing issues on CAF's behalf. CAF needs to assign their own engineers to resolve the problem.
blocking-b2g: 2.0+ → 2.0?
Comment 9•10 years ago
|
||
(In reply to Jason Smith [:jsmith] from comment #4) > If they do, they aren't running in a CI environment. The earliest plan to > have MarionetteJS on device in a CI environment is planned in Q3 for only > the most critical tests on the phone to be ran on a per commit basis. > There's no plans to support a CI-based solution here for MarionetteJS for > all tests that currently exist in the tree as of right now. It would be nice to get a timeline in place to get MarionetteJS integration tests running in CI. The Python-based tests are cumbersome to work with because they rely on a library that is fetched from the PIP Python package manager repo. Any time the application changes, the Python library in the PIP repo also needs to be updated to reflect it and we are not aware of policy for how/when/who this gets updated.
Comment 10•10 years ago
|
||
That's not correct about gaiatest/pypi because you can run it straight from the tree, but in any case QA are also working to get the JS tests on device too.
Comment 11•10 years ago
|
||
(In reply to Zac C (:zac) from comment #10) > That's not correct about gaiatest/pypi because you can run it straight from > the tree, but in any case QA are also working to get the JS tests on device > too. But, are our CI environments set up to pull gaiatest from the tree? I'm fairly certain at some point, CI was pulling it from PIP, but I might be wrong.
Comment 12•10 years ago
|
||
(In reply to Justin D'Arcangelo [:justindarc] from comment #11) > > But, are our CI environments set up to pull gaiatest from the tree? I'm > fairly certain at some point, CI was pulling it from PIP, but I might be > wrong. Yes they are using in-tree! but we package and push it to pypi so that some outside-tree test suites can take advantage of the test framework.
Comment 13•10 years ago
|
||
(In reply to Zac C (:zac) from comment #12) > Yes they are using in-tree! > but we package and push it to pypi so that some outside-tree test suites can > take advantage of the test framework. Ok, good to know! Us Camera devs are Python n00bs, so its also good news to know that QA is working on getting the JS integration tests working in CI :-)
Comment 14•10 years ago
|
||
(In reply to Justin D'Arcangelo [:justindarc] from comment #13) > (In reply to Zac C (:zac) from comment #12) > > Yes they are using in-tree! > > but we package and push it to pypi so that some outside-tree test suites can > > take advantage of the test framework. > > Ok, good to know! Us Camera devs are Python n00bs, so its also good news to > know that QA is working on getting the JS integration tests working in CI :-) FTW - you can see the definition of the goal around this here - https://wiki.mozilla.org/QA/Goals/2014q3#Firefox_OS
Comment 15•10 years ago
|
||
(In reply to Jason Smith [:jsmith] from comment #14) > (In reply to Justin D'Arcangelo [:justindarc] from comment #13) > > (In reply to Zac C (:zac) from comment #12) > > > Yes they are using in-tree! > > > but we package and push it to pypi so that some outside-tree test suites can > > > take advantage of the test framework. > > > > Ok, good to know! Us Camera devs are Python n00bs, so its also good news to > > know that QA is working on getting the JS integration tests working in CI :-) > > FTW - you can see the definition of the goal around this here - > https://wiki.mozilla.org/QA/Goals/2014q3#Firefox_OS Meant to say FWIW, not FTW
Updated•10 years ago
|
Keywords: regression
Comment 16•10 years ago
|
||
Rob, can you take a look at this when you get back?
Flags: needinfo?(rwood)
Comment 17•10 years ago
|
||
Indra, it will be good to know if things changed based on our offline discussion about CAF running the tests per recommendation in comment #6. In parallel, we have someone from our A-team investigating this as well.
Flags: needinfo?(ikumar)
Comment 18•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #16) > Rob, can you take a look at this when you get back? We fixed this on master inside the framework, no need for Rob to investigate.
Flags: needinfo?(rwood)
Updated•10 years ago
|
blocking-b2g: 2.0? → 2.0+
Comment 19•10 years ago
|
||
(In reply to Zac C (:zac) from comment #18) > (In reply to Jonathan Griffin (:jgriffin) from comment #16) > > Rob, can you take a look at this when you get back? > > We fixed this on master inside the framework, no need for Rob to investigate. Will that also be patched on the gaia 2.0v branch which is where CAF pulls from and is hitting this issue ?
Flags: needinfo?(zcampbell)
Comment 20•10 years ago
|
||
>
> We fixed this on master inside the framework, no need for Rob to investigate.
:zac -- Can you please also point us to the bug with the fix.
Flags: needinfo?(ikumar)
Comment 21•10 years ago
|
||
The patch was unrelated to this bug but it was put on v2.0 about a month ago.
Flags: needinfo?(zcampbell)
Comment 23•10 years ago
|
||
Yes please.
Status: NEW → RESOLVED
Closed: 10 years ago
Flags: needinfo?(zcampbell)
Resolution: --- → FIXED
Updated•10 years ago
|
status-b2g-v2.0:
--- → fixed
status-b2g-v2.1:
--- → fixed
Comment 24•9 years ago
|
||
No STR is present to create test case to address bug.
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(ktucker)
Flags: in-moztrap?
Updated•9 years ago
|
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(ktucker)
Flags: in-moztrap?
Flags: in-moztrap-
You need to log in
before you can comment on or make changes to this bug.
Description
•