Closed Bug 1033033 Opened 6 years ago Closed 6 years ago

gaiatest test_endurance_camera_photo.py fails at session_capability attribute

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

ARM
Gonk (Firefox OS)
defect

Tracking

(blocking-b2g:2.0+, firefox31 wontfix, firefox32 fixed, firefox33 fixed, b2g-v2.0 fixed, b2g-v2.1 fixed)

RESOLVED FIXED
2.0 S5 (4july)
blocking-b2g 2.0+
Tracking Status
firefox31 --- wontfix
firefox32 --- fixed
firefox33 --- fixed
b2g-v2.0 --- fixed
b2g-v2.1 --- fixed

People

(Reporter: tkundu, Assigned: jgriffin)

References

Details

Attachments

(2 files)

test_endurance_camera_photo.py fails with following logs :


  if self.device_manager:
  File "/net/cros-bs-p4/local/mnt/workspace/tkundu/kk_36/out/host/marionette/client/marionette/marionette_auto_venv/lib/python2.7/site-packages/marionette_client-0.7.10-py2.7.egg/marionette/runner/mixins/b2g.py", line 50, in device_manager
    return self.get_device_manager()
  File "/net/cros-bs-p4/local/mnt/workspace/tkundu/kk_36/out/host/marionette/client/marionette/marionette_auto_venv/lib/python2.7/site-packages/marionette_client-0.7.10-py2.7.egg/marionette/runner/mixins/b2g.py", line 43, in get_device_manager
    capabilities = self.marionette.session_capabilities
AttributeError: 'NoneType' object has no attribute 'session_capabilities'
(marionette_auto_venv)tkundu@tkundu-linux:/net/cros-bs-p4/local/mnt/workspace/tkundu/kk_36$ 
(marionette_auto_venv)tkundu@tkundu-linux:/net/cros-bs-p4/local/mnt/workspace/tkundu/kk_36$ 


I found that we are using marionette_client-0.7.10-py2.7.egg package and this python package contains patch from bug 1019043

If we use same marionette_client-0.7.10-py2.7.egg but revert bug 1019043 from that python package (marionette_client-0.7.10-py2.7.egg) then issue goes away.


Issue is not visible for all gaiatest . 
But some gaiatest like test_endurance_camera_photo.py gives above error.
This is a regression .
Crash Signature: [@ mozalloc_abort(char const*) | NS_DebugBreak | mozilla::ipc::FatalError(char const*, char const*, int, bool) | mozilla::layers::PSharedBufferManagerChild::SendAllocateGrallocBuffer(mozilla::gfx::IntSizeTyped const&, unsigned int const&, unsigned int con…
blocking-b2g: --- → 2.0?
Flags: needinfo?(dave.hunt)
Clearing the blocking flag - we will not block on test only changes.
blocking-b2g: 2.0? → ---
Component: General → Gaia::UI Tests
Jason, this needs to be fixed ASAP as we are blocked with our testing as a result of all this mozrunner churn.  If we can't test then we can't make the release clearly.
blocking-b2g: --- → 2.0+
Tapas, I think you have a workaround for this issue.  Could you please attach that here, so we can look at it and discuss if/how to fix properly.
The traceback in comment 0 is incomplete. It appears that at the time the device_manager property is accessed there is no Marionette object. I have replicated this and include the full trace below:

Traceback (most recent call last):
  File "/Users/dhunt/.virtualenvs/gaia/bin/gaiatest", line 9, in <module>
    load_entry_point('gaiatest==0.25', 'console_scripts', 'gaiatest')()
  File "/Users/dhunt/workspace/gaia/tests/python/gaia-ui-tests/gaiatest/runtests.py", line 60, in main
    cli(runner_class=GaiaTestRunner, parser_class=GaiaTestOptions)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runtests.py", line 28, in cli
    runner = startTestRunner(runner_class, options, tests)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runtests.py", line 19, in startTestRunner
    runner.run_tests(tests)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/base.py", line 823, in run_tests
    self.run_test_sets()
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/base.py", line 1054, in run_test_sets
    self.run_test_set(in_process_tests)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/base.py", line 1031, in run_test_set
    self.run_test(test['filepath'], test['expected'], test['oop'])
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/base.py", line 1000, in run_test
    **self.test_kwargs)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/marionette_test.py", line 336, in add_tests_to_suite
    **kwargs))
  File "/Users/dhunt/workspace/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 915, in __init__
    MemoryEnduranceTestCaseMixin.__init__(self, *args, **kwargs)
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/mixins/endurance.py", line 136, in __init__
    if self.device_manager:
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/mixins/b2g.py", line 50, in device_manager
    return self.get_device_manager()
  File "/Users/dhunt/workspace/mozilla/mozilla-central/testing/marionette/client/marionette/runner/mixins/b2g.py", line 43, in get_device_manager
    capabilities = self.marionette.session_capabilities
AttributeError: 'NoneType' object has no attribute 'session_capabilities'

It looks like the endurance tests were previously relying on the existence of a device manager to suggest we're running against a physical device. Since bug 1019043 we use the Marionette session's capabilities to determine if we're running on a desktop build. Previously, we would have to try/catch the device_manager call. This means that a Marionette session is required when getting the device manager, and one is not available in the initialisation of MemoryEnduranceTestCaseMixin.

Removing the if statement from http://hg.mozilla.org/mozilla-central/file/default/testing/marionette/client/marionette/runner/mixins/endurance.py#l136 appears to fix this for me, however we will then likely see exceptions if these are run against desktop builds. Alternatively, we will need to delay the initialisation of MemoryEnduranceTestCaseMixin until after a Marionette session has been started.
Flags: needinfo?(dave.hunt)
Looks like Dave has your work around there, as long as you don't need to run against desktop builds. We've been making a lot of changes to this code base to update it, and it could be that these tests just need to be updated. Some of these tests may not have been updated since there were a few weeks where we considered retiring them in favor of the AreWeSlimYet tests. In the end we decided to keep the endurance tests, but we likely missed updates as a result of that prioritization change. NeedInfo'ing Jgriffin to take a look.
Flags: needinfo?(jgriffin)
Flags: needinfo?(jgriffin)
This is the simplest and lowest-impact fix that will cause execution to be the same.
Attachment #8449696 - Flags: review?(mdas)
Assignee: nobody → jgriffin
Comment on attachment 8449696 [details] [diff] [review]
Don't access the device_manager during __init__,

Review of attachment 8449696 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm, thanks
Attachment #8449696 - Flags: review?(mdas) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/73a651b7e30a

Depending on how you're running the endurance tests, we may need to bump and release a new marionette-client for this.
Blocks: 1033824
https://hg.mozilla.org/mozilla-central/rev/73a651b7e30a
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
(In reply to Jonathan Griffin (:jgriffin) from comment #6)
> Created attachment 8449696 [details] [diff] [review]
> Don't access the device_manager during __init__,
> 
> This is the simplest and lowest-impact fix that will cause execution to be
> the same.

Thanks for fixing this issue in gecko.

But I am still seeing this issue is present in [1][2] 

[1] https://github.com/mozilla-b2g/gaia/blob/v2.0/tests/python/gaia-ui-tests/requirements.txt
[2] https://pypi.python.org/pypi/marionette_client/0.7.10
[3] Please check this file in 7.10: marionette_client-0.7.10-py2.7.egg/marionette/runner/mixins
/endurance.py

Do we need to change requirments.txt[1] manually and upgrade to different python packages ?
Flags: needinfo?(mdas)
Yes, you need 0.7.11.

I'll update the branches to use this version later today.
Flags: needinfo?(mdas)
Sorry 0.7.11 is still not out yet
(In reply to Zac C (:zac) from comment #13)
> Sorry 0.7.11 is still not out yet

I think that we should keep it opened till we see 0.7.11 is released. Our automation tool is using 0.7.10 [1] [2] [3] and reproducing this bug always.


[1] https://github.com/mozilla-b2g/gaia/blob/v2.0/tests/python/gaia-ui-tests/requirements.txt
[2] https://pypi.python.org/pypi/marionette_client/0.7.10
[3] Please check this file in 7.10: marionette_client-0.7.10-py2.7.egg/marionette/runner/mixins
/endurance.py
Status: RESOLVED → REOPENED
Flags: needinfo?(zcampbell)
Resolution: FIXED → ---
(In reply to Tapas Kumar Kundu from comment #14)
> (In reply to Zac C (:zac) from comment #13)
> > Sorry 0.7.11 is still not out yet
> 
> I think that we should keep it opened till we see 0.7.11 is released. Our
> automation tool is using 0.7.10 [1] [2] [3] and reproducing this bug always.

That isn't how we track bugs in bugzilla. When a patch lands in the respective branches, then the bug is considered fixed. A release of a product is tracked separately.
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
Duplicate of this bug: 1035061
(In reply to Tapas Kumar Kundu from comment #14)
> (In reply to Zac C (:zac) from comment #13)
> > Sorry 0.7.11 is still not out yet
> 
> I think that we should keep it opened till we see 0.7.11 is released. Our
> automation tool is using 0.7.10 [1] [2] [3] and reproducing this bug always.
> 
> 
> [1]
> https://github.com/mozilla-b2g/gaia/blob/v2.0/tests/python/gaia-ui-tests/
> requirements.txt
> [2] https://pypi.python.org/pypi/marionette_client/0.7.10
> [3] Please check this file in 7.10:
> marionette_client-0.7.10-py2.7.egg/marionette/runner/mixins
> /endurance.py

You need to wait until 0.7.11 is relased.
Flags: needinfo?(zcampbell)
No longer blocks: 1035061
(In reply to Zac C (:zac) from comment #17)
> (In reply to Tapas Kumar Kundu from comment #14)
> > (In reply to Zac C (:zac) from comment #13)
> > > Sorry 0.7.11 is still not out yet
> > 
> 
> You need to wait until 0.7.11 is relased.

How do we track this?
Flags: needinfo?(zcampbell)
Flags: needinfo?(jgriffin)
It is marked as a blocking bug of this one.
Flags: needinfo?(zcampbell)
Flags: needinfo?(jgriffin)
Flags: in-moztrap?(bzumwalt)
STR involves automation processes that are unusable to create test case.
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(ktucker)
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(ktucker)
Flags: in-moztrap?(bzumwalt)
Flags: in-moztrap-
You need to log in before you can comment on or make changes to this bug.