Closed Bug 925688 Opened 12 years ago Closed 9 years ago

Intermittent test_getactiveframe_oop.py test_getactiveframe_oop.TestGetActiveFrameOOP.test_active_frame_oop | AssertionError: False is not true

Categories

(Testing :: Marionette Client and Harness, defect)

Product:
Testing
Component:
Marionette Client and Harness
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: cbook, Unassigned)

References

(
URL
)

Details

(Keywords: intermittent-failure, pi-marionette-intermittent, Whiteboard: [leave open][test disabled])

Attachments

(1 file)

bug925688.patch
871 bytes, patch
mdas
: review+
Details | Diff | Splinter Review
Ubuntu VM 12.04 mozilla-inbound debug test marionette on 2013-10-11 01:56:59 PDT for push 266532efd5cb slave: tst-linux32-ec2-087 https://tbpl.mozilla.org/php/getParsedLog.php?id=28973371&tree=Mozilla-Inbound TEST-UNEXPECTED-FAIL | test_getactiveframe_oop.py test_getactiveframe_oop.TestGetActiveFrameOOP.test_active_frame_oop | AssertionError: False is not true Return code: 2560 Marionette exited with return code 2560: harness failures # TBPL FAILURE #
This is happening pretty frequently. Andrew, can you help find an owner please?
Flags: needinfo?(overholt)
Rob, what do you think is the problem here? IPC-related?
Flags: needinfo?(overholt) → needinfo?(rwood)
Wow, I'll have a look, thanks
Flags: needinfo?(rwood)
I grabbed the latest tbpl Firefox desktop build for ubuntu from above (where the test most recently failed) and ran the test 25x on my local VM, it passed each time. Also ran the entire marionette unit test suite on that same build 10x, and no repro. I'm not really sure where to go from here... maybe some kind of timing issue on the test VMs? Any ideas, Mdas? Perhaps this test should be disabled for now?
Flags: needinfo?(mdas)
(In reply to Robert J. Wood [:rwood] from comment #57) > I grabbed the latest tbpl Firefox desktop build for ubuntu from above (where > the test most recently failed) and ran the test 25x on my local VM, it > passed each time. Also ran the entire marionette unit test suite on that > same build 10x, and no repro. I'm not really sure where to go from here... > maybe some kind of timing issue on the test VMs? Any ideas, Mdas? Perhaps > this test should be disabled for now? I haven't been able to reproduce this locally on either Firefox desktop, B2G desktop or the emulator. We might have to get access to the test slave to investigate
Flags: needinfo?(mdas)
It seems to be happening fairly often so reproducing on the slave should be easy enough (famous last words).
Rob, can you try this on one of the slaves?
Flags: needinfo?(rwood)
Yes but I'm not sure how to get access to one of the slaves - I believe I need to ask releng, is that correct Rail?
Flags: needinfo?(rwood) → needinfo?(rail)
Yes, just files a bug and a buildduty will handle this.
Flags: needinfo?(rail)
Thanks Rail, I submitted Bug 949441.
Update: I've spent the afternoon trying to setup the test environment on the loaner VM, but no success. This is the error when I try to run the marionette unit test that is failing: root@tst-linux32-ec2-rwood:/builds/slave/test/build/tests/marionette/marionette# python runtests.py --type browser --address localhost:2828 --binary /builds/slave/test/build/application/firefox/firefox /builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_getactiveframe_oop.py starting httpd running webserver on http://10.134.59.202:48684/ Traceback (most recent call last): File "runtests.py", line 33, in <module> cli() File "runtests.py", line 28, in cli runner = startTestRunner(runner_class, options, tests) File "runtests.py", line 19, in startTestRunner runner.run_tests(tests) File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 715, in run_tests self.run_test(test) File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 756, in run_test self.start_marionette() File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 618, in start_marionette device_serial=self.device_serial) File "/builds/slave/test/build/tests/marionette/marionette/marionette.py", line 465, in __init__ assert(self.wait_for_port()), "Timed out waiting for port!" AssertionError: Timed out waiting for port! I followed along in the TBPL output log as best I could, to setup the env to run the test on the Firefox desktop browser since that is where it is failing: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux-debug/1386786325/mozilla-central_ubuntu32_vm-debug_test-marionette-bm51-tests1-linux-build233.txt.gz Mdas: Do you know how I can fix the test runner failure above? If I can't get the test running on the loaner VM in a relative short amount of time I suggest we just disable the test for now and investigate down the road, unless you want to have a look at the VM, or feel this particular test is really important to have running on TBPL. Thanks!
Flags: needinfo?(mdas)
Rob, you should get a gecko.log file that contains the output of the Firefox process; can you attach that?
(In reply to Robert J. Wood [:rwood] from comment #75) > Update: I've spent the afternoon trying to setup the test environment on the > loaner VM, but no success. This is the error when I try to run the > marionette unit test that is failing: > > root@tst-linux32-ec2-rwood:/builds/slave/test/build/tests/marionette/ > marionette# python runtests.py --type browser --address localhost:2828 > --binary /builds/slave/test/build/application/firefox/firefox > /builds/slave/test/build/tests/marionette/tests/testing/marionette/client/ > marionette/tests/unit/test_getactiveframe_oop.py > starting httpd > running webserver on http://10.134.59.202:48684/ > Traceback (most recent call last): > File "runtests.py", line 33, in <module> > cli() > File "runtests.py", line 28, in cli > runner = startTestRunner(runner_class, options, tests) > File "runtests.py", line 19, in startTestRunner > runner.run_tests(tests) > File > "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line > 715, in run_tests > self.run_test(test) > File > "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line > 756, in run_test > self.start_marionette() > File > "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line > 618, in start_marionette > device_serial=self.device_serial) > File "/builds/slave/test/build/tests/marionette/marionette/marionette.py", > line 465, in __init__ > assert(self.wait_for_port()), "Timed out waiting for port!" > AssertionError: Timed out waiting for port! > > I followed along in the TBPL output log as best I could, to setup the env to > run the test on the Firefox desktop browser since that is where it is > failing: > > http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla- > central-linux-debug/1386786325/mozilla-central_ubuntu32_vm-debug_test- > marionette-bm51-tests1-linux-build233.txt.gz > > Mdas: Do you know how I can fix the test runner failure above? If I can't > get the test running on the loaner VM in a relative short amount of time I > suggest we just disable the test for now and investigate down the road, > unless you want to have a look at the VM, or feel this particular test is > really important to have running on TBPL. Thanks! I'd try and see what the gecko.log says like jgriffin mentioned, also I'd make sure that there are no other firefox/b2g instances running in the background of the test slave that could interfere with the test, just in case!
Flags: needinfo?(mdas)
Sorry, took me awhile to get back to this. Here's the contents of the gecko.log: Error: no display specified nsStringStats => mAllocCount: 14 => mReallocCount: 3 => mFreeCount: 14 => mShareCount: 12 => mAdoptCount: 0 => mAdoptFreeCount: 0 => Process ID: 7043, Thread ID: 3075139328 Here's the command line I'm running and the ouput: root@tst-linux32-ec2-rwood:/builds/slave/test/build/tests/marionette/marionette# python runtests.py --type browser --address localhost:2828 --binary /builds/slave/test/build/application/firefox/firefox /builds/slave/test/build/tests/marionette/tests/testing/marionette/client/marionette/tests/unit/test_getactiveframe_oop.py starting httpd running webserver on http://10.134.59.202:43712/ Traceback (most recent call last): File "runtests.py", line 33, in <module> cli() File "runtests.py", line 28, in cli runner = startTestRunner(runner_class, options, tests) File "runtests.py", line 19, in startTestRunner runner.run_tests(tests) File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 715, in run_tests self.run_test(test) File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 756, in run_test self.start_marionette() File "/builds/slave/test/build/tests/marionette/marionette/runner/base.py", line 618, in start_marionette device_serial=self.device_serial) File "/builds/slave/test/build/tests/marionette/marionette/marionette.py", line 465, in __init__ assert(self.wait_for_port()), "Timed out waiting for port!" AssertionError: Timed out waiting for port! Is this because I am running the test via SSH'ing in to a terminal only with no gui manager or something? Thanks!
Flags: needinfo?(jgriffin)
It is, yes. You may be able to get it to work by setting DISPLAY=:0, otherwise you may need to do this over a VNC connection.
Flags: needinfo?(jgriffin)
Thanks Jonathan, that worked and I was able to run the test on the VM from releng. I ran the test (test_getactiveframe_oop) 125x, that includes 25x of running the entire unit-test suite, and every test passed every single time. I ran the tests on one of the desktop builds that was used in one of the failure links above. I'm not any closer to figuring this out, so I'm going to temporarily disable the test to at least stop this TBPL failure (as it fails quite often). This bug should remain open though even after the patch is applied, as it's just temporary and the intermittent failure should be investigated again when there is time.
Attached patch bug925688.patch — — Splinter Review
Temporarily disable this marionette unit test.
Attachment #8349760 - Flags: review?(mdas)
Attachment #8349760 - Flags: review?(mdas) → review+
Whiteboard: [leave open]
Summary: Intermittent TEST-UNEXPECTED-FAIL | test_getactiveframe_oop.py test_getactiveframe_oop.TestGetActiveFrameOOP.test_active_frame_oop | AssertionError: False is not true → Intermittent test_getactiveframe_oop.py test_getactiveframe_oop.TestGetActiveFrameOOP.test_active_frame_oop | AssertionError: False is not true
Whiteboard: [leave open]
There has not been any recurrences around this for some time so closing
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
I think this bug was accidentally closed, the test is still disabled, see: http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/unit-tests.ini#126 Currently, when trying to run this test, I get: ./mach marionette-test testing/marionette/client/marionette/tests/unit/test_getactiveframe_oop.py Traceback (most recent call last): File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/marionette_test.py", line 289, in run testMethod() File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/tests/unit/test_getactiveframe_oop.py", line 51, in test_active_frame_oop self.marionette.switch_to_frame('remote_iframe1') File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/marionette.py", line 1149, in switch_to_frame response = self._send_message('switchToFrame', 'ok', id=frame, focus=focus) File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/decorators.py", line 36, in _ return func(*args, **kwargs) File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/marionette.py", line 679, in _send_message self._handle_error(response) File "/Users/mwargers/mozilla-central/testing/marionette/client/marionette/marionette.py", line 720, in _handle_error raise errors.NoSuchFrameException(message=message, status=status, stacktrace=stacktrace) NoSuchFrameException: NoSuchFrameException: Unable to locate frame: remote_iframe1 It seems like the id searching in switch_to_frame() doesn't seem to work. Switching by element reference seems to work just fine as in: http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_switch_remote_frame.py So it seems to me that there is something more wrong with the test (or the framework?)? Also, it would have been nice if the line number would have been given in the TEST-UNEXPECTED-FAIL. It's not clear where the test is failing (regarding this bug).
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Whiteboard: [leave open]
Whiteboard: [leave open] → [leave open][test disabled]
Closing as intermittent has not been seen in last 45 days
Status: REOPENED → RESOLVED
Closed: 11 years ago → 9 years ago
Resolution: --- → WORKSFORME
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: