Closed
Bug 1050511
Opened 9 years ago
Closed 9 years ago
marionette-test mach target is broken
Categories
(Remote Protocol :: Marionette, defect, P1)
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla35
People
(Reporter: jgraham, Assigned: chmanchester)
References
Details
(Keywords: pi-marionette-runner)
Attachments
(2 files, 1 obsolete file)
2.48 KB,
patch
|
jgraham
:
review+
|
Details | Diff | Splinter Review |
2.64 KB,
patch
|
chmanchester
:
review+
|
Details | Diff | Splinter Review |
In several ways; let me know if this needs to be several bugs. Initially I get errors like: 0:06.46 LOG: MainThread ERROR Got test_start message before suite_start for test test_text_chrome.py 0:06.46 LOG: MainThread ERROR test_end for test_text_chrome.py logged while not in progress. Logged with data: {"status": "SKIP", "message": "\"Bug 896046\"", "test": "test_text_chrome.py"} Which suggests that a suite_start log action is missing. Later some tests start running and I get errors like: Traceback (most recent call last): File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette_test.py", line 171, in run testMethod() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/tests/unit/test_import_script.py", line 86, in test_import_script_and_clear self.assertRaises(JavascriptException, self.marionette.execute_script, "return testFunc();") File "/usr/lib/python2.7/unittest/case.py", line 475, in assertRaises callableObj(*args, **kwargs) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 1199, in execute_script filename=os.path.basename(frame[0])) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/decorators.py", line 35, in _ return func(*args, **kwargs) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 634, in _send_message self._handle_error(response) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 682, in _handle_error raise errors.JavascriptException(message=message, status=status, stacktrace=stacktrace) JavascriptException: JavascriptException: ReferenceError: testFunc is not defined stacktrace: execute_script @case.py, line 475 inline javascript, line 0 src: "let __marionetteFunc = function(){return testFunc();};__marionetteFunc.apply(null, __marionetteParams);" 0:10.14 TEST_START: MainThread test_click.py TestClick.testClickingALinkMadeUpOfNumbersIsHandledCorrectly 0:10.44 TEST_END: MainThread ERROR, expected PASS Traceback (most recent call last): File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette_test.py", line 171, in run testMethod() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/tests/unit/test_click.py", line 24, in testClickingALinkMadeUpOfNumbersIsHandledCorrectly lambda m: m.find_element(By.ID, 'username')) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/wait.py", line 122, in until rv = condition(self.marionette) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/tests/unit/test_click.py", line 24, in <lambda> lambda m: m.find_element(By.ID, 'username')) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 1274, in find_element response = self._send_message('findElement', 'value', **kwargs) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/decorators.py", line 35, in _ return func(*args, **kwargs) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 634, in _send_message self._handle_error(response) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 668, in _handle_error raise errors.NoSuchElementException(message=message, status=status, stacktrace=stacktrace) NoSuchElementException: NoSuchElementException: Unable to locate element: username and a bunch more similar things. Finally the whole harness dies with: MarionetteException: MarionetteException: Could not delete session stacktrace: TypeError: this.browser.removeTab is not a function File "/home/jgraham/develop/gecko-dev-1/testing/marionette/mach_commands.py", line 124, in run_marionette_test topsrcdir=self.topsrcdir, address=address) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/mach_commands.py", line 70, in run_marionette runner = startTestRunner(MarionetteTestRunner, options, tests) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runtests.py", line 41, in startTestRunner runner.run_tests(tests) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runner/base.py", line 664, in run_tests self.run_test_sets() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runner/base.py", line 887, in run_test_sets self.run_test_set(in_process_tests) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runner/base.py", line 864, in run_test_set self.run_test(test['filepath'], test['expected'], test['oop']) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runner/base.py", line 840, in run_test results = runner.run(suite) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/runner/base.py", line 230, in run result = super(MarionetteTextTestRunner, self).run(test) File "/home/jgraham/develop/gecko-dev-1/testing/mozbase/moztest/moztest/adapters/unit.py", line 133, in run test(result) File "/usr/lib/python2.7/unittest/suite.py", line 70, in __call__ return self.run(*args, **kwds) File "/usr/lib/python2.7/unittest/suite.py", line 108, in run test(result) File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__ return self.run(*args, **kwds) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette_test.py", line 208, in run self.cleanTest() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette_test.py", line 294, in cleanTest self._deleteSession() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette_test.py", line 306, in _deleteSession self.marionette.delete_session() File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 826, in delete_session response = self._send_message('deleteSession', 'ok') File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/decorators.py", line 35, in _ return func(*args, **kwargs) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 634, in _send_message self._handle_error(response) File "/home/jgraham/develop/gecko-dev-1/testing/marionette/client/marionette/marionette.py", line 708, in _handle_error raise errors.MarionetteException(message=message, status=status, stacktrace=stacktrace)
Comment 1•9 years ago
|
||
(In reply to James Graham [:jgraham] from comment #0) > In several ways; let me know if this needs to be several bugs. > > Initially I get errors like: > 0:06.46 LOG: MainThread ERROR Got test_start message before suite_start for > test test_text_chrome.py > 0:06.46 LOG: MainThread ERROR test_end for test_text_chrome.py logged while > not in progress. Logged with data: {"status": "SKIP", "message": "\"Bug > 896046\"", "test": "test_text_chrome.py"} > > Which suggests that a suite_start log action is missing. > This seems to show ERROR whenever a test has been skipped.
Comment 2•9 years ago
|
||
The removeTab issue is in bug 1018904
Updated•9 years ago
|
Keywords: ateam-marionette-runner
Assignee | ||
Comment 3•9 years ago
|
||
This should fix the first issue. I logged all the skips before suite_start, which is an error since bug 1036427.
Attachment #8469637 -
Flags: review?(james)
Assignee | ||
Updated•9 years ago
|
Assignee: nobody → cmanchester
Status: NEW → ASSIGNED
Assignee | ||
Updated•9 years ago
|
Assignee: cmanchester → nobody
Status: ASSIGNED → NEW
Reporter | ||
Updated•9 years ago
|
Attachment #8469637 -
Flags: review?(james) → review+
Reporter | ||
Comment 4•9 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/c5531819433c
Assignee | ||
Updated•9 years ago
|
Keywords: leave-open
Assignee | ||
Comment 6•9 years ago
|
||
I set leave-open so this wouldn't be closed when my patch got merged to central, because it only addresses the first issue in the bug report. As far as I know the "testFunc is not defined" error is still present.
Assignee | ||
Comment 7•9 years ago
|
||
Collusion between mach's effects on sys.path and marionette's folder structure means that when assertRaises checks exception types it ends up looking at different copies of the module, gets a mismatch, reraises, and reports a failure. This fixes the issue for me locally. Is there a better way?
Attachment #8502627 -
Flags: review?(gps)
Assignee | ||
Updated•9 years ago
|
Assignee: nobody → cmanchester
Status: NEW → ASSIGNED
Assignee | ||
Comment 8•9 years ago
|
||
Comment on attachment 8502627 [details] [diff] [review] Modify imports in python marionette test files to avoid a mismatch in assertRaises when running through mach. This commit message needs to be updated, but you get the idea.
Comment 9•9 years ago
|
||
Comment on attachment 8502627 [details] [diff] [review] Modify imports in python marionette test files to avoid a mismatch in assertRaises when running through mach. Review of attachment 8502627 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/marionette/mach_commands.py @@ +37,5 @@ > > def run_marionette(tests, b2g_path=None, emulator=None, testtype=None, > address=None, binary=None, topsrcdir=None, **kwargs): > + > + # See bug 1050511. Nit: I always prefer expanding the reason in-line so others don't have to look at yet-another-system to understand code.
Attachment #8502627 -
Flags: review?(gps) → review+
Assignee | ||
Comment 10•9 years ago
|
||
Assignee | ||
Updated•9 years ago
|
Attachment #8502627 -
Attachment is obsolete: true
Assignee | ||
Updated•9 years ago
|
Attachment #8503276 -
Flags: review+
Assignee | ||
Comment 11•9 years ago
|
||
Setting checkin-needed without a try run because this is npotb.
Keywords: leave-open → checkin-needed
Comment 12•9 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/51c57f6ffd34
Keywords: checkin-needed
Comment 13•9 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/51c57f6ffd34
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla35
Updated•8 months ago
|
Product: Testing → Remote Protocol
You need to log in
before you can comment on or make changes to this bug.
Description
•