Closed Bug 928635 Opened 11 years ago Closed 11 years ago

persistent mozContacts failures in gaia-ui-tests

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bkelly, Assigned: reuben)

References

Details

(Whiteboard: [xfail])

Attachments

(1 file)

It seems we are getting consistent failures like the following in gaia-ui-tests.  Possibly related to bug 850430 landing?

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette_test.py", line 123, in run
    self.setUp()
  File "/srv/gaia-master/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_sms_contact.py", line 17, in setUp
    self.data_layer.insert_contact(self.contact)
  File "/srv/gaia-master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 159, in insert_contact 
    result = self.marionette.execute_async_script('return GaiaDataLayer.insertContact(%s);' % json.dumps(contact), special_powers=True)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 612, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: TypeError: contact.init is not a function
        stacktrace:
        execute_async_script @gaia_test.py, line 159
        inline javascript, line 397
        src: "    contact.init(aContact);"
TEST-UNEXPECTED-FAIL | test_sms_contact.py test_sms_contact.TestContacts.test_sms_contact
Moving back to gaia rev 6b0a8108eac92ab3b5e73946c57f750fe8744356 changes the output to this:

  OK
  TEST-START test_sms_contact.py
  test_sms_contact (test_sms_contact.TestContacts) ... ERROR

But there is no stack or "TEST-UNEXPECTED-FAIL".  Perhaps this has always been happening and we didn't realize.  Some of the commits I rolled back to test this were for bug 927404 which improves error reporting.

Thoughts?  I won't have any more time to look at this today unfortunately.
Reuben also indicates over in bug 914191 comment 22 that the underlying failure could be due to webidl changes.
So I fixed users of the Contacts API, but now I'm getting timeouts on tests that use the Contacts app: https://travis-ci.org/mozilla-b2g/gaia/jobs/12764456

TEST-START test_keyboard.py
test_keyboard_basic (test_keyboard.TestKeyboard) ... ERROR
======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette_test.py", line 132, in run
 testMethod()
File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/keyboard/test_keyboard.py", line 40, in test_keyboard_basic
 self.wait_for_condition(lambda m: len(contacts_app.contacts) == 1)
File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 691, in wait_for_condition
 raise TimeoutException(message)

TEST-UNEXPECTED-FAIL | test_keyboard.py test_keyboard.TestKeyboard.test_keyboard_basic | TimeoutException: Condition timed out
Yeah, I think it's just "multiple tests were broken, but reporting that a test was broken was also broken so nobody knew." But they sure were nice and green, while green was a lie, weren't they?

Blocks reopening b2g-inbound, though, unless you want to declare the current state of the suite worthless, hide them everywhere, and go through a round of make-it-acceptable-to-unhide.
Severity: normal → blocker
Reuben, I backed out the keyboard changes causing that earlier today.  Do you have that in your pull?
(In reply to Ben Kelly [:bkelly] from comment #5)
> Reuben, I backed out the keyboard changes causing that earlier today.  Do
> you have that in your pull?

I rebased my PR and got the same failures: https://travis-ci.org/mozilla-b2g/gaia/jobs/12769292
Oh, nevermind, the Contacts tests are timing out on master right now, so I'm just going to merge.
Re comment #0, Ben that test shouldn't run on Travis (it sends a real SMS). Were you running locally?

Re comment #7, Reuben a timeout in gaia-ui is usually the same as a failure. The pattern of failures looks like something in the pull is not working and as such all the tests are broken. I'm not in a position to debug right now but I'd say something in the mock_contact.py is not right so when the test tries to setUp by pushing the mock into the contacts Db it's failing and the test has no data to use.
Did you run them locally before submitting the pull and get a pass? 
test_clock_set_alarm_repeat failure is unrelated.

Either that or a HTML Locator needs to be updated in the tests but that seems unlikely as the app itself wasn't change in the pull.
Meh, I'm not gaining anything by having b2g-inbound closed, it just batches up all the pushes that continue going to gaia anyway.
Severity: blocker → normal
(In reply to Zac C (:zac) from comment #8)
> Re comment #7, Reuben a timeout in gaia-ui is usually the same as a failure.
> The pattern of failures looks like something in the pull is not working and
> as such all the tests are broken. I'm not in a position to debug right now
> but I'd say something in the mock_contact.py is not right so when the test
> tries to setUp by pushing the mock into the contacts Db it's failing and the
> test has no data to use.

The exact same failure was there before I merged my PR so I assumed it was an existing problem.
That's not true, you can see in the job history of Travis that the failure types after each of your commits were different as you progressively fixed the problems!
What I mean is the failures in those jobs are not the same as the failures we see in your merge pull request commit. Were you trying to fix those failures?

Was there a Gecko or contacts API changed that caused the original failures? because these tests were fine when I left the office on Friday evening, they are definitely not some long-term persisent reds.
Attached file Gaia PR pointer
Assignee: nobody → reuben.bmo
Attachment #819549 - Flags: review?(bkelly)
Attachment #819549 - Attachment description: killme.html → Gaia PR pointer
Attachment #819549 - Attachment mime type: text/plain → text/html
Reuben, looking better!
I'll fix the alarmclock tests this morning and we'll try and get Travis green.
Blocks: 850430
Comment on attachment 819549 [details]
Gaia PR pointer

Reuben, please address the comments from Github:

  - photo is a sequence<blob> so the I think we need an array there
  - correct bug number in commit/PR subject

With those changes r=me.  We have more fixes that need to be made, but lets not block getting these in.
Attachment #819549 - Flags: review?(bkelly) → review+
note, the tree is closed till this issue get fixed
Blocks: 928915
Depends on: 928922
Depends on: 928939
Depends on: 928937
No longer depends on: 928939
Due to the general lack of coordination on these fixes Rik filed bug #928915 to disable all of the failing tests.
Blocks: 928407
Blocks: 928412
Whiteboard: [xfail]
Since I believe reuben had addressed the questions/issues from GH I went ahead and merged this:

  https://github.com/mozilla-b2g/gaia/commit/ef8730359a3eb8033c60ef93a01a757b9876f792

I will look at re-enabling tests now.
Depends on: 929078
Test re-enabled.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: