persistent mozContacts failures in gaia-ui-tests

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: bkelly, Assigned: reuben)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [xfail])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
It seems we are getting consistent failures like the following in gaia-ui-tests.  Possibly related to bug 850430 landing?

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette_test.py", line 123, in run
    self.setUp()
  File "/srv/gaia-master/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_sms_contact.py", line 17, in setUp
    self.data_layer.insert_contact(self.contact)
  File "/srv/gaia-master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 159, in insert_contact 
    result = self.marionette.execute_async_script('return GaiaDataLayer.insertContact(%s);' % json.dumps(contact), special_powers=True)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette.py", line 612, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: TypeError: contact.init is not a function
        stacktrace:
        execute_async_script @gaia_test.py, line 159
        inline javascript, line 397
        src: "    contact.init(aContact);"
TEST-UNEXPECTED-FAIL | test_sms_contact.py test_sms_contact.TestContacts.test_sms_contact
(Reporter)

Comment 1

5 years ago
Moving back to gaia rev 6b0a8108eac92ab3b5e73946c57f750fe8744356 changes the output to this:

  OK
  TEST-START test_sms_contact.py
  test_sms_contact (test_sms_contact.TestContacts) ... ERROR

But there is no stack or "TEST-UNEXPECTED-FAIL".  Perhaps this has always been happening and we didn't realize.  Some of the commits I rolled back to test this were for bug 927404 which improves error reporting.

Thoughts?  I won't have any more time to look at this today unfortunately.
(Reporter)

Comment 2

5 years ago
Reuben also indicates over in bug 914191 comment 22 that the underlying failure could be due to webidl changes.
(Assignee)

Comment 3

5 years ago
So I fixed users of the Contacts API, but now I'm getting timeouts on tests that use the Contacts app: https://travis-ci.org/mozilla-b2g/gaia/jobs/12764456

TEST-START test_keyboard.py
test_keyboard_basic (test_keyboard.TestKeyboard) ... ERROR
======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.6.0-py2.7.egg/marionette/marionette_test.py", line 132, in run
 testMethod()
File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/keyboard/test_keyboard.py", line 40, in test_keyboard_basic
 self.wait_for_condition(lambda m: len(contacts_app.contacts) == 1)
File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 691, in wait_for_condition
 raise TimeoutException(message)

TEST-UNEXPECTED-FAIL | test_keyboard.py test_keyboard.TestKeyboard.test_keyboard_basic | TimeoutException: Condition timed out
Yeah, I think it's just "multiple tests were broken, but reporting that a test was broken was also broken so nobody knew." But they sure were nice and green, while green was a lie, weren't they?

Blocks reopening b2g-inbound, though, unless you want to declare the current state of the suite worthless, hide them everywhere, and go through a round of make-it-acceptable-to-unhide.
Severity: normal → blocker
(Reporter)

Comment 5

5 years ago
Reuben, I backed out the keyboard changes causing that earlier today.  Do you have that in your pull?
(Assignee)

Comment 6

5 years ago
(In reply to Ben Kelly [:bkelly] from comment #5)
> Reuben, I backed out the keyboard changes causing that earlier today.  Do
> you have that in your pull?

I rebased my PR and got the same failures: https://travis-ci.org/mozilla-b2g/gaia/jobs/12769292
(Assignee)

Comment 7

5 years ago
Oh, nevermind, the Contacts tests are timing out on master right now, so I'm just going to merge.

Comment 8

5 years ago
Re comment #0, Ben that test shouldn't run on Travis (it sends a real SMS). Were you running locally?

Re comment #7, Reuben a timeout in gaia-ui is usually the same as a failure. The pattern of failures looks like something in the pull is not working and as such all the tests are broken. I'm not in a position to debug right now but I'd say something in the mock_contact.py is not right so when the test tries to setUp by pushing the mock into the contacts Db it's failing and the test has no data to use.
Did you run them locally before submitting the pull and get a pass? 
test_clock_set_alarm_repeat failure is unrelated.

Either that or a HTML Locator needs to be updated in the tests but that seems unlikely as the app itself wasn't change in the pull.
Meh, I'm not gaining anything by having b2g-inbound closed, it just batches up all the pushes that continue going to gaia anyway.
Severity: blocker → normal
(Assignee)

Comment 10

5 years ago
(In reply to Zac C (:zac) from comment #8)
> Re comment #7, Reuben a timeout in gaia-ui is usually the same as a failure.
> The pattern of failures looks like something in the pull is not working and
> as such all the tests are broken. I'm not in a position to debug right now
> but I'd say something in the mock_contact.py is not right so when the test
> tries to setUp by pushing the mock into the contacts Db it's failing and the
> test has no data to use.

The exact same failure was there before I merged my PR so I assumed it was an existing problem.

Comment 11

5 years ago
That's not true, you can see in the job history of Travis that the failure types after each of your commits were different as you progressively fixed the problems!

Comment 13

5 years ago
What I mean is the failures in those jobs are not the same as the failures we see in your merge pull request commit. Were you trying to fix those failures?

Was there a Gecko or contacts API changed that caused the original failures? because these tests were fine when I left the office on Friday evening, they are definitely not some long-term persisent reds.
(Assignee)

Comment 14

5 years ago
Created attachment 819549 [details]
Gaia PR pointer
Assignee: nobody → reuben.bmo
Attachment #819549 - Flags: review?(bkelly)
(Assignee)

Updated

5 years ago
Attachment #819549 - Attachment description: killme.html → Gaia PR pointer
Attachment #819549 - Attachment mime type: text/plain → text/html

Comment 15

5 years ago
Reuben, looking better!
I'll fix the alarmclock tests this morning and we'll try and get Travis green.
(Reporter)

Updated

5 years ago
Blocks: 850430
(Reporter)

Comment 16

5 years ago
Comment on attachment 819549 [details]
Gaia PR pointer

Reuben, please address the comments from Github:

  - photo is a sequence<blob> so the I think we need an array there
  - correct bug number in commit/PR subject

With those changes r=me.  We have more fixes that need to be made, but lets not block getting these in.
Attachment #819549 - Flags: review?(bkelly) → review+
note, the tree is closed till this issue get fixed

Updated

5 years ago
Blocks: 928915
(Reporter)

Updated

5 years ago
Depends on: 928922
(Assignee)

Updated

5 years ago
Duplicate of this bug: 928785
(Reporter)

Updated

5 years ago
Depends on: 928939
(Reporter)

Updated

5 years ago
Depends on: 928937
No longer depends on: 928939

Comment 19

5 years ago
Due to the general lack of coordination on these fixes Rik filed bug #928915 to disable all of the failing tests.
Blocks: 928407
Blocks: 928412

Updated

5 years ago
Whiteboard: [xfail]
(Reporter)

Comment 20

5 years ago
Since I believe reuben had addressed the questions/issues from GH I went ahead and merged this:

  https://github.com/mozilla-b2g/gaia/commit/ef8730359a3eb8033c60ef93a01a757b9876f792

I will look at re-enabling tests now.
(Reporter)

Updated

5 years ago
Depends on: 929078

Updated

5 years ago
Duplicate of this bug: 929026
(Reporter)

Comment 22

5 years ago
Test re-enabled.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.