1428391 - Login list does not update fast enough, causing issues in Unit/UI Tests ( XCTAssertEqual failed: ("20") is not equal to ("1") )

Status: NEW

(Firefox for iOS :: Theme & Visual Design, defect, P3)

Description

[No-Jun Park [:njpark]]

In UITests/LoginManagerTests/testListFiltering, the test occasionally fails with the following error:

 XCTAssertEqual failed: ("20") is not equal to ("1")

(see https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5a4e81633c3dfb0001c7fc2e/test/850b8d60-64d5-4a56-81d6-e1f28dce0ee0/7bbdf522-b1a8-4d0e-a94c-46cb1fc38c51)

When reproduced locally, it appears that the table update action does not keep up to speed with the input action. (The table tried to catch up with the input by clearing the list, but then it just showed everything in the end) Hence Bug 1428065 tries workaround by adding 3-second pause on every input that would update the table.  (This bug only fixes the UITest, and I think one of the unit test fails with same reason as well)

Also, when table update is going on with the test, there are many lines of following warning appears in the log:

AX ERROR: Could not find my mock parent, most likely I am stale.

This could also be caused by same underlying slow table update issue (https://stackoverflow.com/questions/6878530/ax-error-when-using-accessibility-inspector-for-ios-app)

Comment 1

[No-Jun Park [:njpark]]

Should be noted that this would never happen with actual human input (as it's too slow to trigger the issue).  This becomes a problem only when unit tests are being run against the app, hence a low priority item.

Comment 2

[:garvan]

We can use an NSPredicate with an expectation to wait for the table to be populated, such as

+        let loaded = NSPredicate(format: "count == 100")
+        expectation(for: loaded, evaluatedWith: app.tables["History List"].cells, handler: nil)
+        waitForExpectations(timeout: 5, handler: nil)