Closed Bug 2080318 Opened 1 month ago Closed 1 month ago

GeckoView junit tests pass when a delegateUntilTestEnd callback is called fewer times than its @AssertCalled count

Categories

(GeckoView :: General, defect)

All
Android
defect

Tracking

(firefox160 fixed)

RESOLVED FIXED
160 Branch
Tracking Status
firefox160 --- fixed

People

(Reporter: RyanVM, Assigned: RyanVM)

Details

Attachments

(5 external files)

In GeckoView junit tests that extend BaseSessionTest, the end-of-test @AssertCalled count check on delegates registered with delegateUntilTestEnd never fails. A test that expects count = 1 passes when the callback is never called. For example, HistoryDelegateTest#flushSessionStateTriggersHistoryChange passed while onHistoryStateChange was not called before the test ended.

BaseSessionTest registers an ErrorCollector and hands it to GeckoSessionTestRule with setErrorCollector. The collector is a Verifier that runs inside the session rule's statement, so it checks its errors when the test method returns. GeckoSessionTestRule calls performTestEndCheck() after base.evaluate() returns, and assertMatchesCount records its mismatches through checkThat into the collector, which has already been verified. These failures are never reported. Checks that run while the test body is still executing, such as the "call count should be within limit" check and the counts checked by waitUntilCalled, still fail as expected.

testExtensionProcessCrash never gets the onDisabledProcessSpawning call it counts in the junit runtime, so it
is disabled until bug 2080467.

A waitUntilCalled given a delegate interface and method names now treats a call to any overload as satisfying
its name, instead of waiting for every overload, so adding an overload to a delegate no longer makes those
waits time out.

These tests passed only because the end-of-test counts were never checked. Several ended before the callback
could be delivered, and enableDisable and translateTest declared counts their flows do not produce in
automation.

The rule already records every proxied delegate call, so waitUntilCalled on the registered interface replaces
a GeckoResult that only signalled the call, and the registration's @AssertCalled count still checks it. Action
delegates that were set directly on the extension or session are registered through
addExternalDelegateDuringNextWait so their calls are recorded. Flags completed from a Handler after a fixed
delay are replaced the same way. In the fetch tests that delay only kept the count = 1 scope open to catch a
second fetch, so their delegates are now registered until test end instead.

Attachment #9657302 - Attachment description: Bug 2080318 - Fail the test when an end-of-test @AssertCalled count does not match. r=#geckoview-reviewers → WIP: Bug 2080318 - Fail the test when an end-of-test @AssertCalled count does not match. r=#geckoview-reviewers
Attachment #9657303 - Attachment description: Bug 2080318 - Fix the GeckoView junit tests whose end-of-test @AssertCalled counts did not match. r=#geckoview-reviewers → WIP: Bug 2080318 - Fix the GeckoView junit tests whose end-of-test @AssertCalled counts did not match. r=#geckoview-reviewers
Attachment #9657304 - Attachment description: Bug 2080318 - Wait for recorded delegate calls in GeckoView junit tests instead of completing a GeckoResult in the delegate. r=#geckoview-reviewers → WIP: Bug 2080318 - Wait for recorded delegate calls in GeckoView junit tests instead of completing a GeckoResult in the delegate. r=#geckoview-reviewers

The action delegates were set directly on the extension or session, so the rule never recorded their calls and
the tests completed a GeckoResult instead. Registering them through addExternalDelegateDuringNextWait records
the calls, so the tests wait with waitUntilCalled. A wait on several method names waits for every one of them,
so the tests that handle either action wait only on the one their id selects.

In the fetch tests the delay kept the count = 1 scope open after the fetch, the only way they could see a
second one. Nothing marks the end of Gecko's fetching, so they now check that the fetch happened, with the
delegate registered until test end so an extra fetch recorded before then still fails the count.

Attachment #9657302 - Attachment description: WIP: Bug 2080318 - Fail the test when an end-of-test @AssertCalled count does not match. r=#geckoview-reviewers → Bug 2080318 - Fail the test when an end-of-test @AssertCalled count does not match. r=#geckoview-reviewers
Attachment #9657303 - Attachment description: WIP: Bug 2080318 - Fix the GeckoView junit tests whose end-of-test @AssertCalled counts did not match. r=#geckoview-reviewers → Bug 2080318 - Fix the GeckoView junit tests whose end-of-test @AssertCalled counts did not match. r=#geckoview-reviewers
Attachment #9657304 - Attachment description: WIP: Bug 2080318 - Wait for recorded delegate calls in GeckoView junit tests instead of completing a GeckoResult in the delegate. r=#geckoview-reviewers → Bug 2080318 - Wait for recorded delegate calls in GeckoView junit tests instead of completing a GeckoResult in the delegate. r=#geckoview-reviewers
Attachment #9657322 - Attachment description: WIP: Bug 2080318 - Register ExtensionActionTest's action delegates with the test rule and wait for their calls. r=#geckoview-reviewers → Bug 2080318 - Register ExtensionActionTest's action delegates with the test rule and wait for their calls. r=#geckoview-reviewers
Attachment #9657323 - Attachment description: WIP: Bug 2080318 - Wait for the delegate calls in AutocompleteTest instead of completing a GeckoResult after a fixed delay. r=#geckoview-reviewers → Bug 2080318 - Wait for the delegate calls in AutocompleteTest instead of completing a GeckoResult after a fixed delay. r=#geckoview-reviewers
Pushed by rvandermeulen@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/f9c3ca788294 https://hg.mozilla.org/integration/autoland/rev/9a8349aafa35 Fail the test when an end-of-test @AssertCalled count does not match. r=geckoview-reviewers,owlish https://github.com/mozilla-firefox/firefox/commit/01c3edc54593 https://hg.mozilla.org/integration/autoland/rev/95bc0cee2a60 Fix the GeckoView junit tests whose end-of-test @AssertCalled counts did not match. r=geckoview-reviewers,owlish https://github.com/mozilla-firefox/firefox/commit/2479fbdef22b https://hg.mozilla.org/integration/autoland/rev/fc1572b1f68a Wait for recorded delegate calls in GeckoView junit tests instead of completing a GeckoResult in the delegate. r=geckoview-reviewers,owlish https://github.com/mozilla-firefox/firefox/commit/3e6de9d1c454 https://hg.mozilla.org/integration/autoland/rev/147705b0d242 Register ExtensionActionTest's action delegates with the test rule and wait for their calls. r=geckoview-reviewers,owlish https://github.com/mozilla-firefox/firefox/commit/91733db835c9 https://hg.mozilla.org/integration/autoland/rev/ab6f5126163e Wait for the delegate calls in AutocompleteTest instead of completing a GeckoResult after a fixed delay. r=geckoview-reviewers,owlish
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: