1751369 - "Unable to find a matching set of capabilities" for Android with browser version and no --binary argument specified

Status: NEW

(Testing :: geckodriver, defect, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That's a pretty annoying oversight because it was never tested for Android. Right now we do not add the --remote-debugging-port argument to the GeckoView config file if it's not explicitly passed-in via moz:firefoxOptions["args"]:

https://searchfox.org/mozilla-central/source/testing/geckodriver/src/android.rs#300-316

But we should do that based on the webSocketUrl and moz:debuggerAddress capabilities.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Actually this doesn't seem to be related to geckodriver itself given that with a basic client it works pretty well:

1643029780345	webdriver::server	DEBUG	-> POST /session {"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"args": ["--profile", "/Users/henrik/code/gecko/_profile/"], "prefs": {"remote.log.level": "Trace"}, "androidPackage": "org.mozilla.geckoview.test_runner"}, "webSocketUrl": true}}}
[..]
1643029783323	webdriver::server	DEBUG	<- 200 OK {"value":{"sessionId":"b749c0b7-5c3f-4595-a05c-9829a288392b","capabilities":{"acceptInsecureCerts":false,"browserName":"firefox","browserVersion":"98.0a1","moz:accessibilityChecks":false,"moz:buildID":"20220120214317","moz:debuggerAddress":"localhost:9222","moz:geckodriverVersion":"0.30.0","moz:headless":false,"moz:processID":9522,"moz:profile":"/storage/emulated/0/Android/data/org.mozilla.geckoview.test_runner/files/test_root/org.mozilla.geckoview.test_runner-geckodriver-profile","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"android","platformVersion":24,"proxy":{},"setWindowRect":false,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webSocketUrl":"ws://localhost:9222/session/b749c0b7-5c3f-4595-a05c-9829a288392b"}}}

But running Wdspec tests I end-up with:

 0:33.91 pid:56839 1643029961966	webdriver::server	DEBUG	-> POST /session {"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"args": ["--profile", "/var/folders/q9/vhvs7hf57lgbbvkylx6cdp_w0000gn/T/tmpwbsprzum.mozrunner"], "androidPackage": "org.mozilla.geckoview.test_runner"}, "webSocketUrl": true}}}
 0:33.92 pid:56839 1643029961967	webdriver::server	DEBUG	<- 500 Internal Server Error {"value":{"error":"session not created","message":"Unable to find a matching set of capabilities","stacktrace":""}}

Note that Firefox doesn't get started at all.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, so the problem here is actually that capability matching will fail if no browser binary has been specified. I haven't seen that with my simple client because it accidentally added that argument. With bug 1751196 fixed it should become visible.

Also we might have to fix bug 1751165 to have version information available. If we don't have any we currently bail out with the mentioned failure.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

There is probably more work needed here involving a refactoring of the code. As discussed James will have at least a look soon. Thanks!

Comment 4

[James Graham [:jgraham]]

So, a refactoring here is necessary to fix the underlying problem, but there's a one line hack that might get things working in the short term: since we support the webSocketURL capability for firefox >= 90 and latest ESR is 91, we could just remove the version check.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to James Graham [:jgraham] from comment #4)

So, a refactoring here is necessary to fix the underlying problem, but there's a one line hack that might get things working in the short term: since we support the webSocketURL capability for firefox >= 90 and latest ESR is 91, we could just remove the version check.

Sadly that's not true anymore given that you added a new version check for the MarionetteActivePort file over on bug 1421766.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #5)

(In reply to James Graham [:jgraham] from comment #4)

So, a refactoring here is necessary to fix the underlying problem, but there's a one line hack that might get things working in the short term: since we support the webSocketURL capability for firefox >= 90 and latest ESR is 91, we could just remove the version check.

Sadly that's not true anymore given that you added a new version check for the MarionetteActivePort file over on bug 1421766.

Actually I was wrong and the causing browser version check is really only the one in capabilities.rs. Removing the check for webSocketUrl makes it indeed work. As James suggested I will create a workaround patch for now. It will be part of bug 1753083.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Given that it works fine with the workaround it doesn't have to block the 0.31.0 release. Would be still good to see it fixed for 0.32.0.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We should discuss the priority of this bug given that it could cause us trouble whenever we have to add version checks for Firefox to capability matching again.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It's not that clear how much work this actually would be but at the same time it might not be that important given that most consumers might not even use capability matching. As such we will put it into the backlog for now.