1481444 - Encode which versions of Firefox are compatible with geckodriver

Status: NEW

(Testing :: geckodriver, enhancement, P3)

Description

[Andreas Tolfsen ❲:ato❳]

Users of geckodriver are having a hard time knowing if a particular
geckodriver version is compatible with the Firefox they have at
their disposal.  Because we don’t ship a new driver along with each
release of the browsers, like Safari and Chrome does, one particular
version of geckodriver is typically compatible with a wider range
of Firefox versions.

This can lead to cryptic error messages such as this which are hard
to figure out:

> {
>   "value": {
>     "error": "unknown error",
>     "message": "missing field `applicationType` at line 1 column 24",
>     "stack": ""
>   }
> }

What we could do to mitigate this is encode which versions of Firefox
is compatible so that we could error if the Firefox you attempt to
use is too old and known to not work to a satisfactory level.  This
would allow us to present the user with clearer error messages at
a much earlier stage, before we start having trouble deserialising
messages from Marionette.

An alternative proposal would be to make better use of the
marionetteProtocol field geckodriver receives when opening a
connection to Marionette.  But there may be reasons externally to
Marionette that could cause a Firefox to be incompatible, so it
should maybe be reserved for just telling the protocol design
version.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)]

Note that we cannot hard-bake the maximum version of Firefox into geckodriver. At time of the release of geckodriver this isn't known.

Comment 2

[Andreas Tolfsen ❲:ato❳]

Yes, we wouldn’t be able to proof which versions of future Firefoxen
is compatible.  This means users of older geckodrivers might end
up in a situation like the above if they connect to a Firefox that
has made changes to its protocol.  Maybe this means we should
reconsider combine this proposal with checking marionetteProtocol,
and being more aggressive with bumping that?

So far marionetteProtocol has been meant for major revisions of the
protocol, e.g. moving from one format to another or changing data
structures or serialisation.  At the moment we don’t minute minor
changes, where a field is deprecated or removed.  We try to be
concious and keep aliases around for some reasonable time but so
far we haven’t incremented marionetteProtocol as a result of it.

Maybe _because_ we are being cautious when making minor changes to
the protocol and try our best to retain a long period of backwards
compatibility, we could allow ourselves to be more liberal with
incrementing marionetteProtocol when something changes.

The question I don’t entirely have an answer to yet is whether it
needs to be a combination of Firefox version and marionetteProtocol,
or just marionetteProtocol.