Bugzilla

The [kotlin tokenizer spec](https://searchfox.org/mozilla-central/rev/09b59a7de6848639f27297888889a06e97561074/mobile/android/android-components/components/feature/customtabs/src/test/java/mozilla/components/feature/customtabs/CustomTabsToolbarFeatureTest.kt#246) doesn't seem to set [backtick_strings](https://github.com/mozsearch/mozsearch/blob/ada3e73856a35052dab9b8ffcfc473950d132e6a/tools/src/languages.rs#L11) to true, and it seems like perhaps the [c-like tokenizer](https://github.com/mozsearch/mozsearch/blob/ada3e73856a35052dab9b8ffcfc473950d132e6a/tools/src/tokenize.rs#L170) doesn't really understand backtick strings if that's not enabled.

It seems that kotlin's use of backticks is for escaping [sketchy identifiers](https://kotlinlang.org/spec/syntax-and-grammar.html#identifiers) so the additional state machine that `backtick_strings` enables which exists for JS backtick strings might not be desirable to turn on.  Probably the most  pragmatic thing would be to do something like the `rust_tweaks` mechanism.

That said, there's an open enhancement bug to move to just using tree-sitter-kotlin as our tokenizer and https://github.com/mozsearch/mozsearch/blob/master/tools/src/tree_sitter_support/cst_tokenizer.rs from the hyperblame enhancement work has a building block of that in place.

The [kotlin tokenizer spec](https://searchfox.org/mozilla-central/rev/09b59a7de6848639f27297888889a06e97561074/mobile/android/android-components/components/feature/customtabs/src/test/java/mozilla/components/feature/customtabs/CustomTabsToolbarFeatureTest.kt#246) doesn't seem to set [backtick_strings](https://github.com/mozsearch/mozsearch/blob/ada3e73856a35052dab9b8ffcfc473950d132e6a/tools/src/languages.rs#L11) to true, and it seems like perhaps the [c-like tokenizer](https://github.com/mozsearch/mozsearch/blob/ada3e73856a35052dab9b8ffcfc473950d132e6a/tools/src/tokenize.rs#L170) doesn't really understand backtick strings if that's not enabled.

It seems that kotlin's use of backticks is for escaping [sketchy identifiers](https://kotlinlang.org/spec/syntax-and-grammar.html#identifiers) so the additional state machine that `backtick_strings` enables which exists for JS backtick strings might not be desirable to turn on.  Probably the most  pragmatic thing would be to do something like the `rust_tweaks` mechanism.

That said, there's an open enhancement bug to move to just using tree-sitter as our tokenizer and https://github.com/mozsearch/mozsearch/blob/master/tools/src/tree_sitter_support/cst_tokenizer.rs from the hyperblame enhancement work has a building block of that in place.