1334611 - Add a probe to measure location bar query length

Status: NEW

(Firefox :: Address Bar, task, P3)

Description

[Panos Astithas (he/him) [:past] (please ni?)]

Measure the string length when an action was taken (result click, enter, etc.). A histogram seems pertinent for this.

Comment 1

[Marco Bonardo [:mak]]

Is this the user typed string, or the string we put into the urlbar on selection? See for example a search suggestion, on selection we replace the typed text with the search suggestions (we still have the typed value available, clearly).
I'm assuming we want length of the user typed string.

Comment 2

[Panos Astithas (he/him) [:past] (please ni?)]

Yes, this is about what the user typed in the box.

Comment 3

[Mark Banner (:standard8)]

Javaun, what sort of data collection do we want here?

Are we looking at typical ranges of lengths of queries (e.g. 0 - 5, 5 - 10, 10 - 15) or individual values? (0, 1, 2, 3, 4... up to a maximum)?

Would we prefer this to be opt-in or opt-out?

Comment 4

[Javaun Moradi [:javaun]]

Mark, I'd look at individual values (0,1,2,3,4,5... ) up to some max. That metric could be a proxy for many things, like how overall how fast is one-lookup engine vs. another and also whether the user is doing a simple query vs. a more complex/specific one. It will be opt-out, and we should be clear on policy under  the new data review policy. I will have to complete a data review form and will link it from bugs.

Comment 5

[Mark Banner (:standard8)]

Attachment: Bug 1334611 - Add a probe to measure location bar query length.

Review commit: https://reviewboard.mozilla.org/r/115268/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/115268/

Comment 6

[Mark Banner (:standard8)]

Comment on attachment 8840832 [details]
Bug 1334611 - Add a probe to measure location bar query length.

Marco, there's still a few bits that I need to tidy up here, so only requesting feedback at the moment.

Mainly I'd value your comments on:

- Am I getting the query length in a suitable & reliable place?
- Have I correctly covered all of the use cases, or are there more?

Comment 7

[Mark Banner (:standard8)]

Comment on attachment 8840832 [details]
Bug 1334611 - Add a probe to measure location bar query length.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/115268/diff/1-2/

Comment 8

[Marco Bonardo [:mak]]

Comment on attachment 8840832 [details]
Bug 1334611 - Add a probe to measure location bar query length.

https://reviewboard.mozilla.org/r/115268/#review117358

Just to be sure, are we sure we just need an uncorrelated count? My doubt is that this way we don't have correlations between the string length and the kind of result selected. So the only question that this seems to be able to answer is "how many chars users type before selecting any kind of entry", that by itself is a good measure of how good our matching is, but nothing about every single match type. A breakdown by match type would answer the same question, plus many more.

::: browser/modules/BrowserUsageTelemetry.jsm:280
(Diff revision 2)
> +    // Store in telemetry the length of the original query. If this is an
> +    // action with a searchQuery, then we'll use that rather than the input
> +    // field value, as for an action the value of the input will have changed.
> +    let originalQuery = action && action.params && action.params.searchQuery ?
> +                        action.params.searchQuery :
> +                        input.inputField.value;

Here you are in _handleURLBarTelemetry, that is invoked by autocomplete-did-enter-text. sounds like an acceptable point to hook. If we'd ever find issues with this, the alternative may be handleCommand.

I don't think at this point we can use inputField.value, since it's possible autocomplete has modified the user value.
For example your autofill test is searching for the whole string that would be autofilled, rather than a part of it, and it's unlikely to catch a mistake here(you should search for "moc" for example, and telemetry should report 3).

.searchQuery is also not intended to be the string that the user typed, for example in _maybeRestyleSearchMatch (currently a disabled feature) we reverse-parse a search url and extract the search string into .searchQuery, that's not what the user typed. In future we could add more cases like that or have to change this value to solve subtle bugs and we'd mess up telemetry.

I think we need to hook up our own code to collect what the user typed. We have an "onInput" handler in urlbarBindings.xml that may be a good starting point.
We could maybe even reuse gBrowser.userTypedValue (but we should pay attention to capture it before _loadURL overwrites it, should not be a big deal, your patch already acts before that afaict). I think we can't rely on .valueIsTyped since it looks to be set intermittently (probably .userTypedValue should rather be setUserTypedValue(value, isTyped) to make .valueIsTyped reliable).

Comment 9

[Dave Zeber [:dzeber]]

(In reply to Marco Bonardo [::mak] from comment #8)
> Just to be sure, are we sure we just need an uncorrelated count? My doubt is
> that this way we don't have correlations between the string length and the
> kind of result selected. So the only question that this seems to be able to
> answer is "how many chars users type before selecting any kind of entry",
> that by itself is a good measure of how good our matching is, but nothing
> about every single match type. A breakdown by match type would answer the
> same question, plus many more.

I agree that to maximize utility we'd really want to have this metric broken down by match type. Could we make this into a keyed histogram, maybe using the URLBAR_SELECTED_RESULT_TYPES (https://dxr.mozilla.org/mozilla-central/source/browser/modules/BrowserUsageTelemetry.jsm#56) as the keys?

Comment 10

[Mark Banner (:standard8)]

Javaun, can you comment on comments 8 & 9?

Comment 11

[Javaun Moradi [:javaun]]

Sorry Mark! Dave and I chatted about this one and he responded. We do want the counts by match type, the uncorrelated account is much less useful. The question is how much work it'll take to bucket counts by type...