1506664 - browser.engagement.total_uri_count includes same_document changes as 'real' uri changes, which means we count a single duckduckgo search result as 2 visited uris

Status: RESOLVED INVALID

(Firefox :: General, enhancement)

Description

[Mike Kaply [:mkaply]]

If you do a search via DuckDuckGo in the one off searches, you'll see

 browser.engagement.total_uri_count

Incremented by 2. This is because BrowserUsageTelemetry counts every URL change (even if LOCATION_CHANGE_SAME_DOCUMENT is true).

In the DuckDuckGo case, they said: 

they are client-side changes to the URL using the browser's history API. That's how we support history state on the SERP which allows us to make the back button work as user's expect it to as they navigate the SERP.

So the question is, are we counting more URLs than we should be?

Comment 1

[Alessio Placitelli [:Dexter]]

Moving this to Firefox:General, as we're keeping the Telemetry component for the collection system rather than the probes themselves :)

@Ryan, do you know who this should be escalated to?

Comment 2

[:Gijs (he/him)]

I'm pretty confused by this bug - the metric is unrelated to search, AFAICS, and we pretty clearly need to count SAME_DOCUMENT changes to the URI because e.g. youtube uses this for all internal links (e.g. going from one video to the next).

I'd say this is wontfix. Mike, can you maybe elaborate on why this is a serious problem for duckduckgo specifically?

Comment 3

[Mike Kaply [:mkaply]]

(In reply to :Gijs (he/him) from comment #2)
> I'm pretty confused by this bug - the metric is unrelated to search, AFAICS,
> and we pretty clearly need to count SAME_DOCUMENT changes to the URI because
> e.g. youtube uses this for all internal links (e.g. going from one video to
> the next).
> 
> I'd say this is wontfix. Mike, can you maybe elaborate on why this is a
> serious problem for duckduckgo specifically?

I was just using DuckDuckGo as an example. My concern is that there are some cases where we are counting URLs as unique when they actually aren't.

This happens with history manipulation as well as anchor tags.

For instance clicking internal links in https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIFile, browser.engagement.total_uri_count is incremented with each click.

If this is what we're supposed to do, I'm fine with that, I just think we might want to document it somewhere.

Comment 4

[Ryan Harter [:harter]]

Unfortunately, I don't have a great read on who to escalate to. It only looks tangentially related to search. The question seems to be whether our URI count probe is collecting something reasonable.

Comment 5

[:Gijs (he/him)]

(In reply to Mike Kaply [:mkaply] from comment #3)
> I was just using DuckDuckGo as an example. My concern is that there are some
> cases where we are counting URLs as unique when they actually aren't.

The semantics of whether 2 URLs are unique or the same aren't well-defined. https://example.org/foo/ and https://example.org/bar/ can point to the same page and be history.pushState changes, or be completely different pages. Ditto for hash changes. Other pages load 1 document and continuously update the .hash of document.location for no reason.

I think it's reasonable to err on the side of considering these separate pages for now, but it's still unclear why/how this affects DDG and why we needed to contact them about it...