1835321 - Change the logic around categorizing non ad clicks to be less restrictive

Status: RESOLVED FIXED

(Firefox :: Search, task, P1)

Description

[James Teow [:jteow]]

We're not accurately registering clicks on non-ads on Google.

In SearchSERPTelemetry, we observe network events and check if a channel is a top level load and is a document. This to avoid parsing requests that are unlikely to request a change of the location of a tracked browser.

This seemingly works for ad links and non ad clicks for most SERPs. However, on clicking non-ad links on Google that don't open a new tab, the one time we experience a channel that is is both a top level load and is a document, wrappedChannel.browserElement returns null.

This is annoying because we are using browsers to determine whether the request belongs to a source that we care about.

The webidl comment mentions "for requests that don't originate from a remote browser, this is null."

Comment 1

[James Teow [:jteow]]

More descriptive bug title.

Comment 2

[James Teow [:jteow]]

Attachment: Bug 1835321 - Allow any link on SERPs that don't match an ad expression to be considered a non-ad - r?standard8,scunnane

Comment 3

[James Teow [:jteow]]

I've changed the gist of this bug to be about being less restrictive about the categorization of non ad link clicks.

I wrongly made the assumption SERPs are consistent about encoding non ad results with redirects. Bing for example has the ability to enable/disable links to open in new tabs. And Google loads a link in a subframe first before changing the location of the page which throws a wrench in the assumption that only channels that are top level and containing a document should be inspected.

That said, I think we can be safely assume any traffic that a) has an originURL closely matching a SERP, b) is a top level load (except for links that match a nonAdRegexp), c) is a document, should be a click event, and d) originates from a browser that is currently being tracked, is a click event.

The tricky part is to ensure that any redirects are recorded once.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74657e032184
Allow any link on SERPs that don't match an ad expression to be considered a non-ad - r=Standard8

Comment 5

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/74657e032184