Closed Bug 1580621 Opened 6 years ago Closed 5 years ago

Add touch counts of properties to main_summary

Categories

(Toolkit :: Telemetry, task, P3)

Product:
Toolkit
Component:
Telemetry
Type:
task

Tracking

()

Status:
RESOLVED INACTIVE

People

(Reporter: xluo, Unassigned, NeedInfo)

Details

We would like to add to main_summary counts that user touches whitelist of properties for (respectively)

  • lockwise
  • send
  • monitor
  • pocket
  • sync
  • FxA
  • FPN
  • moz.org

Hi Xuan,

Are these touch counts available in Main Pings already? If not, we will need to instrument these touch points on the client side. If so, where can they be found?

Flags: needinfo?(xluo)
Flags: needinfo?(xluo)

AFAIK (95% confident here) that we do not record these touches on the client currently.

This would involve setting a measurement in the client whenever a user navigated to one of the mozilla-owned web properties above. For example, a keyed histogram that would look like { monitor: 1, lockwise: 0, FPN: 1 ... } if the client visited monitor.firefox.com and private-network.firefox.com during a (sub)session.

Does this sound right to you :xluo?

I don't know if there is any precedent for us doing something like this. It seems like it would certainly be category 3 data at least.

Yeah, it makes perfect sense to me. Thank you!

(In reply to Leif Oines [:loines] from comment #2)

AFAIK (95% confident here) that we do not record these touches on the client currently.

This would involve setting a measurement in the client whenever a user navigated to one of the mozilla-owned web properties above. For example, a keyed histogram that would look like { monitor: 1, lockwise: 0, FPN: 1 ... } if the client visited monitor.firefox.com and private-network.firefox.com during a (sub)session.

Does this sound right to you :xluo?

I don't know if there is any precedent for us doing something like this. It seems like it would certainly be category 3 data at least.

ni? :chutten in his role as Data Steward for precedent on collecting this kind of data.

As I understand it, this would need to be opt-in data collection.

Flags: needinfo?(chutten)

(that would be a keyed Scalar, not a keyed Histogram, but...)

I'm not clear on what the planned collection is. If it is a pageload count of mozilla-owned web properties as Leif describes in comment#2, then it is Category 3 Web Activity data and cannot be collected (default on) in release without mitigating the privacy risk it poses to users. That they are mozilla-owned might be sufficient mitigation, but might not be (that is a question for Trust (agray), not stewardship).

But we might be going too fast here. What are the questions we hope to answer with this data? Maybe we can answer those questions in a different way, and maybe we already have the data for that.

Flags: needinfo?(chutten) → needinfo?(xluo)

By collecting these data, we should have a better understanding of each client's interest in Mozilla's products and services. It may help us in following aspects:

  1. develop an efficient approach to connect with each client - starting with the product/service that the client viewed the most in the past 7 days?
  2. discover potential product/service that client might benefit from - among a group of clients sharing similar behavior on Mozilla-owned properties, what is the product that a client didn't touch while majority others used already?
  3. ... many other possibilities

(In reply to Chris H-C :chutten from comment #5)

(that would be a keyed Scalar, not a keyed Histogram, but...)

I'm not clear on what the planned collection is. If it is a pageload count of mozilla-owned web properties as Leif describes in comment#2, then it is Category 3 Web Activity data and cannot be collected (default on) in release without mitigating the privacy risk it poses to users. That they are mozilla-owned might be sufficient mitigation, but might not be (that is a question for Trust (agray), not stewardship).

But we might be going too fast here. What are the questions we hope to answer with this data? Maybe we can answer those questions in a different way, and maybe we already have the data for that.

Flags: needinfo?(xluo)

This certainly does sound like there's no other way but Category 3 data to answer these sorts of questions. A categorical histogram where each category is one of the predetermined list of mozilla web properties would be the best choice for recording and reporting the data.

Whether we're allowed to do this as a part of the usual Telemetry opt-in or not is up to Trust. ni?agray for Category 3 data collection consult.

Flags: needinfo?(agray)

What are we planning on doing with this data collection? Comment 6 includes this information but it's not clear how this would actually be used. Is this to direct marketing activities or outreach? What are the "many other possibilities"?

By collecting these data, we should have a better understanding of each client's interest in Mozilla's products and services. It may help us in following aspects:

develop an efficient approach to connect with each client - starting with the product/service that the client viewed the most in the past 7 days?
discover potential product/service that client might benefit from - among a group of clients sharing similar behavior on Mozilla-owned properties, what is the product that a client didn't touch while majority others used already?
... many other possibilities
Flags: needinfo?(agray)

Here's one example of how we might think about this data - -

STORY - -
As a <PM/MM> I'd like to segment our desktop user based by the dimensions of time and product/service touch in order to better target owned media w/ the hope of increasing conversion rates (and therefore FxA MAU). Downstream, I'd like to take the same approach for subscription services.

From my perspective this is only slightly different form the way we treat ad clicks (count of times a user clicked a Google ad).

(In reply to Daniel Zielaski from comment #9)

Here's one example of how we might think about this data - -

STORY - -
As a <PM/MM> I'd like to segment our desktop user based by the dimensions of time and product/service touch in order to better target owned media w/ the hope of increasing conversion rates (and therefore FxA MAU). Downstream, I'd like to take the same approach for subscription services.

From my perspective this is only slightly different form the way we treat ad clicks (count of times a user clicked a Google ad).

HI Daniel,

I'll set up a call for us to talk through.

Status update: Review remains in progress. Trust will provide final review decision/related information by EOD 10/29.

We can give a Trust approval for data collection as default opt-in on Mozilla properties, but would like to see clear definition of exactly what this collection is, where it needs to go, how long the collection is intended to be, and the use cases before final signoff.

For instance, can we keep this collection on the client or does the message server need the data? Based on my discussion with Daniel, the initial goals are:

  1. Ability to stop surfacing ads/messaging for folks who are already using a Mozilla service. For example, if I'm an active Firefox Monitor user, there is no need to show me messaging to encourage me to use Firefox Monitor.

  2. Ability to get messaging back in front of users who have not fully engaged with a service, but may have landed on a particular page and abandoned it or shown a transient interest. For instance, If I click on Firefox Monitor set-up but don't finish, what do we resurface and how do we do that?

  3. If I sign into a Mozilla service, how did that user get there? For example, if I sign into Monitor, where did I come from?

NI'ing xluo: has your team talked to Kate Hudson who works on the messaging system to see if any of what you want to do is already measured/collected in way that could be useful for your needs?

Flags: needinfo?(xluo)

Bumping over to Telemetry component as this seems more like an instrumentation thing than a dataset thing.

Component: Datasets: Main Summary → Telemetry
Product: Data Platform and Tools → Toolkit

The instrumentation will need to be written in a component that knows something about the pages being loaded, but Telemetry's happy to host the discussion between here and there.

Priority: -- → P3

Closing this bug as inactive. Use of telemetry for marketing purposes requires a policy level decision before any telemetry instrumentation/use can take place. Contact Alicia Gray or Marshall Erwin with questions.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.