1223625 - differentiate between in-product feedback and Input-hosted feedback for Firefox for Android

Status: RESOLVED FIXED

(Input :: General, defect)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

We need to know how much feedback we're getting from the in-product form vs. the generic feedback form on Input.

One possibility is to do statsd pings, but ew.

Another possibility is to take advantage of the fact that in-product form doesn't send along a utm_source and give it one.

This bug covers figuring out a good path, implementing it and throwing together a dashboard using it so we can see what's going on.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to work on soon.

Further, we want to make sure this is available via the histograms api otherwise it's hard to see what's going on.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

So... it looks like fennec is using the API to post feedback. I had no idea. Either that or I did know and completely forgot.

https://dxr.mozilla.org/mozilla-central/source/mobile/android/app/mobile.js#462
https://dxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/aboutFeedback.js

Further, they tack on a source:

https://dxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/aboutFeedback.js#148

which is set to feedback-prompt if the user was prompted to give feedback:

https://dxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/Feedback.js#15

If the user was not prompted to give feedback, then there's no source.

So we have the following:

1. api=1 AND product=Firefox for Android AND source=feedback-prompt == Fennec in-product form and the user was prompted

2. api=1 AND product=Firefox for Android == Fennec in-product form organic feedback

3. api=None AND product=Firefox for Android == Input generic form or the old Fennec feedback form


The feedback api (GET /api/v1/feedback) shows "source", "product" and "api" fields, so we could set up a dashboard that figures out the numbers over time and graphs them. That's kind of painful, but doable.

Also, the bulk of our feedback comes from the current version and the previous version and then very rapidly trails off from there. Version 34 is the most popular of the old versions.

https://input.mozilla.org/en-US/analytics/search?product=Firefox%20for%20Android&version=34.0


Given all that, I don't think we should make any code changes here. Instead, we should just set up a dashboard that shows items 1, 2 and 3 against each other over time.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I added some code to tag old fennec feedback with source="oldfennec-in-product". That'll let us differentiate item 3. So now we have product=Firefox for Android:

1. api=1 AND source=feedback-prompt == Fennec in-product form and the user was prompted

2. api=1 == Fennec in-product form organic feedback

3. api=None AND source=oldfennec-in-product == the old Fennec feedback form

4. api=None == Input generic form

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

In a PR: https://github.com/mozilla/fjord/pull/708

Landed in https://github.com/mozilla/fjord/commit/b5e92d85f309dc17fc21017eb2d8cfea9b5ba2a3

Further, I added "source" and "api" as params to the secret histogram API so we can query these things.

In a PR: https://github.com/mozilla/fjord/pull/710

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Landed in https://github.com/mozilla/fjord/commit/1cb4aa3c3e0932c586cdd2c4ee3b6b9974a6976a

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pushed to prod just now.