Closed Bug 1111265 Opened 8 years ago Closed 8 years ago

don't ask for browser data when it's not appropriate

Categories

(Input Graveyard :: Submission, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

Details

(Whiteboard: u=user c=feedback p=2 s=input.2015q1)

Bug #1104935 adds form fields and code to collect remote-troubleshooting data in the generic feedback form. It currently requires the user to have the en-US locale since we don't have the strings translated. That will get lifted.

However, we probably don't want to collect browser data unless the user is using the product that the user is leaving feedback for. e.g.:

1. user is using Firefox and leaving feedback for Firefox

2. user is using Firefox for Android and leaving feedback for Firefox for Android

3. user is using Hello on Firefox and leaving feedback for Hello on Firefox

4. etc.


This bug covers the following things:

1. figure out asap how often users are using Firefox for Android to leave Firefox feedback or vice-versa (those are probably the two problem scenarios)

2. if it's significant (for some hand-wavey threshold), then figure out how to limit that section of the form to only show in "appropriate" scenarios
Grabbing this now to work on on Monday. I think I can skim a db dump focusing on the last couple of months and see when browser and product don't match vs. when they do match to get a feel for how often this happens. Then I can figure out whether this blocks bug #1104935 from going to production.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
I ran this on a fresh db import:

def run(*args):
    recent_responses = (
        Response.objects
        .exclude(api=1)
        .filter(created__gte=datetime.now() - timedelta(days=60))
    )

    # Figure out number of responses for product Firefox and Firefox
    # for Android in the last 2 months.
    total = (
        recent_responses
        .filter(Q(product=u'Firefox') | Q(product=u'Firefox for Android'))
    )

    print 'Total:', total.count()

    # Figure out number of responses for product Firefox which have
    # browser as Firefox for Android in last two months.
    fx_mismatch = (
        recent_responses
        .filter(product=u'Firefox')
        .filter(browser=u'Android')
    )
    print 'Fx mismatch:', fx_mismatch.count()

    # Figure out number of responses for product Firefox for Android
    # which have browser as Firefox in last two months.
    fx_mismatch = (
        recent_responses
        .filter(product=u'Android')
        .filter(browser=u'Firefox')
    )
    print 'FxA mismatch:', fx_mismatch.count()


Pretty sure that's right. It prints out:

Total: 82323
Fx mismatch: 0
FxA mismatch: 0


Most Firefox for Android feedback comes over the API so those users aren't seeing our generic feedback form at all. I figured at least some of the people coming to the generic feedback form are leaving feedback for things other than Firefox, but maybe not.

Still, I'm a little surprised those numbers are 0. Does that code look right? I thought they'd be small non-zero numbers.
Flags: needinfo?(rrosario)
Ricky pointed out I could use another cup of coffee this morning before I start writing code.

I'm getting these numbers now:

Total: 82266
Fx mismatch: 60
FxA mismatch: 21834

That makes a lot more sense.

There are some complications here, though. First is that we infer browser="Firefox" browser_platform="Android" for Firefox for Android and that doesn't easily match the product nor is it easy to write general matching rules.

I think what I'm going to do is just restrict this remote-troubleshooting stuff to "Firefox" and "Firefox dev" products and defer figuring this out for realz until later.
Flags: needinfo?(rrosario)
Alleviating the problem in https://github.com/mozilla/fjord/pull/414

We'll do a real/better fix later when:

1. I have more time to think about it,
2. I have a good solution.
Landed in master in https://github.com/mozilla/fjord/commit/8588b429b8600aba29aa801cae77c72f614c4e73

Making this block bug #1104935. We need to fix this before we can call that done.
Blocks: 1104935
Blocks: 1104932
No longer blocks: 1104935
Bumping this to 2015q1 because we're out of time for this quarter.
Whiteboard: u=user c=feedback p= s=input.2014q4 → u=user c=feedback p= s=input.2015q1
I think we should add another field to the products table which is essentially "Grab browser data for browser: []".

If the field is empty, then we don't show browser data ask and we don't collect anything.

If the field is non-empty and that value matches the user agent inferred browser, then we show the browser data ask and we try to collect browser data if the user opts in.

Then we set "Firefox" for all the Firefox-related products and "Android" for the "Firefox for Android" related products.

Plus we should add a method to Product called something like "collect_browserdata" which figures out whether to do it or not.

Then we need to write some tests to make sure this is all working and continues to work later when/if we change the user agent parser.
Whiteboard: u=user c=feedback p= s=input.2015q1 → u=user c=feedback p=2 s=input.2015q1
Pushed to production. Also, I updated the products in production that should be collecting Firefox browser data and I verified they're working correctly.

Marking as FIXED.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
This PR adds the browser_data_browser field to the admin and product forms: https://github.com/mozilla/fjord/pull/473
Product: Input → Input Graveyard
You need to log in before you can comment on or make changes to this bug.