KPI Dashboard: count recurring contributors when displaying active forum contributors numbers

RESOLVED FIXED in 2014Q3

Status

support.mozilla.org
General
P1
normal
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: madalina, Assigned: rrosario)

Tracking

unspecified
2014Q3

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: u=sumo-team c=kpidash p=1 s=2014.17)

(Reporter)

Description

3 years ago
The KPI dashboard is displaying the number of active forum contributors per day. If we change to weekly or monthly the numbers are being added up without taking into consideration that we might be dealing with the same people.

Example: we have 50 contributors per day, if we change to weekly we will get a number of 350 contributors although we might have the same 50 people contributing every day. This makes difficult to see what is the real number of contributors that are actually active.

We need to reset the dashboard to show weekly and monthly numbers by taking into consideration recurring contributors.
I could swear there was a bug for this already, but I can't find it.

The idea – I'm assuming – is using something like the average for a week or a month instead of aggregating.
Priority: -- → P1
Whiteboard: u=sumo-team c=kpidash p= s=2014.17
Target Milestone: --- → 2014Q3
(Assignee)

Comment 2

3 years ago
I'm pretty sure this is just a bucketing option we have to change when creating the graph.
Whiteboard: u=sumo-team c=kpidash p= s=2014.17 → u=sumo-team c=kpidash p=1 s=2014.17
I don't think this is just a bucketing option. This is a change to what metrics we are measuring, and the graphing "library" we've built up probably wont' deal with that very well.

We could simple average the values, and that would be simple, but to actually count the weekly visitors on the graph would be quite a bit of re-work in the graphing code. This is at least 2 points, if not more.
Whiteboard: u=sumo-team c=kpidash p=1 s=2014.17 → u=sumo-team c=kpidash p=2 s=2014.17
Taking this bug.
Status: NEW → ASSIGNED
QA Contact: djohnson
Whiteboard: u=sumo-team c=kpidash p=2 s=2014.17 → u=sumo-team c=kpidash p=1 s=2014.17
Fixing points and QA contact.
Assignee: nobody → djohnson
QA Contact: djohnson
Whiteboard: u=sumo-team c=kpidash p=1 s=2014.17 → u=sumo-team c=kpidash p=2 s=2014.17
(Assignee)

Comment 6

3 years ago
(In reply to Mike Cooper [:mythmon] from comment #3)
> We could simple average the values, and that would be simple, but to
> actually count the weekly visitors on the graph would be quite a bit of
> re-work in the graphing code. This is at least 2 points, if not more.


Yeah, I think we just want to average the daily numbers instead of sum them. Is that good enough, Kadir?
Flags: needinfo?(a.topal)
Yeah, I think we are really interested in the daily numbers, no point in counting those who come to SUMO once per month for a day. With that assumption, averages would make more sense.

This is up to you though, Madalina, Michael. Are you okay with averages?
Flags: needinfo?(mdziewonski)
Flags: needinfo?(mana)
Flags: needinfo?(a.topal)
So, seeing as I've only got two days left on my internship after today, I doubt I'll have the time to complete this bug. That being said, after some review of how we currently do things for that page, and how metrics are aggregated, I can provide some insight into how things currently work and a possible solution.

Problem: Unique users are not counted with aggregated data on the graphs. The same 50 users may occur every day for a single week, misleading us to believe we have 350 users contributing per week.

Current Aggregation Solution: There is a cron job that runs once a day that counts up the number of unique contributors by distinct ids, then stores that number in the 'kpi_metrics' table with the type of metric, start date of gathering metric data and end date of gathering metric data. 

Possible Solution: Immediately after creating the metric, store each user's ID in a new independent model called UserMetric that refers straight back to the newly created metric. Then, when the server is setting up the response with all the metric data, do a join on the MetricID with each UserMetric so you can get unique IDs for users as well. Now, when the client receives the data on the front-end, it can process data from any time range, and refer to each day's metric of unique user ids for an accurate count of the unique contributors for a specific time period.
Status: ASSIGNED → NEW

Updated

3 years ago
Assignee: djohnson → nobody

Comment 9

3 years ago
+1 for averages from my side, if that's the simpler solution.

Dean's idea sounds great, but I know you may be stretched for manpower and time.
Flags: needinfo?(mdziewonski)
We don't have to rush this, what's really important is what you guys need to do your job. 

* Averaging would give you unique daily contributors (eg. on average how many forum contributors did we have on any given day in August). 

* Aggregation would give you unique weekly or monthly contributors (eg. How many forum contributors did we have in August)

I guess the metric we want depends a lot on what we want to optimize for.
(Assignee)

Comment 11

3 years ago
I think there is some confusion here. Those numbers aren't even daily unique contributors. They are more like monthly unique contributors. Here is the definition right below the chart:

Number of active forum contributors
Definition: The number of people who have contributed to the forums in the last 30 days.
Definition of contribution: wrote 10+ posts


Honestly, the bucketing doesn't even make any sense. I think we should just remove it.
(Assignee)

Comment 12

3 years ago
We decided at the last platform meeting that we would remove the bucketing option from that chart. Reducing to 1pt
Assignee: nobody → rrosario
Flags: needinfo?(mana)
Whiteboard: u=sumo-team c=kpidash p=2 s=2014.17 → u=sumo-team c=kpidash p=1 s=2014.17
(Assignee)

Comment 13

3 years ago
In a pull request:
https://github.com/mozilla/kitsune/pull/2133
(Assignee)

Comment 14

3 years ago
Landed on master:
https://github.com/mozilla/kitsune/commit/58a36ac0955109f3e1bbf86fb0f11cbfbb08fab5
(Assignee)

Comment 15

3 years ago
Deployed to prod.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.