697362 - Minor application versions aren't being displayed

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Statistics, defect, P3)

Description

[Wladimir Palant]

Please go to https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/statistics/usage/applications/?last=365 and switch to daily view. You can clearly see that something is wrong - there is a large gap between Firefox 4 and Firefox 5, also between Firefox 6 and Firefox 7. It seems that the graph only shows Firefox 4.0 and Firefox 6.0 but the numbers for Firefox 4.0.1 or Firefox 6.0.1 are simply dropped and not displayed anywhere.

Comment 1

[Potch [:potch]]

The new code allows me to change version grouping precision- I'll expose that in the UI.

Comment 2

[Potch [:potch]]

It also appears that the precision grouping is causing some data to get dropped (As you've noticed). I'm looking into that now.

Comment 3

[Wladimir Palant]

(In reply to Potch [:potch] from comment #1)
> The new code allows me to change version grouping precision- I'll expose
> that in the UI.

This probably belongs into bug 697364.

Comment 4

[Wil Clouser [:clouserw]]

Thanks for the bugs Wladimir.  Dropping data is important -> 6.3.0

Comment 5

[Jeff Balogh (:jbalogh)]

Attachment: 3.6.* updates

This is due to https://github.com/jbalogh/zamboni/blob/master/apps/stats/search.py#L56. It tries to make sure we only collect stats for app/version combinations we know about, but that doesn't take into account all the minor versions that get collected under something like 7.0.*.

Collecting minor versions under 3.6.* seems like a good compromise unless we want every version number ever to get into the stats dashboard. Attached is the 3.6 swarm for adblock plus updates.

Comment 6

[Potch [:potch]]

do you mean *not* collecting the minor versions for 4.0 and up?

Comment 8

[Wladimir Palant]

(In reply to Jeff Balogh (:jbalogh) from comment #5)
> Collecting minor versions under 3.6.* seems like a good compromise unless we
> want every version number ever to get into the stats dashboard.

As long as I can still access the data for "every version number ever". Current occasion - next Adblock Plus version will change minVersion to Firefox 3.6.13 (up to 3.6.12 there was some Gecko bug, work-around for this bug has been removed now). When making such decisions it is useful to learn how many people didn't update to the latest minor release.

Comment 9

[Potch [:potch]]

Wil,
What are the next steps for filling in this data? I'm not sure how to proceed.

Comment 10

[Wil Clouser [:clouserw]]

(In reply to Wladimir Palant from comment #8)
> (In reply to Jeff Balogh (:jbalogh) from comment #5)
> > Collecting minor versions under 3.6.* seems like a good compromise unless we
> > want every version number ever to get into the stats dashboard.
> 
> As long as I can still access the data for "every version number ever".
> Current occasion - next Adblock Plus version will change minVersion to
> Firefox 3.6.13 (up to 3.6.12 there was some Gecko bug, work-around for this
> bug has been removed now). When making such decisions it is useful to learn
> how many people didn't update to the latest minor release.

Is that information in the CSVs right now?  We could expose it there if not.

Comment 11

[Potch [:potch]]

The CSVs are not presently reporting data for these minor versions- the data is simply not in the index at all.

Comment 12

[Wil Clouser [:clouserw]]

Looks like the code Jeff linked to in comment 5 is skipping the minor versions in the index.  We'll need to adjust that before you can do the front end.

Comment 13

[Potch [:potch]]

After discussing it with Wil, I think we're going to remove the requirement that we index only known app/version pairs (too much maintenance and risk) and filter based on "compliant" looking versions. This may involve using some of the VersionCompare logic to verify a version string is in a form we expect and isn't "6.0.yourmomlulz"

Comment 14

[Simon Paquet [:sipaq]]

As long as this takes, can you please revert to the old statistics code, because as it is, the application breakdown is totally useless at this point.

And why aren't you just filtering for the first first two parts of a major.minor.bugfix release number? That would get you releases like Firefox 8.0, but also Firefox 3.6 or SeaMonkey 2.4.

May I also ask, whether any QA was performed before this went into production? Because if someone had done a simple 2-second before-after analysis with a single popular add-on (e.g. Adblock Plus for Firefox or Lightning for Thunderbird). This would have been caught early on.

Comment 15

[krupa raj[:krupa]]

Simon,

The stats pages were indeed QA'ed before they were made live. Data on staging server is very different from Prod and it is not easy to discern content-related issues.

This bug was identified before we made the pages live but we went ahead with the push in the opinion that the pros outweigh the con. This may have been an oversight on our behalf and I apologize for any inconvenience it may have caused.

We are working on fixing this problem. We appreciate your patience.

Comment 16

[Simon Paquet [:sipaq]]

Thanks for the info, Krupa.

May I then suggest that someone looks into aligning staging and production data for better QA.

Comment 18

[Potch [:potch]]

https://github.com/mozilla/zamboni/commit/240cab4 

I removed the version/app pair check. When this lands in production, we will need to re-index all the stats since Firefox 4's launch to rebuild them accurately.