675810 - what's different about the new compatibility numbers?

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Compatibility Tools, defect, P1)

Description

[Justin Scott [:fligtar]]

https://addons.allizom.org/z/en-US/firefox/compatibility/5.0?appver=1-5.0&type=all shows 72% and https://addons.allizom.org/en-US/firefox/compatibility/dashboard/5.0 shows 84%.

I'd like to understand what the difference between them is and make sure it's correct. Lots of people look at these numbers and they are monitored for changes every week.

Comment 1

[Jeff Balogh (:jbalogh)]

I fixed some bugs but my number is still higher. I think this is because my top_95 number is for the whole app, not for the app + version. When counting Firefox 5.0 compat PHP discards add-ons that aren't at least compatible with Fx 4.0 and figures out the top 95% of add-ons from there. Python figures out the top 95% of add-ons without discarding anything, so it sees a longer tail.

https://github.com/jbalogh/zamboni/commit/e97ace1 
https://github.com/jbalogh/zamboni/commit/3714d9e

Comment 2

[Justin Scott [:fligtar]]

I don't think I understand. The number I'm reading specifically says "Top 95% compatible with previous version", so why would it not be looking at 4.0 compatible add-ons?

Comment 3

[Jeff Balogh (:jbalogh)]

(In reply to Justin Scott [:fligtar] from comment #2)
> I don't think I understand. The number I'm reading specifically says "Top
> 95% compatible with previous version", so why would it not be looking at 4.0
> compatible add-ons?

Because a global top_95 number is easier to generate with all the other filtering we're doing.

Comment 4

[Justin Scott [:fligtar]]

(In reply to Jeff Balogh (:jbalogh) from comment #3)
> (In reply to Justin Scott [:fligtar] from comment #2)
> > I don't think I understand. The number I'm reading specifically says "Top
> > 95% compatible with previous version", so why would it not be looking at 4.0
> > compatible add-ons?
> 
> Because a global top_95 number is easier to generate with all the other
> filtering we're doing.

I still don't understand. You're taking the top 95%, then removing the ones that don't meet the labels, then calculating the percentage based on that smaller percentage?

The 4 columns of each row should add up to 100%, and they don't.

Comment 5

[Jeff Balogh (:jbalogh)]

Once all the stats are in elastic I can calculate this properly.

Comment 6

[Jeff Balogh (:jbalogh)]

It's hard to tell for sure since I can't see the PHP on -dev, but I think I have this all cleared up now. https://addons-dev.allizom.org/en-US/firefox/compatibility/6.0 is showing close to 100%; 7.0 is different from prod since the dbs are drifting.

https://github.com/jbalogh/zamboni/commit/0cdfc10 
https://github.com/jbalogh/zamboni/commit/b682213