598162 - Download counts have dropped significantly since 09/12

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Statistics, defect, P1)

Description

[Jorge Villalobos [:jorgev] (he/him)]

An add-on author reported this and it looks consistent with many add-ons:

https://addons.mozilla.org/en-US/statistics/addon/1865
https://addons.mozilla.org/en-US/statistics/addon/7684
https://addons.mozilla.org/en-US/statistics/addon/6622

Comment 1

[Daniel Einspanjer [:dre] [:deinspanjer]]

There is a noticeable drop in downloads after the 15th.  Most of it from the Addons Manager src.  Were there any changes in the format of those URLs?  Was there an AMO upgrade on the 15th?

I have an IT bug open to get some mount points configured on our research hadoop cluster so I can pull in some recent AMO logs and do some grepping.

Comment 2

[Wil Clouser [:clouserw]]

We had a push on the 2nd and a small push on the 14th.

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

'k, I'd point a strong finger at the 14th.  Could you dig up any potential changes that might have been made to the format of addon download requests?

Comment 4

[Wil Clouser [:clouserw]]

You're talking about something that would change the format of the links with .xpi in them?  We haven't changed that script yet (although it's on the list soon).

The push on the 14th was mainly for the discovery pane.  http://github.com/jbalogh/zamboni/compare/5.11.9.4...5.11.10

Comment 5

[Nick Nguyen [:osunick]]

This is still pretty important, any updates?

Comment 6

[Daniel Einspanjer [:dre] [:deinspanjer]]

I don't have an update on this today, we spent the entire day yesterday and today fighting a huge Socorro fire.  I'll take a look at this over the weekend and also check with Anurag to see what the status is with the hive stuff.

Comment 7

[Daniel Einspanjer [:dre] [:deinspanjer]]

IT hasn't managed to close out that bug yet so I went back and did some by-hand grepping.  Something significant changed on the 14th in regards to the src=api downloads.  There is a spot check from just noon of each day before and after:

$ for d in {12..16}; do echo 2010-09-$d-12; zcat access_2010-09-$d-12* | zgrep 'downloads/file/[^/]*/adblock' | grep -o 'src=[^& ]*' | sort | uniq -c; done
2010-09-12-12
     20 src=addondetail
     60 src=addon-detail-version
   2212 src=api
      2 src=collection
     30 src=search
     16 src=version-history
2010-09-13-12
     36 src=addondetail
     50 src=addon-detail-version
   4925 src=api
     34 src=search
      3 src=sharingapi
     23 src=version-history
2010-09-14-12
     16 src=addondetail
     68 src=addon-detail-version
   4888 src=api
      2 src=api"
      4 src=category
      4 src=oftenusedwith
     58 src=search
     17 src=version-history
2010-09-15-12
     76 src=addondetail
     80 src=addon-detail-version
      2 src=api
     50 src=search
      7 src=sharingapi
     64 src=version-history
2010-09-16-12
     56 src=addondetail
     97 src=addon-detail-version
      2 src=discovery-pane"
     52 src=search
      4 src=sharingapi
     52 src=version-history

Comment 8

[Justin Scott [:fligtar]]

Looks like this bug wasn't updated, but it seems like bug 599883 is the cause of this. I think we'll need to backfill logs from that subdomain for the affected range.

Comment 9

[Wil Clouser [:clouserw]]

(In reply to comment #8)
> Looks like this bug wasn't updated, but it seems like bug 599883 is the cause
> of this. I think we'll need to backfill logs from that subdomain for the
> affected range.

Yep, that's bug 599877

Comment 10

[Justin Scott [:fligtar]]

I blogged about this since we're still getting a lot of questions.

http://blog.mozilla.com/addons/2010/09/28/problems-with-add-on-statistics/

Comment 11

[Wil Clouser [:clouserw]]

Tracking appears to have recovered; not much more we can do here.