Closed Bug 564541 Opened 12 years ago Closed 11 years ago

Have your way with the new statistics CSVs

Categories

(addons.mozilla.org Graveyard :: Code Quality, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: clouserw, Assigned: mozwebqa)

References

()

Details

(Whiteboard: [fat-bug-in-da-house])

Our statistics CSVs have been converted to zamboni!  We need to verify that they are working correctly.  These come from all the "View this table in CSV format" on the add-on statistics pages.  This is also what provides the data for the graphs.  As a bonus wrench-in-the-works all the URLs are changing formats too.[1]  

These are all the patterns that will generate statistics output:

^addon/(\d+)/statistics/downloads-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/usage-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/contributions-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/sources-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/os-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/locales-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/statuses-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/versions-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/apps-(day|week|month)-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$
^addon/(\d+)/statistics/contributions-detail-(?P<start>\d{8})-(?P<end>\d{8})\.(json|csv)$

Oh yes.

So, as an example, here is a remora URL for the first pattern:
https://preview.addons.mozilla.org/en-US/thunderbird/statistics/csv/1865/downloads?start=2010-01-01&end=2010-05-07

And it's equivalent with the new format from zamboni:
https://preview.addons.mozilla.org/z/en-US/firefox/addon/1865/statistics/downloads-day-20100101-20100507.csv

And since we're always trying to be a little more awesome, that same URL works if you add .json on the end instead of .csv:
https://preview.addons.mozilla.org/z/en-US/firefox/addon/1865/statistics/downloads-day-20100101-20100507.json

As you can see, there are a lot of combinations here (161000 addons * 10 views * 3 periods * a zillion date combinations * 2 formats).  So, check a few and if you're happy we can catch any minor details later.  It will also be easier once they are hooked up to the visual graphs instead of just numbers.

Also, /contributions/ and /contributions-detail/ shouldn't be available to anyone except people with the right permissions for that add-on.

I'm putting this in 4.x triaged because I think we need to make the CSVs faster before we're ready to push them live (bug 564540) and, although it shouldn't change the output at all, it's worth delaying serious testing until that's ready.  It will be a substantial change to the backend.

[1] Details: http://docs.google.com/Doc?docid=0Acwo2Bn17-PrZGZudHRobnJfMzBkOTZkdnNodg&hl=en
Depends on: 564544
I wrote a new bug for this, and then remembered I already wrote this one.  So, here is the content of my new bug with a couple more URLs:

Example old remora pages:

https://preview.addons.mozilla.org/en-US/statistics/csv/1865/downloads?group_by=date

https://preview.addons.mozilla.org/en-US/statistics/csv/1865/version?group_by=date

Example new Zamboni pages:

https://preview.addons.mozilla.org/z/en-US/firefox/addon/1865/statistics/downloads-day-20100115-20100715.csv

https://preview.addons.mozilla.org/z/en-US/firefox/addon/1865/statistics/versions-day-20100115-20100715.csv

I don't think you need to spend a ton of time on this.
Target Milestone: 4.x (triaged) → 5.11.5
Depends on: 580136
Depends on: 580459
Target Milestone: 5.11.5 → 5.11.7
Target Milestone: 5.11.7 → 5.11.6
Just a reminder that I'd like the new header in bug 564544 (marked as blocking this) to go live with these new CSVs.
Target Milestone: 5.11.6 → 5.12.4
Priority: -- → P1
Target Milestone: 5.12.4 → 5.12.5
All the dependent bugs are closed.  Is there more to do here?
Target Milestone: 5.12.5 → Q2 2011
Wil, QA hasn't looked at the new stats page in a while. I'll close the bug after doing one quick round of QA (by EOW).
Target Milestone: Q2 2011 → 6.0.9
This is looking good. Any new bugs will be filed and tracked separately. Marking this fixed.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.