559385 - [Fx4Launch] Please fix live download counter (total count)

Status: VERIFIED FIXED

(www.mozilla.org :: General, defect)

Description

[ken kovash]

The total download number at http://www.mozilla.com/en-US/firefox/stats/ appeared as 230M last week, dropped down to 220M, and now sits at 350M.  Can we determine where that total count is coming from, and if we need to change the definition?

Comment 1

[Daniel Einspanjer [:dre] [:deinspanjer]]

We can either switch to a total with a three hour lag, or we can implement this with changes to SQLstream.  I believe the SQLstream solution will be deployed mid to late June, so if we want a quicker fix, we should do the data warehouse query.

Comment 2

[Daniel Einspanjer [:dre] [:deinspanjer]]

For the moment, quickest fix we can implement would be to switch to the total downloads tracked in the data warehouse.  First step is to use the numbers as of last night since those can be easily queried.  Later, we can also add in the total as of three hours ago.

Could webdev guys take a look at possibly using this json file that I've constructed?
https://metrics.mozilla.com/stats/firefox_downloads_total.json

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

I remember someone in webdev mentioning they were going to look at using this json.  Did work get done there?

Comment 4

[Daniel Einspanjer [:dre] [:deinspanjer]]

Reassigning to webdev.  They've gotten the info for the new totals file to use.  Just needs to be integrated.

Comment 5

[Alex Buchanan [:abuchanan]]

Any news on the SQLstream fixes mentioned in comment #1?  Is this new JSON file a temporary fix until SQLstream is running again?

Is the generated JSON file mentioned in comment #2 updated daily?

We can use that JSON file to update the total count blocks in the sidebar of pages like,
http://www.mozilla.com/en-US/firefox/stats/
http://www.mozilla.com/en-US/firefox/personal.html

...but that doesn't include a fix for the SVG map and real-time, per-locale download counter on the /firefox/stats page.  So are we supposed to take that page down?

TBH, I'd rather not change any code and instead fix SQLstream, since it seems that's where the core issue is.  Is there a bug detailing the issues with SQLstream?

Comment 6

[Shawn]

Now it says zero 3.6 downloads.

Comment 7

[John Slater]

(In reply to comment #6)
> Now it says zero 3.6 downloads.
This is bad. Steven or Alex, can you pull down little promo widgets that are linking to this page? They're on these four pages:

http://www.mozilla.com/en-US/firefox/ie.html
http://www.mozilla.com/en-US/firefox/firefox.html
http://www.mozilla.com/en-US/firefox/upgrade.html
http://www.mozilla.com/en-US/firefox/personal.html

We should put them up again when this is fixed - past testing has shown that the download counter positively affects the download conversion rate on these pages, so I don't want to lose it.

On that note, how are we going to fix it? I don't see a lot of progress happening on that front. Who can answer Alex's questions from comment #5? Daniel, can you?

Comment 8

[Daniel Einspanjer [:dre] [:deinspanjer]]

I'll touch base with SQLStream and update on this bug when I have details.

I'd probably recommend starting a separate webdev bug to pull the link until we have a verified fix.

Comment 9

[John Slater]

Thanks Daniel. I just filed bug 596027 to track pulling the link.

Comment 10

[Laura Forrest]

Hi Daniel or Alex - What's the latest on getting the download counter functioning again? It was removed once it stopped working but we'd like to add it back to the download pages once it's working again. Past tests have shown it has a positive effect on download rate, so it's a important piece of the page. 

Current designs have taken its 'out of commission' state into mind and were built to exclude it, but we'd like to know when we can phase it back in.

Thanks!
Laura

Comment 11

[Daniel Einspanjer [:dre] [:deinspanjer]]

A while back, there was a bug where a json feed of the cumulative download total per version.  I can't find the bug, but the feed is available here:
https://metrics.mozilla.com/stats/firefox_downloads_total.json

Is it of any immediate use to you guys?

Comment 12

[Laura Forrest]

Partially - what we need is for the whole thing to work again, meaning being able to see in near real-time (up to a day lag is okay) the amount of downloads 3.6 (and soon, 4) has received contained within a snazzy graphic on various download pages (/ie, /firefox/firefox, mozilla.com).

Daniel - If this json data source is accurate then is it just a matter of building it into a graphic that shows on the front-end?

Comment 13

[Laura Forrest]

Hey Daniel - Any update on this? We're currently planning when to add more elements onto the download page, and this is the top element we'd like to add in, given that it helps conversions. 

We will also want this functionality ready for the 4.0 launch. 

Let me know what I can do to help. 

Thanks!

Comment 14

[John Slater]

Daniel...any updates?

Comment 15

[Daniel Einspanjer [:dre] [:deinspanjer]]

That json source is as accurate as we have available right now and it is updated nightly.

I am still working on the realtime project, but we are dealing with a few snags with deployment that have delayed us by weeks.  It is still very reasonable to expect that the system will be deployed and ready well before 4.0 launch.

Comment 16

[John Slater]

Thanks for the update. Having this in time for the Fx4 launch is definitely the goal. Sooner is great, of course, but that's the main deadline.

Are there other bugs filed on this that we should know about?

Comment 17

[Michael Morgan [:morgamic]]

Can I get a what what?

Comment 18

[Laura Forrest]

Hey Daniel - We're trying to figure out which pre Fx4 Mozilla.com site release this should go into - can you give us a better idea of timing on this? Thanks!

Comment 19

[Daniel Einspanjer [:dre] [:deinspanjer]]

I am still struggling with the realtime data storage project.  The team working on that have hit a wall in their interface with our Hadoop cluster and they are having to re-engineer that part.  I am trying to get a revised timeline set up with them this week and will post more details on that here when I have it.

Comment 20

[John Slater]

Thanks Daniel. Please keep us posted!

Comment 21

[Michael Morgan [:morgamic]]

Is there some kind of dirty hack (tm) that we can do to get the json feed updated?

Comment 22

[Michael Morgan [:morgamic]]

Potch and Jeff -- could we repurpose what you're tackling for the map to fix this bug?

Comment 23

[Jeff Balogh (:jbalogh)]

(In reply to comment #22)
> Potch and Jeff -- could we repurpose what you're tackling for the map to fix
> this bug?

I think we're depending on the same data that is blocking the json, but our plan is to replace the 3.6 map with a shiny 4.0 map. This bug should probably stay with Daniel until the data is ready to stream.

Comment 24

[Daniel Einspanjer [:dre] [:deinspanjer]]

Sorry, I'm confused about what json we are talking about at this point.

The bug summary mentions total count.  The json feed I pointed to in comment #2 can provide that on a nightly basis.

Jeff and Potch are working on the realtime system that will allow building a map of downloads in particular regions and tables of data that can be broken down into multiple levels.

What specifically are we needing here that isn't covered by comment #2?

Comment 25

[Laura Forrest]

(In reply to comment #24)
> Sorry, I'm confused about what json we are talking about at this point.
> 
> The bug summary mentions total count.  The json feed I pointed to in comment #2
> can provide that on a nightly basis.
> 
> Jeff and Potch are working on the realtime system that will allow building a
> map of downloads in particular regions and tables of data that can be broken
> down into multiple levels.
> 
> What specifically are we needing here that isn't covered by comment #2?

Daniel - It seems like the total count is coming through and that the raw numbers are there and working again. If Jeff and Potch don't need any other numbers this can be closed. 

Jeff? Potch?

Comment 26

[Daniel Einspanjer [:dre] [:deinspanjer]]

Just to sync up:  I commented in bug 600708 that we still have to resolve whether the new glow site's map will replace the map currently found on the stats page.  If it doesn't replace it, then this page will have to either be redesigned or shut down because the map and table on the existing page will not be fixed.

Comment 27

[raymond [:retornam] (inactive)]

 http://www.mozilla.com/en-US/firefox/stats/ ---> http://glow.mozilla.org