632795 - Statistics for the hosted Bugzilla Project services

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Teemu Mannermaa (:wicked)]

There's certain services for Bugzilla Project hosted by Mozilla (a big thank you for that, BTW!) for which we would like to have statistics generated to analyze user patterns. We'd also prefer to have direct access to "live feed" of stats and not just manual export from certain point in time.

The services we are interested in include:

 1) Download stats for our tarballs at http://ftp.mozilla.org/pub/mozilla.org/webtools/ directory,
 2) Website statistics for http://www.bugzilla.org and
 3) Anonymous access stats to our public repository files at CVS mirror (cvs-mirror.mozilla.org directory /mozilla/webtools/bugzilla) and Bazaar (bzr://bzr.mozilla.org/bugzilla).

Could Mozilla Metrics team (or somebody else at Mozilla) help us generate such stats? Is there already some stats generated for some or all of these services? If so, can we get access to it? (Other than justdave, that is.)

Comment 1

[Daniel Einspanjer [:dre] [:deinspanjer]]

By anonymous access stats to cvs and bz, do you mean when people use the web interface or do you also want thinkgs like checkouts and updates?

Comment 2

[Frédéric Buclin]

(In reply to comment #1)
> By anonymous access stats to cvs and bz, do you mean when people use the web
> interface or do you also want thinkgs like checkouts and updates?

I don't think we are interested by "when people use the web interface". wicked, why did you ask for 3)? Stats about cvs and bzr won't be very significant as devs and Tinderbox access them several times per day.

Comment 3

[Max Kanat-Alexander]

  Yeah, stats about cvs and bzr are not a priority.

Comment 4

[Teemu Mannermaa (:wicked)]

(In reply to comment #1)
> By anonymous access stats to cvs and bz, do you mean when people use the web
> interface or do you also want things like checkouts and updates?

I meant only actual checkouts and updates. And especially anonymous ones so that anybody with an actual account is not counted. Those with accounts are most likely known users and as such are not that interesting.

(In reply to comment #2)
> why did you ask for 3)? Stats about cvs and bzr won't be very significant as
> devs and Tinderbox access them several times per day.

It's one way that people "download" our releases so to get a complete picture of that I added the third point to my request. I don't think dev and Tinderbox access are problematic here because hopefully most of those can easily be filtered out from the final statistics output.

Comment 5

[Frédéric Buclin]

(In reply to comment #4)
> I don't think dev and Tinderbox
> access are problematic here because hopefully most of those can easily be
> filtered out from the final statistics output.

How do you filter them? Most of my connections are anonymous, and my IP address is dynamic.

Comment 6

[Teemu Mannermaa (:wicked)]

(In reply to comment #5)
> How do you filter them? Most of my connections are anonymous, and my IP address
> is dynamic.

What I was thinking filtering out all from same ISP based on the reverse name of your IP address. That should take care of every access you make from that location. Of course, if somebody else is using the same ISP they will be filtered out also.

Comment 7

[Frédéric Buclin]

Any progress?

Comment 8

[Annie Elliott]

Closing 2011 ext still marked as NEW req. 

Not in current scope of work.

Comment 9

[Frédéric Buclin]

Could I know why this bug is marked as wontfix? This is a legitimate request made by the Bugzilla team about its product.

Comment 10

[Dave Miller [:justdave]]

Any motion on this?  #1 we used to be able to get at a long time ago back when ftp was a single server in San Jose, and we lost access to it when Mozilla moved things around between datacenters a year or so ago.  As far as the Bugzilla project is concerned this is a regression that we can't easily get at it anymore (the logs in question are no longer on the same server we upload to).

As far as #2 goes, there used to be stats for this on metrics.mozilla.com already which I had a login to get to, but it doesn't seem to be there anymore?  It was data exported from Webtrends or something like that I think.  Maybe I'm just not remembering the right place to go to get at it?

#3 is hard and optional - probably want devops to poke at that one rather than metrics if we really want it.

Comment 11

[Annie Elliott]

:justdave  #'s 1 & 3 are IT/ops issues.

For #2, we no longer use Webtrends. Looking at the page source for this page, GA has been turned on. You would need to submit a bug to Chris More's group for GA access.

Comment 12

[Dave Miller [:justdave]]

Per comment 11 this is apparently an ops issue.

Comment 13

[Dave Miller [:justdave]]

(In reply to Annie Elliott from comment #11)
> For #2, we no longer use Webtrends. Looking at the page source for this
> page, GA has been turned on. You would need to submit a bug to Chris More's
> group for GA access.

This page is not bugzilla.org.  bugzilla.org does not have GA.

Comment 14

[Annie Elliott]

Yes. You would need to get a GA snippet for that property from Chris More.

Comment 15

[Dave Miller [:justdave]]

This is not exactly an enhancement, since we used to have access to this stuff and now we don't.

We just released Bugzilla 4.4.  We have no idea how well it's doing because we don't have any download counts.  That's the main thing we want to solve here.  These download counts would need to be coming from ftp.mozilla.org.

Comment 16

[Shyam Mani [:fox2mike]]

No need to page oncall over the weekend, lowering priority. If it is that serious, bump up again and we'll find someone from webops.

Comment 17

[Dave Miller [:justdave]]

ping?

Comment 18

[Dave Miller [:justdave]]

ping?

Comment 19

[Jake Maul [:jakem]]

Not sure how to do #1 offhand, will have to think about it.

#2 should be redirected to Chris More, he maintains our Google Analytics installation, and that's all we have for website analytics at the moment.

#3 should be redirected to Laura Thomson (specifically, Ben Kero and/or Kendall Libby), since they manage the source control systems. Note that IIRC both BZR and CVS are slated to be decommissioned soon, so that might factor in here.

Comment 20

[:Atoll]

I spoke with :justdave and we won't be able to provide useful stats from the CDN. It's possible future changes re ftp and AWS might be of more use, but those aren't available today. So between that, and comment 19 #2 and #3, we've reached the conclusion of IT's ability to help and so I'm closing this.