Closed Bug 554342 Opened 15 years ago Closed 15 years ago

Install awstats on www.drumbeat.org and make reports available

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: gerv, Assigned: aravind)

References

Details

Please install the awstats package on the www.drumbeat.org machine and make its reports available - perhaps on a different HTTP port, and definitely behind a login. Gerv
Blocks: 549384
There's probably a better way to do analytics than this.
Assignee: server-ops → nobody
Group: mozilla-stats
Component: Server Operations → Data/Backend Reports
Product: mozilla.org → Mozilla Stats
QA Contact: mrz → data-reports
Version: other → 0.1
mrz: who are you hoping is going to tell us about this better way? We are definitely open to suggestions, but we need to implement something quickly, so don't want to get this lost in the long grass. And this bug is currently unassigned. Gerv
I moved it over to Mozilla Stats - they'll help you.
Ken or Blake, could we quickly and easily get this set up on one of our web analytics services?
We can track drumbeat.org on WebTrends. We must ensure that drumbeat's privacy policy accounts for this change before moving forward: http://www.drumbeat.org/privacy-policy
The drumbeat privacy policy is (or was; the original seems to have been updated since) a copy of the standard Mozilla one. http://www.mozilla.org/about/policies/privacy-policy.html Does that have the necessary clauses? If so, we can copy it over again, and we're all set. Gerv
We're going to make some changes to the Mozilla privacy policy in the next couple of weeks. I'd check with legal and keep and eye on these bugs: * https://bugzilla.mozilla.org/show_bug.cgi?id=554815 * https://bugzilla.mozilla.org/show_bug.cgi?id=554731
Adding to Blake's comment... There won't be any actual changes to the Mozilla privacy policy. We're just working on a phased roll-out and implementation of a new web analytics solution for Mozilla's web sites. Gerv -- if it takes a few weeks before we implement our solution on drumbeat.org, is that a problem?
Gerv, in the long run, you'll be much happier with the webtrends solution. I know you mentioned you were very eager to get *something* up and running though, so I'd say let's ask mrz if they can install awstats as a stopgap until we are ready to roll out webtrends for you. If they can't do so, then I'll see what I can do.
(In reply to comment #8) > Gerv -- if it takes a few weeks before we implement our solution on > drumbeat.org, is that a problem? That's a question for Matt :-) (In reply to comment #9) > I'd say let's ask mrz if they can install awstats as a stopgap until we are > ready to roll out webtrends for you. Matt, Paul: can we figure out whether we want a stopgap and, if so, whether awstats is the way to go or whether we want to install the Drupal "Google Analytics" module, or something else? Gerv
@Gerv We will install Google Analytics as it's easy to do so and there's lots of interesting information we can start collating now Not familiar with Webtrends and what it can offer but we should try to adopt open source solutions like Piwik / Awstats if we can. http://piwik.org/ Best, Paul Booker Appcoast
Under the mozilla.com (and by extension, drumbeat.org) privacy policy, Google Analytics is not okay to use. Given the policy, we can use solutions where either of the followed is fulfilled: (1) the data is maintained internally on Mozilla webservers (e.g., Piwik or Awstats), or (2) the policies of the solution have been reviewed and approved by Mozilla's legal team (e.g., Omniture or Webtrends).
(In reply to comment #1) > There's probably a better way to do analytics than this. mrz: it would seem not. WebTrends is the long-term solution, but awstats seems to be the only viable short-term one, and we are looking for data ASAP. Please can awstats be installed on drumbeat.org? Thanks, Gerv
Assignee: nobody → server-ops
Group: mozilla-stats
Component: Data/Backend Reports → Server Operations
Product: Mozilla Stats → mozilla.org
QA Contact: data-reports → mrz
Version: 0.1 → other
@mrz and (preferably) / or piwik ? Best, Paul Booker Appcoast
We're flying blind with zero site metrics right now. Any update appreciated.
Severity: normal → critical
There isn't a great place to host awstats - drumbeat.org is on internal servers that aren't publicly accessible. Offhand, I don't know where I'd put this so you would have access to it. With four releases going on right now, this isn't at the top of the to-do list.
Severity: critical → normal
I can give you awstats, I can't really work on other stuff like piwik. Use MPT VPN and hit this URL: http://cm-metricsapp01.mozilla.org/awstats/bin/awstats.pl?config=drumbeat
Hrm.. it is taking a long time to import the initial set of data. Hopefully it will have something ready by tomorrow.
Okay. I figured out what was wrong. it was set to do DNS lookups which is dog slow. It should take maybe an hour more and it will be done processing all the logs for Feb and March. Check the URL above a little later tonight.
Please file a new bug to get moved over to Webtrends so we can track that.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Thanks :dre! Done. Sorry for the delay. Bug 557527 - Please set up WebTrends metrics tracking for Drumbeat.org
:dre: Unfortunately, I don't have MPT VPN access, so can't access that URL. :( Is there another way I might be able to get it?
Matt, I'm going to take another look to see if I can put this somewhere that you can get to it using your LDAP account without needing MPT VPN.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Okay. You can now view the site at this location: https://metrics.mozilla.com/awstats/bin/awstats.pl?config=drumbeat If your LDAP login does not grant you access, please ask IT to supply me with the name of some LDAP group of which you are a member for which I can grant access to this page.
Status: REOPENED → RESOLVED
Closed: 15 years ago → 15 years ago
Resolution: --- → FIXED
@Daniel, I couldn't get access to awstats with my LDAP account (tried resetting my LDAP password) @Gerv Would you mind helping me to determine my LDAP group ? Best, Paul Booker Appcoast
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
My LDAP login does not grant me access either. @Gerv: as per asking IT to supply an appropriate alternate LDAP group, can you help us out with process? Do we need to file another bug to do that?
My LDAP login does grant me access. :dre: Which LDAP group did you use? Paul: a person can be a member of multiple LDAP groups, so there's no such thing as "your LDAP group" singular. I'm not sure which ones you are a member of (although I could guess some) as I don't know all the names and functions, and one needs access to the LDAP server to find out. We can either ask IT to create a new group specially, or we can just open up access to an existing group e.g. everyone with commit access. How public can we make this data before we get into some sort of difficulty? Gerv
Thanks Gerv It sounds as though creating a new group may be better but i agree we should let someone in IT lead the way. Best, Paul booker Appcoast
Gerv, if you have access and they don't, then that means you have the StatsDashboard group. I was sorta hoping that there was some group that everyone interested in the drumbeat stats would have that wasn't extremely widely used. I am not very comfortable with opening the awstats site up to a wide group without clearance from IT/Security because awstats is a third party Perl script. I don't know for sure how safe it is, and I am hosting it on a machine that contains sensitive information. This bug is already in the IT queue. We just need their help to figure out a reasonable group for me to use for this site. If they are comfortable with it being the committers group, that's fine too, it will take me just a few moments to change the config.
@Daniel , That sounds perfect , thanks for you help :-) Best, Paul booker Appcoast
Aravind, what do you think would be a good way to move forward? I'm thinking we could create a custom group, not sure about the amount of work it involves or if there's a better solution?
Assignee: server-ops → aravind
just looking at this bug, are we trying to grant external folks access to this data? Don't we already have a metrics group in ldap? Why can't we re-use that one for this?
The site is currently restricted to StatsDashboard. Since they don't have access I assumed that meant they were non moco. If they are moco then they just need to have that group added. Otherwise, I need a different group to restrict this site to.
Aravind: we need to grant access to a group of people which includes MoCo people, MoFo people and people who are (SVN) committers but neither MoCo nor MoFo. So I doubt any existing group fits very well. If it's not too much trouble, a new LDAP group would be great. Initial members: gerv@gerv.net mthompson2000@gmail.com paulbooker@appcoast.com (or whatever his LDAP username is! :-)) mhaggerty@trellon.com (Matt: do add more names to this list.) Thanks, Gerv
(In reply to comment #35) > Aravind: we need to grant access to a group of people which includes MoCo > people, MoFo people and people who are (SVN) committers but neither MoCo nor > MoFo. So I doubt any existing group fits very well. > > If it's not too much trouble, a new LDAP group would be great. Initial members: > gerv@gerv.net > mthompson2000@gmail.com > paulbooker@appcoast.com (or whatever his LDAP username is! :-)) My name is Paul and my LDAP username is paulbooker@appcoast.com > mhaggerty@trellon.com > > (Matt: do add more names to this list.) > > Thanks, > > Gerv
I tried to tweak the config for that directory to either accept a list of additional usernames or to just allow any valid LDAP user, but I couldn't figure it out. We'll have to wait for IT to either create a new group, or tweak the config in a way that I'm not familiar with.
dre: we are waiting for IT to create the new group. This bug is currently assigned to Aravind. Gerv
Okay, new ldap group created. The name is cn=StatsDrumbeat,ou=groups,dc=mozilla Here are the members. member: mail=deinspanjer@mozilla.com member: mail=gerv@mozilla.org member: mail=mthompson2000@gmail.com member: mail=paulbooker@appcoast.com member: mail=mhaggerty@trellon.com I am assuming all those folks have ldap accounts. If not, please file separate bugs to get them accounts.
Status: REOPENED → RESOLVED
Closed: 15 years ago → 15 years ago
Resolution: --- → FIXED
Config updated to use new group. It works for me now. Please try it out you guys and see if we are in business. https://metrics.mozilla.com/awstats/bin/awstats.pl?config=drumbeat
I still have access. Gerv
@Daniel Were in business! Thanks for you help. Best, Paul Booker Appcoast
@ Aravind: Thanks for setting up that ldap group! My existing LDAP account is for matthew@mozillafoundation.org Can you update the group to use that account?
I'd suggest making a new IT bug for that since this one is closed and it might not get prompt attention. Switching membership of an existing group is something that several people in IT can do.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.