Put logic for AMO stats processing in a public repository



9 years ago
6 years ago


(Reporter: clouserw, Assigned: dre)




(Whiteboard: Open support case with Pentaho. Update Friday 2010-07-16)



9 years ago
Please put all the code for processing AMO statistics in a public repository.  This will help these areas:

1)  The AMO team often has to figure out where the division between what AMO does and what metrics does is.  Having both sides of the code will help put that line in focus.

2) The AMO team should be able to find precise details on how stats processing works.  We get people that ask us questions and are unable to answer (Do you filter duplicate IPs?  What is the time threshold?  What is the order of the filtering rules?  If a request for the same IP but a different UA comes in, is it counted? etc. etc.)

3) We have problems with stats pretty often (the latest is bug 552162).  Since that was a specific date that it begin failing on it would be easy to see there was a change in the stats repository then instead of combing our systems and getting IT involved to figure out what was going on.

4) By seeing the source we'll have a better idea of what the possibilities are and could find ways to generate more effective reports for add-on authors.

5) Open source helps find/fix bugs
We'll implement this by 2010-04-15
Whiteboard: Implement by 2010-04-15
I've done the first stage of this which is pushing my private git repository to the svn.mozilla.org/moco/metrics/etl/trunk repository.

It isn't where I want to end up because the /moco repository isn't publicly viewable, but it is a necessary first stage since I need to do further work to extract out database connection information that is currently encrypted in the files.  Once I can pull that out, we'll be able to migrate everything else into the public svn repository.

Of course, what is checked in is nothing more than a bunch of XML files.  You guys won't really be able to make much sense of them until you are brought up to speed with the tool we use, Kettle.

That is part of what I want to discuss with you in our meeting next week.
Last Resolved: 9 years ago
Resolution: --- → FIXED
Actually, based on the summary of this bug, I can't actually close it yet because the moco repository isn't public.  You can only see it if you are an authenticated user (and they probably have further permissions per module).

As I said in comment #2, I have to work on splitting out the DB connection data from these files before I can check them into a public repository.  There are other things that have higher priority at the moment though so I don't have an ETA for that work.
Resolution: FIXED → ---
Whiteboard: Implement by 2010-04-15 → Needs to be migrated over to public repository
Whiteboard: Needs to be migrated over to public repository → Open support case with Pentaho. Update Friday 2010-07-16
Triage to set target milestone
Target Milestone: --- → 2010-08.1
De-prioritizing a few weeks.
Target Milestone: 2010-08.1 → 2010-09.1
Moving project level "bugs" (i.e. new features, new data analysis requests) to enhancement rather than bug.
Severity: normal → enhancement
Target Milestone: 2010-09.1 → ---

Comment 8

6 years ago
Closing old issues. Not in current scope of work.
Last Resolved: 9 years ago6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.