email from Shane: I was just looking at pentaho and our numbers have gone through the roof over the last couple of days. 40k/day last week to 100k/day yesterday and today. The growth comes from "undefined://" Is this FirefoxOS coming online?
Assignee: nobody → pedro
Severity: normal → critical
The source of the spike is the RP 'undefined://'. This information is already present in the files digested by the ETL, ie, they are generated by Identity services that produces the logs. This URL should be filtered from the dashboard according to Bug 690107. However this spike in the logs should be investigated by Identity team. Requesting more info from the Identity team.
Status: NEW → ASSIGNED
It seems to be related: Bug 885467
Apologies. This was most likely me. In trying to understand a production problem from this Monday, I had set up a job to poll the /sign_in URL to check the content. I did not realize (a.k.a., didn't think through) that this would show up in metrics. It's off now, so if the number drops tomorrow, I was the cause (I'm pretty sure this was all me). But yeah, filtering out the undefined would be good (web spiders will also show up as undefined).
We can't just filter out "undefined", because then we'd be filtering out the FFOS data (see Bug 885467). Is there a good way to distinguish FFOS from spiders/testing/etc? John's explanation makes sense. FWIW, the KPI data that we collect shows no unusual activity in the last few days.
Katie, Are you saying that part of those "undefined" are legitimate FFOs, and the rest are spiders,etc.? We can distinguish spiders/etc. and filter them out in the Kettle process.
Yes, that is correct. It would be great to filter out the spiders/etc in the Kettle process, given that we're going to do the work to recognize the legit FFOS users. I think we can assume any "undefined"/localhost referrers that are not FFOS are from testing environments.
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.