1028307 - work with BI/DW to change releases.mozilla.com log processing to use input from Zeus instead of FTP Cluster

Status: RESOLVED FIXED

(Infrastructure & Operations :: IT-Managed Tools, task)

Description

[Jake Maul [:jakem]]

Bug 989239 unveiled a one-off we should fix. It appears that BI/DW is using logs delivered from the FTP web nodes for processing releases.mozilla.com traffic. This should be changed to use standard Zeus logging.

Comment 1

[Anurag Phadke[:[e-mail]]]

jake - afaik, zeus doesn't support the partial header flag inside Apache. I had worked with dumitru in getting the the partial/complete header flag implemented and it was agreed that this was going to be a one-off solution.

I am happy to get this integrated with Zeus as long we don't lose those flags.

Comment 2

[Anurag Phadke[:[e-mail]]]

Sample log line:
xx.xx.xx.xx - - [19/Jun/2014:05:51:53 +0000] "GET /bundles/bing/win32/en-US/Firefox-Bing%20Setup.exe HTTP/1.1" 200 29936560 "http://www.firefoxwithbing.com/?form=BNPVIP&publ=BNPSERP&crea=19011" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36" "X"

The last value "X" is what we are looking for in terms of partial/complete information.

Comment 3

[Jake Maul [:jakem]]

This node can now properly send its log data to the metrics-logger service.

Not particularly interested right now in figuring out how to re-do this to use Zeus logging, so I'm going to close this out.