<div dir="ltr">I never used it, but I saw ads about splunk for log management.<div><br></div><div><a href="http://www.splunk.com/">http://www.splunk.com/</a> <br><br><div class="gmail_quote">2011/2/16 Amos Shapira <span dir="ltr"><<a href="mailto:amos.shapira@gmail.com">amos.shapira@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div dir="ltr">Hello,<br><br>As part of PCI-DSS compliance I'm working on (ref: <a href="http://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard" target="_blank">http://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard</a>), we need to implement automatic log file analysis and alerting. (It's also a Good Thing(TM) to have such a thing in place in general).<br>
<br>LogWatch is not enough since it can't handle the amount of logs generated by our system (we generate ~6Gb of compressed HTTP daemon access log files every 24 hours alone, not to mention many other log files and more to come as we progress with PCI compliance) and still requires someone to manually go through its reports.<br>
<br>Instead, I see many ads for commercial systems which can analyse log files in near real time and generate custom alerts about suspicious activity outside a learned activity pattern. These systems cost a fortune.<br><br>
On the other hand - I saw mentions of open-source system which dump log files onto a NoSQL database and achieve the same functionality with free tools.<br><br>Alas - I lost the references for the later.<br><br>Closest thing I found is Flume (<a href="https://github.com/cloudera/flume" target="_blank">https://github.com/cloudera/flume</a>). Someone tells me that it also does the actual analysis but I don't see this mentioned on its web site.<br>
<br>Does anyone else here have an idea about such systems?<br><br>Thanks,<br><br>--Amos<br>Does anyone <br></div>
<br>_______________________________________________<br>
Linux-il mailing list<br>
<a href="mailto:Linux-il@cs.huji.ac.il">Linux-il@cs.huji.ac.il</a><br>
<a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br>
<br></blockquote></div><br></div></div>