The document discusses AT&T's use of a Hadoop-based approach for near real-time outlier detection and interpretation of cyber threats. It provides context on the challenges of detecting threats at AT&T's network scale and data volume. The presentation outlines AT&T's history of threat capabilities, the need for a big data solution, and their transition to a Hadoop-based threat analytics platform for ingesting and analyzing over 5 billion daily network events to detect outliers and threats.
Report
Share
Report
Share
1 of 11
More Related Content
Near Real-Time Outlier Detection and Interpretation
Work real quick through agenda
Just set the stage for an Hadoop based threat analytics platform that has NRT capabilities
Set the stage for how a typical network in this industry and how much work there is for securing it.
Presents an industry problem, not an AT&T problem
Address the outside threat to the internal operation of our industry
Amount of traffic related to reflect based DoS attackers. Illustrates activity on the internet not the attacks against the AT&T perimeter.
Hack-ma-geddon
Columbia government
Spam Hause
Syria <- New York Times
Target lost 40M credit/debit cards
Our TAP has evolved a lot over the last few year as we’ve moved into an Hadoop base architecture. I will briefly describe the roadmap.
Proprietary technology and lack of extensibility are killers
Past was SIEM dependent, based on large RDBMS and exclusively dependent on human detection and interpretation. Largely a data reduction system. Industry solution of yesterday.
The challenge is the cognitive intersection with automation.
An environment of innovation. Goal is to automate the security analysis process which are largely cognitive. Granted this is a different use of Hadoop rather than single use data. Its continual ingestion, NRT detections, alerting, etc. Not always a clear problem statement.
Spend some time developing the human dependency and cognitive processes
Takes a lot of data
Left to right, we move all the data through various processing platforms into an Hadoop base system for raw log management, data org, management, access, analysis and finally to visualization and reporting.