Data Mining Pinpoints Network Intrusions

Just because an event occurs rarely doesn't mean it won't have dramatic impacts. Consider heart attacks, power blackouts, credit card frauds or computer virus infections.

Vipin Kumar and colleagues at the University of Minnesota are developing data-mining techniques to detect rare events, such as computer break-ins, that are difficult to detect using traditional methods that recognize attacks only through pre-defined patterns.

The new techniques have been incorporated in the Minnesota Intrusion Detection System (MINDS) software, which helps cybersecurity analysts detect computer break-ins and other undesirable activity in real-world networks, potentially while the break-in is underway.

"MINDS allows cybersecurity experts to quickly analyze massive amounts of network traffic," Kumar said. "They only need to evaluate the most anomalous connections identified by the system." The data-mining research on rare event analysis is supported by a $300,000 award from the National Science Foundation.

MINDS is currently being used to monitor over 40,000 computers at the University of Minnesota. In addition, it is an integral part of the Army's Interrogator architecture, used at the Army Research Laboratory's Center for Intrusion Monitoring and Protection to analyze network traffic from Defense Department sites around the country. MINDS routinely detects novel intrusions, policy violations and insider abuse that are missed by other widely used tools.

Data mining for rare events becomes critical as new technologies allow more and more data to be collected. The signal indicating that a rare event has happened, or is about to, can be drowned in a rapid flow of data, mostly reporting normal behavior.

Detecting computer intrusions is only the first application for the Minnesota team's new data-mining methods. The underlying techniques could be applied to many areas beyond cybersecurity, such as detecting financial or health-care fraud.

-- David Hart