Chuck Faughnan
CEO, Strategic Advisor, & Investor @ Personam.ai | Cybersecurity

November 8, 2024

The pendulum has swung — again.  Way back in the ‘90s, the promise of threat detection was in network traffic analysis. The modern computing method of the time was anomaly detection, and it purported to catch all breaches.  And… it kind of did that!

The problem, however, is that, by the strictest of definitions, everything is an anomaly.  Did you check www.askjeeves.com every day at 12:05pm? Well, that one day you checked it at 12:10pm is an anomaly! That one day you opted for altavista.com? Anomaly! Analysts got bogged down with intense alerting volumes. So, yes, while evidence of an attacker’s tracks were likely caught, you’re still looking for a needle in a haystack.

Enter: log file analytics. This approach became the industry standard. They worked well when networks were simpler and attacks were less sophisticated. Looking at logs provided a relatively effective way to detect and respond to security incidents. However, as networks have grown more complex and cyber threats have evolved, the limitations of log-based security are increasingly apparent. Legacy log-based systems often struggle with scalability, cost, and the ability to detect advanced threats.

It’s time to embrace a more powerful, efficient, cost-effective approach: network traffic analysis!

Limitations of Log-Based Security

Log-based solutions, such as Splunk or QRadar, often have high licensing costs that are variable, and pegged to data storage. As logging utilization increases, so does cost. So there’s that consideration. But what do you do if you have no logs? Microsoft lost weeks worth of log data [1]. TrendMicro notes how savvy attackers manipulate firewall rulesets to block logging facilities meaning you never even see the entries![2] Clearly relying solely on logs can leave organizations exposed to significant risk. The main problems with log file based detecting are:

  1. Limited Visibility: Log-based solutions provide a fragmented view of network activity, focusing on specific events rather than the broader context. This leads to missed threats and delayed incident response. You are also dependent on every single endpoint you want to cover (all of them, right?) being able to generate logs.  IoT devices used in healthcare or manufacturing facilities don’t often have this capability. Your network printer has a hard drive — but does it have antivirus?
  2. Latency: Log data is not uniformly structured. This introduces latency before it gets to your analysts as it needs to be collected and processed first. This delay can hinder timely threat detection and response.
  3. Data Overload: The exponential growth of products and services, and the increasing complexity of cyberattacks generate massive volumes of log data. This can overwhelm security teams and slow down analysis. We’re back to the ‘90s needle in haystack!
  4. False Positives and Negatives: Log-based solutions often struggle to differentiate between legitimate and malicious activity, leading to a high number of false positives and negatives. This burns out your analysts and tarnishes efficiency metrics such as mean time to detection.
  5. Vulnerability to Adversary Tactics: Advanced adversaries employ techniques like EDRSilencer to suppress log generation, effectively evading detection.  Even novice attackers can open up Windows Event Viewer, right click, and “Clear Log”.

The Power of Network Traffic Analysis

Recent advances in computing make a return to network traffic analysis a superior approach. Vectra.ai does a great job here. So do we, at Personam.ai. Leveraging the network – something nearly all attackers do – offers a more comprehensive and cost-effective approach to security. The network is the truth. With advanced techniques, network based detectors can deliver:

  1. Real-Time Visibility: By directly examining network traffic, security teams gain real-time insights into network activity, enabling them to identify and respond to threats as they emerge, wherever they originate. One reason the Mirai botnet was hard to crush was that it hit IoT devices such as web cameras and smart speakers.  How are your organizations covering those devices today? Traffic-based threat detection eliminates those blind spots.
  2. Behavioral Anomaly Detection: Unsupervised machine learning algorithms can analyze network traffic patterns to detect deviations from normal behavior, uncovering hidden threats active on your network that traditional log-based solutions might miss.
  3. Reduced False Positives and Negatives: Network traffic analysis can provide more accurate threat detection by considering the broader context of network activity, leading to fewer false alarms and faster incident response.
  4. Resilience Against Adversary Tactics: Network traffic analysis can detect malicious activity even when log data is compromised or suppressed.
  5. Lower Cost: Network traffic analysis solutions, particularly those leveraging netflow metadata sampling, can be significantly more cost-effective than traditional log-based solutions, reducing the burden on IT budgets.

A New (?) Era of Network Vigilance

As cyber threats continue to evolve, it is imperative to use the best tools available. Hackers live on the network. They do not live in log files. Logs can be tampered with. Logs are abstractions. Honor and respect that your network traffic is the only form of truth. Unsupervised machine learning offers a powerful and innovative approach to safeguarding your digital assets. By shifting focus from logs to network traffic, organizations can achieve superior visibility, faster response times, and reduced burden on budgets, and greater protection against cyber attacks.

What are your experiences with log-based solutions? Are there any other limitations we’ve missed? Leave a comment and let us know!