The rise of cloud-native applications, containerized environments, and dynamic infrastructure has transformed how organizations deploy and manage workloads. While this evolution accelerates innovation, it also introduces new security challenges. Traditional signature-based detection methods, which rely on pre-defined behavioral patterns or known indicators of compromise (IoCs), often fail to keep pace with novel threats targeting modern systems.
Uptycs’ anomaly detection capabilities address these challenges by leveraging eBPF-powered telemetry and machine learning (ML) to detect deviations in expected behavior across containers, hosts, and Kubernetes environments. This blog explores the limitations of traditional detection methods, highlights real-world attack scenarios, and shows how anomaly detection delivers actionable insights for securing dynamic workloads.
Why Signature-Based Detections Fall Short
Signature-based detection has been a cornerstone of cybersecurity, but its reliance on known patterns makes it ineffective against evolving threats. Today’s adversaries exploit this limitation in environments that span containers, hosts, and Kubernetes orchestration layers. Here are some scenarios where behavioral signatures fall short:
1. Cryptojacking Campaigns in Kubernetes Clusters
- Example: Attackers infiltrate Kubernetes clusters, deploying containers to perform cryptocurrency mining using excessive CPU resources. These containers mimic legitimate workloads, using obfuscated scripts to fetch mining software and communicate with mining pools over unusual IPs.
- Challenge: Traditional tools fail to flag these containers because the resource usage aligns with regular operations, and the scripts evade signature-based detection.
2. Supply Chain Attacks via Compromised Images
- Example: A public container image is compromised, embedding a payload that communicates with an external command-and-control (C2) server. The payload executes stealthy actions like downloading scripts or manipulating network traffic.
- Challenge: Signature-based detections miss these payloads if the attack leverages novel techniques not previously cataloged.
3. Fileless Malware on Hosts and Containers
- Example: Fileless malware executes on a host or container by abusing legitimate system tools like bash, curl, or PowerShell. These tools perform lateral movement, credential theft, or data exfiltration.
- Challenge: With no files to monitor or modify, signature-based methods cannot detect these attacks.
These examples highlight the need for adaptive defenses like anomaly detection that monitor deviations in behavior across systems, providing visibility into containerized workloads, host activity, and Kubernetes orchestration layers.
How Uptycs Detects Anomalies Across Hosts and Containers
Uptycs leverages eBPF for telemetry collection and ML models for adaptive baselining, enabling granular insights into behavior. Here’s how it works:
- Deep Telemetry Collection
- Processes: Tracks process creation, execution, and relationships across containers and hosts.
- Sockets: Monitors network connections, identifying suspicious outbound or lateral movement.
- File Activity: Detects unauthorized file reads, writes, or deletions.
- DNS Activity: Detects connections to domains identifying unusual domain names coming from services attached to pods.
- Context-Aware Baselines
- Container Image and Digest: Unique baselines for immutable images and digests.
- Kubernetes Labels: Incorporates Kubernetes labels (e.g., app=backend, team=data-science) to group workloads by logical application or namespace, addressing one of the market’s biggest pain points.
- Anomaly Scoring and Detection
- Uptycs’ ML models continuously analyze telemetry to establish baselines and detect deviations.
- Alerts are enriched with context, allowing IR teams to prioritize critical threats.
Let’s see how it works in action:
1. Model Definition and Grouping: First security teams can use the out of the models or create their own. They can specify learning times (for example 24 hours to perform baselining) of eBPF Activity. What’s also unique about the Uptycs approach is that it allows flexible definition of models. For example, you can go beyond image digest to group by Kubernetes labels which might define application boundaries (e.g. label frontend vs backend).
2. Inclusion & Exclusion: You can perform the modeling on all or a subset of clusters as well as namespaces with the ability to exclude as well. For example, maybe there is a namespace for which binaries are pulled down in a CI/CD pipeline to run some scripts whose activity you want to ignore in the baseline.
3. Baselining and Anomaly Reporting: Once Uptycs conducts the baselining you can view anomalies found for the model across the platform as well as per model.
Anomaly Detection in Action
Anomaly detection focuses on identifying deviations from established baselines of normal behavior, making it ideal for dynamic and distributed environments. Whether it’s detecting unusual process execution on a host or spotting anomalous resource consumption in a container, anomaly detection provides visibility into threats that evade traditional methods.
In the following example we’ve created a model against an nginx image whose baseline learning time is set to 5. We see that the model caught anomalies across process, DNS, and network.
Investigating the Anomalies: A Step-by-Step Analysis
When anomaly detection is triggered, the investigation begins. Let’s dive into what was uncovered in this scenario, piecing together the suspicious activities to form a coherent narrative.
Processes: An Unexpected Command Emerges
During routine monitoring, we noticed that an nginx container, which typically handles web requests, suddenly executed a curl command. This raised an immediate red flag, as such commands are not part of the usual baseline behavior for this container.
- What it means: The curl command was used to pull down an unknown binary, which could be malware or a precursor to further exploitation.
- Why it matters: This deviation is a strong indicator of compromise, possibly signaling an attempt to download malicious payloads or execute unauthorized scripts.
Network Activity: Unfamiliar Connections
Zooming in on the network traffic, additional anomalies emerged. The container initiated outbound connections to unknown IP addresses. Not only were these destinations unrecognized, but the connections occurred across multiple ports, including HTTP and DNS.
- What it means: The container might be communicating with a command-and-control (C2) server, attempting to exfiltrate data or fetch additional instructions from an attacker-controlled endpoint.
- Why it matters: The use of both HTTP and DNS ports suggests evasive tactics to blend malicious traffic into legitimate communication patterns.
DNS: Strange Domains Appear
Delving deeper into DNS logs, further deviations from the baseline were uncovered. While DNS activity was minimal during normal operations, the container suddenly resolved an unusual domain: abc.com. This domain, upon inspection, appeared untrusted and potentially malicious.
- What it means: The domain could be hosting malicious content or acting as a C2 endpoint for the attacker. DNS-based communication often serves as an indirect channel for attackers to bypass traditional network monitoring tools.
- Why it matters: Deviations in DNS activity are particularly significant, as attackers frequently rely on DNS for exfiltration or lateral movement while bypassing security controls.
Piecing It All Together
From process execution to network and DNS anomalies, the evidence points to a likely compromise:
- Initial Vector: The unexpected curl command suggests the container was hijacked to pull a malicious binary.
- Command-and-Control Activity: Network logs reveal the container attempted to communicate with suspicious IP addresses across HTTP and DNS.
- DNS Abnormalities: Resolving an untrusted domain like abc.com further reinforces the likelihood of malicious intent.
Together, these anomalies weave a compelling narrative of an attack in progress, providing invaluable context for security teams.
The Uptycs Blast Radius Mitigation Framework is a five-step journey to cloud security resilience. Read the guide to learn more.
