RANSOMWARE DETECTION USING UNSUPERVISED AUTOENCODERS FOR REAL-TIME FILE SYSTEM BEHAVIOR ANALYSIS

Abstract

Ransomware has rapidly evolved into a formidable cybersecurity menace, capable of paralyzing entire infrastructures by encrypting critical data and demanding monetary ransom for decryption keys. Unlike traditional malware, modern ransomware variants exhibit polymorphic and obfuscated behaviors, rendering conventional signature-based and heuristic rule-based detection systems largely ineffective, particularly against zero-day threats. To address this challenge, we introduce an unsupervised anomaly detection framework leveraging deep autoencoder networks to identify ransomware activity through real-time file system behavior analysis. Our approach systematically monitors fine-grained behavioral indicators such as high-frequency file modifications, entropy shifts indicative of encryption, anomalous file renaming patterns, and rapid extension changes within temporal windows to construct a feature-rich representation of benign system dynamics. The autoencoder is trained solely on normal operational data, enabling it to learn compact latent representations that capture typical file system behavior. Anomalies are detected based on elevated reconstruction errors, calculated using the squared L2-norm between input and output vectors, which signal deviations from learned benign patterns. Comprehensive experimental evaluations on both the EMBER benchmark and a curated real-world ransomware dataset demonstrate that the proposed method achieves a detection accuracy of 96.2%, a precision of 95.1%, and maintains a low false positive rate of 3.5%. These results validate the efficacy of the model in proactively detecting both known and unknown ransomware strains in dynamic and high-throughput environments.