Feature Rich Clustering Techniques for Android Malware Families
DOI:
https://doi.org/10.69996/fmep.2026012Keywords:
Android Malware, Malware Family Classification, Deep Embedding, Spectral Clustering, HDBSCANAbstract
The rapid growth of the Android ecosystem has resulted in a significant increase in sophisticated malware targeting mobile devices. Traditional signature-based detection mechanisms are ineffective against obfuscated and zero-day variants, while supervised learning approaches rely heavily on labeled datasets and often fail to generalize to newly emerging malware families. To address these limitations, this paper proposes a feature-rich unsupervised clustering framework for Android malware family classification that integrates deep embedding representation learning with spectral clustering and HDBSCAN. The approach leverages syscall-binder frequency features to capture fine-grained behavioral characteristics of malware samples. Feature normalization and dimensionality reduction are applied to mitigate high-dimensional sparsity and redundancy. A neural network-based encoder then learns compact latent representations that improve structural separability in the embedding space. Spectral clustering is employed to capture complex non-linear relationships, while HDBSCAN enhances robustness by identifying clusters of varying density and filtering noise samples. The experimental results show that Spectral clusteringachieves 0.8993 in terms of normalized mutual information(NMI),0.9365 in terms of adjusted rand index (ARI) demonstrates that the proposed framework outperforms conventional clustering techniques such as K-Means and standalone density-based methods. The results indicate improved cluster cohesion, inter-family separation, and scalability, making the proposed framework suitable for largescale Android malware family analysis.
References
[1] Google, “Android architecture overview,” 2022, available: https://developer.android.com/guide/platform.
[2] Statista, “Number of android smartphone users worldwide,” Statista Report, 2023.
[3] Y. Zhou and X. Jiang, “Dissecting android malware: Characterization and evolution,” in Proceedings of the IEEE Symposium on Security and Privacy, pp. 95–109, 2012.
[4] P. Faruki, A. Bharmal, V. Laxmi, M. S. Gaur, M. Conti, and M. Rajarajan, “Android security: A survey of issues, malware penetration, and defenses,” IEEE Communications Surveys & Tutorials, vol. 17, no. 2, pp. 998–1022, 2015.
[5] S. Poeplau, Y. Fratantonio, A. Bianchi, C. Kruegel, and G. Vigna, “Execute this! analyzing unsafe and malicious dynamic code loading in android applications,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2014.
[6] T. Petsas, A. Papadogiannakis, M. Polychronakis, E. Markatos, and T. Karagiannis, “Rise of the planet of the apps: A systematic study of the mobile ad and tracking ecosystem,” in Proceedings of the Internet Measurement Conference, pp. 277–290, 2013.
[7] I. You and K. Yim, “Malware obfuscation techniques: A brief survey,” in International Conference on Broadband, Wireless Computing, pp. 297–300, 2010.
[8] I. Santos, F. Brezo, X. Ugarte-Pedrero and P. G. Bringas, “Opcode sequences as representation of executables for data-mining-based unknown malware detection,” Information Sciences, vol. 231, pp. 64–82, 2013.
[9] Karthik Kumar Vaigandla, “A Review on Deep Learning Algorithms for Efficient Mobile Traffic Handling in 5G Systems”, JCAI, vol. 3, no. 5, pp. 1–16, Oct. 2025.
[10] D. Arp, M. Spreitzenbarth, M. Hubner, H. Gascon, K. Rieck, and C. Siemens, “Drebin: Effective and explainable detection of android malware in your pocket,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2014.
[11] J. Saxe and K. Berlin, “Deep neural network based malware detection using two dimensional binary program features,” in 10th International Conference on Malicious and Unwanted Software (MALWARE), 2015.
[12] K. Xu, Y. Qi, Q. Chen, and M. Zhang, “A clustering-based behavioral analysis approach for android malware detection,” IEEE International Conference on Intelligence and Security Informatics, 2012.
[13] C. C. Aggarwal, A. Hinneburg, and D. A. Keim, “On the surprising behavior of distance metrics in high dimensional space,” International Conference on Database Theory, pp. 420–434, 2001.
[14] J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for clustering analysis,” in International Conference on Machine Learning (ICML), pp. 478–487, 2016.
[15] A. Y. Ng, M. I. Jordan and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in Advances in Neural Information Processing Systems (NIPS), pp. 849–856, 2002.
[16] L. McInnes, J. Healy and S. Astels, “Hdbscan: Hierarchical density based clustering,” Journal of Open Source Software, vol. 2, no. 11, pp. 205, 2017.
[17] A. Strehl and J. Ghosh, “Cluster ensembles – a knowledge reuse framework for combining multiple partitions,” Journal of Machine Learning Research, vol. 3, pp. 583–617, 2002.
[18] BH.V.V.S.R.K.K.Pavan, D.Venkata Satish, B.Mounica, and T. Aditya Kumar, “Intelligent System for Secure Tamper Protocol Model with IoT Blockchain Architecture for Data Mitigation”, JCAI, vol. 2, no. 4, pp. 1–19, Aug. 2024.
[19] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD), pp. 226–231, 1996.
[20] I. T. Jolliffe, Principal Component Analysis. Springer, 2002.
[21] S. Lloyd, “Least squares quantization in pcm,” IEEE Transactions on Information Theory, vol. 28, no. 2, pp. 129–137, 1982.
[22] T. e. a. Isohara, “Kernel-based behavior analysis for android malware detection,” IEEE ICC, 2011.
[23] E. Amigo., “Evaluation measures for clustering: A survey,” ACM Computing Surveys, 2009.
[24] M. Sebastian, “The challenges of malware classification using antivirus labels,” DIMVA, 2016.
[25] I. T. Jolliffe, “Principal Component Analysis,” Springer, 2002.
[26] I. Goodfellow, Y. Bengio, and A. Courville, “Deep Learning,” MIT Press, 2016.
[27] S. Venkatramulu, Md. Sharfuddin Waseem, Arshiya Taneem, Sri Yashaswini Thoutam, Snigdha Apuri, and Nachiketh, “Research on SQL Injection Attacks using Word Embedding Techniques and Machine Learning”, JSIHS, vol. 2, no. 1, pp. 55–64, Mar. 2024.
[28] A. Ben-Hur, “Clustering stability: An overview,” Journal of Machine Learning Research, 2002.
[29] A. Shabtai, “Scalable big data analytics for malware detection,” Future Generation Computer Systems, 2018.
[30] C. Kolias, “A survey of iot malware and detection techniques,” IEEE Communications Surveys & Tutorials, 2017.
