Balancing dimensionality reduction and information retention in medical datasets using adaptive relevance scoring and trifold evolutionary optimization
DOI:
https://doi.org/10.65746/jbrha126Keywords:
adaptive relevance score; dimensionality reduction; gene noise filtering layer; mutual information; trifold evolutionary optimizationAbstract
High-dimensional medical datasets often contain redundant, irrelevant, and noisy features that increase computational complexity. Existing approaches for dimensionality face several critical challenges: (i) aggressive feature reduction leading to loss of critical clinical information, (ii) selection unfairness in model-dependent methods such as Recursive Feature Elimination (RFE), (iii) reliance on variance in Principal Component Analysis (PCA) (iv) poor generalization of existing feature selection methods across diverse medical datasets and (v) uncertainty about whether retained features truly capture essential clinical information after DR. To address these limitations, this study proposes an adaptive feature selection framework that balances DR with effective information retention. The proposed method introduces a Gene Noise Filtering Layer along with entropy-based filtering, Mutual Information, and variance filtering to remove low-informative features. An Adaptive Relevance Score (ARS) is developed to identify clinically significant features. A Trifold Evolutionary Optimization (TEO) approach, integrating Particle Swarm Optimization, Quantum-inspired search, and genetic mutation, is employed to determine optimal feature subsets. Experimental evaluation achieves a Feature Stability Index (FSI) of 0.9819, indicating excellent preservation of informative features and high stability of the selected feature subsets. The proposed method improved classification accuracy, reduced dimensionality, and attained better preservation of clinically meaningful information across multiple medical datasets.
References
1. Serani A, Diez M. A survey on design-space dimensionality reduction methods for shape optimization. Archives of Computational Methods in Engineering. 2026; 33(2): 1671–1698. doi: 10.1007/s11831-025-10349-x
2. Azarhoosh Z, Ghazaan MI. A review of recent advances in surrogate models for uncertainty quantification of high-dimensional engineering applications. Computer Methods in Applied Mechanics and Engineering. 2025; 433: 117508. doi: 10.1016/j.cma.2024.117508
3. Morabito A, De Simone G, Pastorelli R, et al. Algorithms and tools for data-driven omics integration to achieve multilayer biological insights: a narrative review. Journal of Translational Medicine. 2025; 23(1): 425. doi: 10.1186/s12967-025-06446-x
4. Javanmard Z, Zarean Shahraki S, Safari K, et al. Artificial intelligence in breast cancer survival prediction: A comprehensive systematic review and meta-analysis. Frontiers in Oncology. 2025; 14: 1420328. doi: 10.3389/fonc.2024.1420328
5. Xu W, Wang J, Zhang Y, et al. An optimized decomposition integration framework for carbon price prediction based on multi-factor two-stage feature dimension reduction. Annals of Operations Research. 2025; 345(2): 1229–1266. doi: 10.1007/s10479-022-04858-2
6. Prabaharan G, Sankar SU, Anusuya V, et al. Optimized disease prediction in healthcare systems using HDBN and CAEN framework. MethodsX. 2025; 14: 103338. doi: 10.1016/j.mex.2025.103338
7. Pantanowitz L, Pearce T, Abukhiran I, et al. Nongenerative artificial intelligence in medicine: Advancements and applications in supervised and unsupervised machine learning. Modern Pathology. 2025, 38(3):100680. doi: 10.1016/j.modpat.2024.100680
8. Naghib A, Gharehchopogh FS, Zamanifar A. A comprehensive and systematic literature review on intrusion detection systems in the internet of medical things: current status, challenges, and opportunities. Artificial Intelligence Review. 2025; 58(4): 114. doi: 10.1007/s10462-024-11101-w
9. Katib I, Albassam E, Sharaf SA, et al. Safeguarding IoT consumer devices: Deep learning with TinyML driven real-time anomaly detection for predictive maintenance. Ain Shams Engineering Journal. 2025; 16(2): 103281. doi: 10.1016/j.asej.2025.103281
10. Das S, Sultana M, Bhattacharya S, et al. XAI–reduct: Accuracy preservation despite dimensionality reduction for heart disease classification using explainable AI. The Journal of Supercomputing. 2023; 79(16): 18167–18197. doi: 10.1007/s11227-023-05356-3
11. Gouiaa F, Vomo-Donfack KL, Tran-Dinh A, et al. Novel dimensionality reduction method, Taelcore, enhances lung transplantation risk prediction. Computers in Biology and Medicine. 2024; 169: 107969. doi: 10.1016/j.compbiomed.2024.107969
12. Dhanka S, Sharma A, Kumar A, et al. Advancements in hybrid machine learning models for biomedical disease classification using integration of hyperparameter-tuning and feature selection methodologies: A comprehensive review. Archives of Computational Methods in Engineering. 2026; 33(1): 289–324. doi: 10.1007/s11831-025-10309-5
13. Ambrosino M, Palarea-Albaladejo J, Albanese S, et al. Assessing natural background concentrations of chemical elements in urban soils: A case study in Benevento (Italy). Science of The Total Environment. 2025; 975: 179298. doi: 10.1016/j.scitotenv.2025.179298
14. Ignat A. Experiments with dimensionality reduction on chest x-ray images. Procedia Computer Science. 2023; 225: 3415–3424. doi: 10.1016/j.procs.2023.10.336
15. Chin FY, Lem KH, Wong KM. Improving handwritten digit recognition using hybrid feature selection algorithm. Applied Computing and Informatics. 2026; 22(1–2): 105–114. doi: 10.1108/ACI-02-2022-0054
16. Anuragi A, Sisodia DS, Pachori RB. Mitigating the curse of dimensionality using feature projection techniques on electroencephalography datasets: an empirical review. Artificial Intelligence Review. 2024; 57(3): 75. doi: 10.1007/s10462-024-10711-8
17. Amin R, Yasmin R, Ruhi S, et al. Prediction of chronic liver disease patients using integrated projection based statistical feature extraction with machine learning algorithms. Informatics in Medicine Unlocked. 2023; 36: 101155. doi: 10.1016/j.imu.2022.101155
18. Wang Z, Gao D, Lu Y, et al. A mutual cross-attention fusion network for surface roughness prediction in robotic machining process using internal and external signals. Journal of Manufacturing Systems. 2025; 82: 284–300. 10.1016/j.jmsy.2025.06.018
19. Dhinakaran D, Srinivasan L, Raja SE, et al. Synergistic feature selection and distributed classification framework for high-dimensional medical data analysis. MethodsX. 2025; 14: 103219. doi: 10.1016/j.mex.2025.103219
20. Gildenblat J, Pahnke J. Dimensionality reduction with strong global structure preservation. Pattern Analysis and Applications. 2026; 29(1): 38. doi: 10.1007/s10044-025-01585-9
21. Xiong L, An J, Hou Y, et al. Improved support vector regression recursive feature elimination based on intragroup representative feature sampling (IRFS-SVR-RFE) for processing correlated gas sensor data. Sensors and Actuators B: Chemical. 2024; 419: 136395. doi: 10.1016/j.snb.2024.136395
22. Vinutha MR, Chandrika J, Krishnan B, et al. EPCA—enhanced principal component analysis for medical data dimensionality reduction. SN Computer Science. 2023; 4(3): 243. doi: 10.1007/s42979-023-01677-5
23. Maruotto I, Ciliberti FK, Gargiulo P, et al. Feature selection in healthcare datasets: Towards a generalizable solution. Computers in Biology and Medicine. 2025; 196: 110812. doi: 10.1016/j.compbiomed.2025.110812
24. Khan S, Mazhar T, Naz NS, et al. Advanced Feature Selection Techniques in Medical Imaging—A Systematic Literature Review. Computers, Materials, & Continua. 2025; 85(2): 2347. doi: 10.32604/cmc.2025.066932
25. Kabir MF, Chen T, Ludwig SA. A performance analysis of dimensionality reduction algorithms in machine learning models for cancer prediction. Healthcare Analytics. 2023, 3:100125. doi: 10.1016/j.health.2022.100125
26. Sanju P, Ahmed NS, Ramachandran P, et al. Enhancing thyroid disease prediction and comorbidity management through advanced machine learning frameworks. Clinical eHealth. 2025; 8: 7–16. doi: 10.1016/j.ceh.2025.01.002
27. Ashika T, Hannah Grace G. Enhancing heart disease prediction with stacked ensemble and MCDM-based ranking: An optimized RST-ML approach. Frontiers in Digital Health. 2025; 7: 1609308. doi: 10.3389/fdgth.2025.1609308
28. Kumar A, Dhanka S, Sharma A, et al. A hybrid framework for heart disease prediction using classical and quantum-inspired machine learning techniques. Scientific reports. 2025; 15(1): 25040. doi: 10.1038/s41598-025-09957-1
29. Razzaque A, Badholia A. PCA based feature extraction and MPSO based feature selection for gene expression microarray medical data classification. Measurement: Sensors. 2024; 31: 100945. doi: 10.1016/j.measen.2023.100945
30. Jin J, Xu Y, He H, et al. A swin transformer-based hybrid reconstruction discriminative network for image anomaly detection. Scientific Reports. 2025; 15(1): 33929. doi: 10.1038/s41598-025-10303-8
31. Breast Cancer Gene Expression (CuMiDa) dataset: https://www.kaggle.com/datasets/brunogrisci/breast-cancer-gene-expression-cumida (accessed on 10 April 2026).
32. Xu H, Deng Q, Zhang Z, et al. A hybrid differential evolution particle swarm optimization algorithm based on dynamic strategies. Scientific reports. 2025; 15(1): 4518. doi: 10.1038/s41598-024-82648-5
33. Islam MR, Ahmed B, Hossain MA, et al. Mutual information-driven feature reduction for hyperspectral image classification. Sensors. 2023; 23(2): 657. doi: 10.3390/s23020657
34. Hoang VT, Dinh N, Le VT, et al. Detecting Anomalies in FinTech: A Graph Neural Network and Feature Selection Perspective. Computers, Materials, & Continua. 2026; 86(1): 1. doi: 10.32604/cmc.2025.068733
35. Li J, Sheng X, Luo X. Chaos–Quantum Particle Swarm Optimized Kriging for Symmetric Response Modeling and Multi-Objective Marketing Optimization in E-Commerce Systems. Symmetry. 2026; 18(5): 770. doi: 10.3390/sym18050770
36. Serani A, Diez M. A survey on design-space dimensionality reduction methods for shape optimization. Archives of Computational Methods in Engineering. 2026; 33(2): 1671–1698. doi: 10.1007/s11831-025-10349-x
37. Valero-Carreras D, Alcaraz J, Landete M. Comparing two SVM models through different metrics based on the confusion matrix. Computers & Operations Research. 2023; 152: 106131. doi: 10.1016/j.cor.2022.106131
38. CuMiDa dataset: https://www.kaggle.com/datasets/orvile/gene-expression-profiles-of-breast-cancer (accessed on 10 April 2026).
39. Leukemia Classification dataset: https://www.kaggle.com/datasets/andrewmvd/leukemia-classification (accessed on 10 April 2026).
40. Colorectal Cancer Global Dataset: https://www.kaggle.com/datasets/ankushpanday2/colorectal-cancer-global-dataset-and-predictions (accessed on 10 April 2026).
41. Islam MR, Chowdhury AI, Shama S, et al. Enhanced thyroid disease prediction using ensemble machine learning: A high-accuracy approach with feature selection and class balancing. Discover Artificial Intelligence. 2025; 5(1): 9. doi: 10.1007/s44163-025-00225-9
42. Gomez-Martinez V, Chushig-Muzo D, Veierød MB, et al. Ensemble feature selection and tabular data augmentation with generative adversarial networks to enhance cutaneous melanoma identification and interpretability. BioData Mining. 2024; 17(1): 46. doi: 10.1186/s13040-024-00397-7
43. Senyer Yapici I, Uzun Arslan R. Predictive analytics for thyroid cancer recurrence: A feature selection and data balancing approach. The European Physical Journal Special Topics. 2025; 234(15): 4751–4771. doi: 10.1140/epjs/s11734-025-01720-x
44. Reduwan NH, Aziz AA, Mohd Razi R, et al. Application of deep learning and feature selection technique on external root resorption identification on CBCT images. BMC Oral Health. 2024; 24(1): 252. doi: 10.1186/s12903-024-03910-w
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Asha Shanmugham, Parvathy Meenakshi Sundaram
This work is licensed under a Creative Commons Attribution 4.0 International License.
