Integrated GenAI Data Observatory Framework for Evidence-based Analytics of Malaria Burden

Abstract

Minimal size of data can degrade the performance of machine learning models, leading to overfitting. This limitation hinders the transferability and generalizability of the models to different contexts or datasets. In resource-constrained settings, the lack of adequate data inhibits the development of robust, data-driven systems capable of delivering accurate insights. However, Generative Artificial Intelligence (GenAI) leverage Large Language Models (LLMs) to create and synthesize a wide array of meaningful content, including text, images, audio, video, and multimedia data. These models generate synthetic data that replicate the statistical properties of real datasets, thereby augmenting existing data and improving the overall quality of inputs available for machine learning tasks. In this paper, an integrated GenAI-enabled data observatory framework designed to enhance the storage and analytical processing is proposed to improve the quality of evidence of predictive analytics tasks. This framework employs advanced data warehouse technology combined with the capabilities of LLMs to facilitate a more effective data management system. To validate the machine analytics layer, data is sourced from the repository of Malaria Indicator Survey (MIS), which provide in-depth insights into the malaria burden faced by rural communities across Nigeria. The machine analytical layer of the proposed framework demonstrates percentage increases in overall accuracies with Decision Tree (DT) and Extreme Gradient Boosting (XGBoost) algorithms, achieving improvements of 23.81% and 15.38%, respectively, when compared to using the MIS data alone.