Multimodal Generative AI Framework for Advancing Sustainable Development Goals

The Sustainable Development Goals (SDGs) established by the United Nations provide an ambitious framework aimed at fostering a more just, sustainable, and prosperous global society. Recently, artificial intelligence (AI) has become a significant asset in striving towards these objectives. This paper outlines an innovative computational framework utilizing multimodal generative AI models to enhance assistive technologies that support various SDG initiatives. The proposed framework consists of three distinct architectural phases, each built upon novel model designs that merge image and text embeddings along with their combined representations. In Stage I, the architecture employs multimodal image-text embedding through contrastive learning and image pre-training, facilitating zero-shot classification. Stage II features a Multimodal RAG architecture that integrates CLIP with open-source large language models (LLMs), specifically leveraging GPT-2 and GPT-J in a cascading manner. Finally, Stage III utilizes a CLIP-assisted diffusion model that iteratively refines images through denoising random noise based on text prompts for high-quality image synthesis. Performance assessments for each stage were conducted using several open-source datasets and validated against image-text descriptions for specific use cases tied to the SDGs. The encouraging results from this study highlight the transformative capabilities of multimodal generative AI frameworks in furthering the United Nations’ Sustainable Development Goals by combining various data types—particularly text and images—for comprehensive insights and practical solutions.