“What Could Possibly Go Wrong?”: A Consumer-Centric Approach to Exploring Vulnerability and Algorithmic Bias in AI-Driven Financial Planning

With the accessibility of generative AI, the uptake of consumers using it for their financial planning queries and needs has exponentially increased. However, algorithmic bias in AI-driven financial planning poses significant challenges, particularly to consumers experiencing vulnerabilities as it can cause further marginalization by the inherent biases in the algorithms that drive decision-making processes. The aim of this research is to determine the extent to which biases manifest in the output of AI models and explore whether consumer vulnerabilities are considered in its recommendations. Therefore, we employ an experimental research design by generating prompts that are based on a financial need with varying states of vulnerability. Through a content and discourse analysis, a comparison of three AI tools, namely Perplexity, ChatGPT, and Claude, is provided. Our findings reveal that while biases were not explicitly pronounced in the output, the vulnerabilities presented were not robustly considered. Perplexity often referred the user to seek professional advice but lacked detail; ChatGPT avoided implicit bias but under-addressed vulnerabilities, and Claude gave thorough recommendations that required self-guided financial planning. Furthermore, we found that at face value, the model outputs were similar, but upon analysis, there were variations in the recommendations provided which had long-term implications for consumers experiencing vulnerabilities. This paper presents key considerations for consumers engaging with AI in the context of financial planning. Understanding how the AI models respond to and account for consumers experiencing vulnerability is crucial for advancing the way in which AI is applied as an avenue for financial advice and how it can be used for a more inclusive financial landscape.