In the ever-evolving landscape of financial technology, the rapid integration of artificial intelligence (AI) and machine learning is poised to reshape the industry. But, amidst this groundbreaking transformation, a critical concern emerges—AI Bias. As financial institutions increasingly embrace AI models, the ethical dimensions surrounding bias, fairness, and societal impact come to the forefront, posing potential risks to both individual firms and the overall stability of the financial system.
AI bias in financial models
Artificial intelligence, a key player in the financial realm, is anticipated to witness a 3.5-fold increase in adoption over the next three years. While AI models promise to revolutionize customer interactions and financial decision-making, a shadow looms over the industry—AI bias. The use of biased data or unethical algorithms, as outlined in DP5/22, not only raises concerns about consumer protection but also introduces financial stability and monetary stability risks.
AI models, distinct from traditional rule-based financial models, possess the ability to learn iteratively and adapt, a feature that empowers them to understand and respond to complex financial scenarios. But, this adaptability comes with a price—potential bias. Pure machine-driven AI models, devoid of human intervention, can produce outputs skewed by biases inherent in training data or model structures. For instance, a healthcare algorithm in the insurance sector exhibited bias, underestimating the severity of health conditions for Black patients compared to their White counterparts, leading to unequal healthcare provision.
The dual faces of AI bias
Understanding AI bias requires delving into its dual facets: data bias and societal bias. Data bias originates from the training data itself, embedding societal biases that can be perpetuated on a larger scale. Joy Buolamwini’s study on facial recognition software exemplifies data bias, where a skewed training dataset led to higher error rates for minority ethnic individuals. Attempts to eliminate data bias by excluding protected characteristics may backfire, as non-protected features can act as proxies, perpetuating discriminatory decision-making.
Societal bias, on the other hand, stems from societal norms and historical legacies. A recruitment algorithm developed by Amazon showcased gender bias, negatively scoring female applicants due to a decade-long training dataset reflecting the male dominance in the industry. Blind spots to gender bias, if not identified, can permeate financial systems, impacting trust and potentially leading to financial instability.
The acknowledgment that AI could influence financial stability introduces a nuanced dimension to the AI bias discourse. Black box models utilized by multiple firms in trading strategies pose challenges for both market participants and supervisors in predicting market impacts. Beyond market dynamics, issues of fairness take center stage as they intertwine with financial stability, emphasizing the pivotal role of trust.
Trust, a cornerstone for financial stability, can be eroded by biased AI applications. The De Nederlandsche Bank underscores the importance of fairness in AI applications to maintain societal trust in the financial sector. Disparities identified by Bartlett et al. in FinTech algorithms, though reduced compared to face-to-face lenders, indicate persistent discrimination. Trust in AI is crucial not only for the overall stability of the financial system but also for individual institutions, as reputational and legal risks associated with biased AI usage can lead to material losses.
While tangible incidents of AI-related risks are yet to materialize on a large scale, the case of Apple and Goldman Sachs’ credit card algorithm offers a glimpse into potential pitfalls. The model, seemingly unbiased in its inputs, demonstrated biased lending decisions, highlighting the nuanced nature of AI bias. This incident, while not violating fair lending requirements, sparked public discourse on the wider implications of sex-based bias in algorithms.
As AI continues its ascent in financial services, the ethical minefield of bias and fairness must be traversed with caution. Beyond its inherent issues, biased AI poses a potential threat to financial stability. Central banks, foreseeing the acceleration of AI adoption, face the crucial task of evaluating the risks posed by bias, fairness, and ethical concerns. The delicate balance between technological advancement and ethical responsibility will shape the future landscape of AI in finance, determining whether it becomes a force for positive transformation or a source of instability.
