TL;DR Breakdown
- LLMs like GPT-4 offer cost-effective data analysis with comparable performance to human analysts.
- GPT-4 is significantly cheaper and faster in completing tasks than human analysts.
- The expanding use of LLMs raises concerns about job security in various industries.
In a recent study conducted by Alibaba’s Damo Academy and Singapore’s Nanyang Technological University, it was found that large language models (LLMs) like GPT-4, developed by US start-up OpenAI, can perform data analysis at a fraction of the cost of hiring a human analyst while delivering comparable performance. The study sheds light on the potential threat to job security as generative artificial intelligence (AI) becomes increasingly adopted across various industries.
GPT-4 gets high marks for alignment and fluency
According to the findings posted on the preprint server arXiv, the cost of employing GPT-4 is only 0.45 percent of hiring a senior data analyst earning around US$90,000 per year, or 0.71 percent of hiring a junior-level employee. The researchers automated the entire data analytics process using GPT-4, providing questions and data for extraction and analysis, ultimately generating insights and graphs.
Comparing the results of the analysis with those produced by human data analysts in terms of performance, time, and costs, the study revealed that GPT-4 is not only significantly cheaper but also much faster in completing tasks. The AI model demonstrated superior performance compared to entry-level human analysts, evaluated through metrics such as chart correctness, fluency, and the quality of insights produced. It also exhibited “comparable performance” to senior-level analysts, with advantages varying across different cases and metrics. In certain instances, GPT-4 even surpassed human analysts in terms of figure correctness, analysis, and complexity of insights.
GPT-4 received full marks for alignment and fluency in generating grammatically correct analysis. However, it fell behind humans in terms of accurately presenting data in graphs, as well as in aspects of presentation and formatting in some cases. Despite these limitations, the study emphasized that GPT-4 could still generate accurate analysis.
Can GPT-4 fully replace human data analysts?
The study highlights the expanding application of LLMs in various industries, where they are expected to improve efficiency while posing a potential threat to human jobs. Chinese tech companies, including Alibaba and Baidu, are actively developing their own ChatGPT-like services. For instance, Alibaba has introduced Tongyi Qianwen, its own AI service, for beta testing with corporate clients, while Baidu recently unveiled its Ernie Bot service.
It is important to note that further research is required before definitively concluding that LLMs like GPT-4 can fully replace human data analysts. Nonetheless, the study’s findings underscore the potential of LLMs in revolutionizing data analysis while raising important considerations about job security and the evolving role of AI in various industries.
