ChatGPT Falls Short of Human Accountants in Accounting Tests

In this post:

TL;DR Breakdown

  • OpenAI’s AI chatbot, ChatGPT, was tested by researchers at Brigham Young University and 186 other universities to assess its performance in accounting exams.
  • Students outperformed ChatGPT, scoring an average of 76.7% compared to ChatGPT’s score of 47.4%.
  • ChatGPT performed well on accounting information systems and auditing but struggled with tax, financial, and managerial assessments.

As artificial intelligence (AI) technology continues to advance, it is transforming the way we work and challenging the expertise of various professions. Microsoft-backed OpenAI’s AI chatbot, ChatGPT, has made significant strides in generating natural language text and passing advanced exams. However, a recent study conducted by researchers at Brigham Young University (BYU) and 186 other universities found that the AI chatbot falls short of human accountants in accounting tests.

ChatGPT’s Limitations and the Future of AI in Accounting

The study found that while ChatGPT performed well on some accounting exams, students outperformed the AI chatbot in overall accounting exams. The findings suggest that while AI technology has its strengths, human expertise remains essential in certain fields, including accounting, where complex mathematical processes and judgment calls are required.

Lead study author and BYU accounting professor, David Wood, recruited professors and students to test ChatGPT’s abilities in accounting. The questions covered topics such as accounting information systems (AIS), financial accounting, auditing, managerial accounting, and tax.

ChatGPT’s performance was impressive, scoring 47.4% on the tests. However, human students outperformed the AI chatbot with an average score of 76.7%. The study found that ChatGPT struggled with mathematical processes required in tax, financial, and managerial assessments. It also had difficulty with short-answer questions and higher-order questions.

One of the key limitations of ChatGPT was its tendency to make nonsensical errors, such as adding two numbers in a subtraction problem or dividing numbers incorrectly. It also provided incorrect explanations for its answers and sometimes made-up facts.

While the researchers noted ChatGPT’s limitations in accounting tests, they believe that future iterations of the AI chatbot, such as GPT-4, will improve exponentially in accounting and other fields. However, as Jessica Wood, a freshman at BYU, noted, “It’s not perfect; you’re not going to be using it for everything. Trying to learn solely by using ChatGPT is a fool’s errand.”

The study’s findings have implications for the use of AI chatbots in education and the accounting profession. While AI chatbots can supplement learning and improve efficiency in certain tasks, they currently fall short of human expertise in accounting. As the accounting profession continues to evolve, accountants will need to adapt and incorporate new technologies while maintaining their expertise and value in the field.

The Future of AI in Various Industries

Despite ChatGPT’s limitations in accounting tests, its abilities in generating natural language text can still be leveraged in various industries. For example, ChatGPT can be used in customer service to generate responses to frequently asked questions or in content creation to generate articles and reports.

The study’s findings highlight the importance of incorporating human expertise in the development and testing of AI chatbots. By working together, human experts and AI developers can create tools that supplement human expertise and improve efficiency in various tasks.

While ChatGPT may not yet match the abilities of human accountants in accounting tests, its capabilities in natural language generation show promise in various industries. As AI technology continues to evolve, it will be interesting to see how it is integrated into different professions and how it changes the way we work.

Share link:

Most read

Loading Most Read articles...

Stay on top of crypto news, get daily updates in your inbox

Related News

Subscribe to CryptoPolitan