AI Detectors Stop Working: US Constitution Case Study

Artificial Intelligence (AI) has taken over modern-day life, particularly in generating written content. While it is all fun and games, the inaccuracy in detecting AI content is alarming. What’s worse is AI detectors marking human written content as GPT generated; Case study the United States Constitution.

Also Read :Y Combinator’s Paul Graham Slammed for Claiming Texts With “Delve” Are Written by AI

The United States Constitution was written in 1787, ratified in 1788, and has been in operation since 1789. It is the world’s longest-surviving written charter of government. The Constitution was around a century before the first car was invented. Yet, an AI detector marked it 97.97% AI-generated.

AI Detectors Mark the US Constitution as Inhuman

2023 saw an increase in artificial intelligence written content. Soon after that, market developers took the opportunity to create detectors that inform AI content percentages. That should have been a good advancement, but that’s not the case.

An AI detector has marked the United States Constitution as AI content. The percentage? A closing 97.97%. The term “artificial intelligence” was coined and became popular in the birth of AI between 1950 and 1956. So, how did AI possibly write a document 163 years older than its invention?

In 1950, Alan Turing published a journal article titled “Computer Machinery and Intelligence,” proposing a test of machine intelligence called The Imitation Game. This article marked the beginning of the world we live in today.

In 1952, a computer scientist named Arthur Samuel developed a program to play checkers, the first to learn the game independently. Soon after, in 1955, John McCarthy held a workshop at Dartmouth on “artificial intelligence,” which was the word’s first use.

AI Detectors Have Limitations

AI detectors have limitations because AI-generated text mimics human writing. These detectors have been trained by the same humans who taught machines how to write. So, how would their AI detectors work, and how accurate are they? Another AI detector scored a 1% AI content on the US Constitution. Where did the 1% come from?

Well, here is the disparity: Some AI detectors look out for certain words like “delve, tapestry, utilize, numerous, etc.” that are usually used by AI but are uncommon to Americans. 

However, for some people from countries with British colonial history, these words are commonly used in both spoken and written language.

Last month, Y Combinator co-founder Paul Graham came under fire following a ChatGPT text attribution theory he made on X (formerly Twitter). Graham posted a cold email proposal he received from someone for a project that he concluded was AI writing because it contained the word “delve.

One X user said, “People who have learnt English in the countries which have British colonial past – this is exactly how we learnt English […] ‘Delve’ is a normal word which we speak.” Another said that it’s an American problem.

Some countries and people still use what I would describe as “literature english”.

The unfortunate thing is that the American vocabulary is primarily based on colloquialisms (what you call slang) and American contemporary idioms, to the extent that the sophisticated use of…

— Africa Research Desk (@MightiJamie) April 10, 2024

This begs the question, should AI detectors and content generators be regional-english specific? Or should all these writing tools and detectors be done with?

AI Detectors Are Inaccurate

AI detectors rely on inconsistent linguistic features. Human language is inherently variable, with a wide range of styles, dialects, and idioms. AI-generated text can fall within these variations, making it difficult for detectors to be accurate.

About 20% of AI-generated texts pass through these detection systems undetected, undermining the reliability of AI detectors. Similarly, about 25% of human written text is falsely flagged as AI.

One user on Quora wrote, “AI detector is detecting my work as 100% AI generated. Because I ignorantly used it as an editing and grammar tool at the request of an academic advisor. The words, concept & life experience in the already published piece is my own. What do I do?”

Many have expressed frustration with this issue. Recently, in a news piece put up by Cryptopolitan, an AI Generator flagged words spoken by Donald Trump. He said, “I am very positive and open-minded to cryptocurrency companies and all things related to this new and burgeoning industry.” ⁤How is AI added to spoken words? Is it in a person’s mind? In expressed thoughts?

Other content was from David Bailey: “⁤We will mobilize to defend ourselves. ⁤⁤We’re not voting for Trump per se, we’re voting against Biden. ⁤⁤The only person to blame for that is Biden. “

Also Read: The Trump Crypto Campaign – Everything You Need to Know

In 2023, OpenAI published tips for educators in a promotional blog post showing how some teachers use ChatGPT as an educational aid.  

In a section of the FAQ titled “Do AI detectors work?” OpenAI wrote, “In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.” The problem of distinguishing human content from AI content remains. The question is: What should be done moving forward?

---

Cryptopolitan reporting by Florence Muchai

From Zero to Web3 Pro: Your 90-Day Career Launch Plan