How to Tell if Something is Written by ChatGPT It may not always but obvious, but there are a few telltale signs that can help you determine if a body of text was written by an AI.
By James Jones
This story originally appeared on Readwrite.com
As we all come to terms with a world in which artificial intelligence is now mainstream and no longer a fantasy only seen in the movies, the need to know what elements of day-to-day life and experiences AI has influenced has never been greater.
OpenAI's ChatGPT was the first mainstream example of how useful AI can be for various people, businesses, and industries, but that doesn't mean it's perfect, accurate or the ultimate assistant to our everyday needs.
ChatGPT has caused controversy due to occasionally providing incorrect, inaccurate, or unfactual responses to questions. This has resulted in people's legal troubles, and it's even gotten its creators, OpenAI, in trouble.
In 2023, it falsely claimed an American law professor had made inappropriate comments and sexual advances towards a student while on a school trip and even cited The Washington Post as the source of the information. There was no school trip, and there was no article in the Washington Post. Put simply, it completely made the whole thing up.
That's just one of several alarming flaws in the chatbot's capabilities. Overall, it can be a very useful source of information, but what it says cannot and should not always be taken as fact.
But as more and more people use ChatGPT to complete everyday tasks online or as part of their work, it's quite possible that every time you go online, you're consuming at least one piece of information that was generated or written by AI, and likely ChatGPT itself.
As a result, the internet is fast becoming a minefield where the lines between human and AI-written text are blurred. There may be a time in the future when that won't be an issue — and ChatGPT-5 could be around the corner — but for now, people need to be vigilant and know when something is written by ChatGPT.
What are the signs that something is written by ChatGPT?
As humans are responsible for prompting ChatGPT, the detail of the chatbot's response is often decided by the detail in the prompt it's given. If it lacks instruction, especially in more detailed subjects, it is more likely to return a vague or inaccurate response.
Related: Chat GPT Urgent Warning: Is Your Information, Job or Business Safe?
To the untrained eye, like someone with no particular background knowledge on the subject, that may not be obvious. But to others, it might be glaringly obvious that the text was written by ChatGPT.
These are the key things to look out for…
General use of language and repetition
ChatGPT is a 'narrow' AI, which means it is not able to understand or replicate human emotion or behavior and cannot think for itself. This means its responses can often be devoid of personality or creative language.
Moreover, even though it does make mistakes, it's trained to make as few mistakes as possible. Therefore, responding in a simple and somewhat robotic tone is designed to help minimize mistakes and inaccuracies.
This is mostly obvious if you were, for example, to ask it to write you a review for your favorite film or product. It might leave out key information about what it's reviewing, such as actors' names or the product's dimensions.
Therefore, if you're reading something like this on a review site and it appears to omit what would normally be considered key information, there is a chance it was written by ChatGPT.
The same applies to the repetition of words or phrases. ChatGPT is trained on vast amounts of data and language but it might still use repetitive language, especially in larger blocks of text.
Hallucinations
The aforementioned case involving a law professor is the perfect example of an AI hallucination, where the AI completely makes something up in its response. This remains one of the biggest issues with generative chatbots like ChatGPT.
AI experts recommend fact-checking ChatGPT responses, particularly when involving more niche information and subjects. There are many more cases of hallucinations, some more serious than others, so it's always advisable to triple-check its responses with other sources of information before trusting its accuracy.
If you're reading about a subject you know well, this will be much easier to spot. For example, a match report on a recent soccer game that you watched will likely make it easy for you to identify any factual inaccuracies. But if you're doing, say, research on the thermic effect of food, it may be harder to spot if it's human or AI-generated.
Copy-and-Paste Errors
This is arguably the easiest one to spot. People have been known to accidentally copy and paste ChatGPT's response and its side comments like 'Sure, here's a movie review for….'
That's a surefire way of knowing if something you're reading was written by ChatGPT or a human, purely based on human error, as opposed to an error on behalf of the AI.
Read the text thoroughly
ChatGPT is trained to sound and respond like a human, so it will be near impossible to tell if something was written by AI if you only read one or two sentences. It will be clearer if you read the entire article thoroughly, which may highlight particular hallucinations, repetition, general language or copy-and-paste errors.
Of course, humans can edit ChatGPT's responses to remove much of the above and make the text more human-like, but often, that will involve a level of editing that would likely make it easier for humans to write the text themselves.
How to detect ChatGPT content
The rise of ChatGPT and similar AI chatbots has led to an array of AI content detectors popping up, which all claim to be able to detect when a piece of text is human or AI-written.
We've already looked at the best AI content detectors, which can even indicate exactly what parts of text are human and what parts are AI. Some will give you a percentage estimate of human vs. AI in the text it's given to analyze.
However, none of these AI content detectors are 100% perfect, and there will always be occasions when they incorrectly detect human writing as AI and vice versa.
However, using an AI detector is a good way to see whether there are any signs of AI in what you're reading, especially if you still have suspicions after checking for repetition, hallucinations and general language.
But the key takeaway, especially when referring to niche subjects you're not completely familiar with, is to always fact-check what ChatGPT tells you. Better to be safe than sorry.
