Most people prefer AI-written stories – and can’t tell when they are made by humans

Martin Parr/Magnum Photos AI-generated literature may have an unfair reputation for poor quality. Despite a cultural backlash to AI-generated content, people seem to prefer it – and if they’re lied to and told that it was created by a human, their opinion rises even higher. Sydney Sears and Deena Weisberg at Villanova University in Pennsylvania recruited 1682 adults – selected to be representative of the US in terms of their gender, age, and race – and asked them to read a story. Half the participants read a story written by a human, and half read one written by AI. Each of these groups was again split in two, with half told that their story was written by a human and half that their story was written by AI, regardless of its true origin. The human-created stories were sourced from a literary journal, while the AI-generated ones were created using ChatGPT 4.0. Those who read an AI-generated story ranked its quality as 1.54, on average, when asked to score it on several qualities on a scale ranging from -3 to 3. Meanwhile, participants who read a human-written story gave it an average score of just 0.97. Participants who read an AI-generated story also ranked it as more absorbing, based on several qualities on a similar scale, with an average score of 1.42. Those who read a human-written story, meanwhile, gave an average score of just 1. In all cases, the story was rated slightly higher if the participant was told it was written by a human. The researchers also ran two further studies, where several hundred people were given an AI-generated story and a human-written story and asked to identify which was which. In the first test, only 39 per cent made a correct identification, while in the second, 52 per cent were correct. The researchers have no explanation for the difference, except that the second test took place several months after the first; society may be adapting to better understand AI output. In any case, they claim that it shows people are no better than chance at distinguishing human work from AI output. Claire Hardaker at Lancaster University, UK, says that humans are having to come to terms with the uncomfortable idea that we aren’t any better than robots at writing short stories. She says that in her own research on testing people’s ability to identify AI-generated speech, text and music, it is the music that angers them most of all when they are tricked. “I think there’s a huge sensitivity [around AI generating art],” she says. “When you have AI that is continually improving at the speed that it’s improving at, there is going to come a point where every single model can produce Shakespeare-like output.