April 25, 2023, #37
ChatGPT and other AI writing tools have been in the news regularly, with widespread interest among universities in finding detection tools. In January 2023 Nadine Yousif wrote an article for BBC News called “ChatGPT: Student builds app to sniff out AI-written essays”. She reported on Princeton senior Edward Tian, who built “an application that can determine, with high accuracy, if a text was written by a human or a bot.” He called his app GPTZero. “The app works by looking at two variables in a text - perplexity and burstiness - and it assigns each of those variables a score.”¹ The first factor depends on how similar the text is to the texts it was trained on. A high level of perplexity means that the text is “more likely to be human-written”.¹ The relevance of this variable may depend on knowing which sources the AI had at its disposal, and that will likely depend on the topic. The other factor is “burstiness” meaning the “mix of short versus long sentences” rather than sentences that are “more levelled and uniform”.¹ “‘If you plot precisely over time, a human-written article will vary a lot,’ Mr Tian said. ‘It would go up and down, it would have sudden spikes.’"¹ His test results were definitely good: GPTZero had “less than 2% false positive rate” in tests when he fed “the app BBC articles written by journalists, versus articles written by ChatGPT using the same headline as a prompt.” A clear danger here is that people will assume an even higher level of reliability, rather than think about what affects GPTZero’s accuracy. One factor could be knowing probable original sources. Guessing those sources could be easier under controlled circumstances such as university classes, where the topics are known. Another factor is the language of the suspicious text, since not all languages will show the same “burstiness”. What is important here is not just the tool itself, which will likely change over time, but how Edward Tian approached the problem, which others can and should learn from. Special thanks goes to our colleague Isaac Sserwanga, who brought this to our attention.
1: Nadine Yousif, ‘ChatGPT: Student Builds App to Sniff out AI-Written Essays’, BBC News, 13 January 2023, sec. US & Canada,https://www.bbc.com/news/world-us-canada-64252570.