We showed 697 people true and false tweets on topics like vaccines and climate change, written either by Twitter users or by the AI model GPT-3. GPT-3’s accurate tweets were recognised faster and more often, its false ones fooled people slightly more, and participants did no better than chance at spotting the machine.
Abstract
Artificial intelligence (AI) is changing the way we create and evaluate information, and this is happening during an infodemic, which has been having marked effects on global health. Here, we evaluate whether recruited individuals can distinguish disinformation from accurate information, structured in the form of tweets, and determine whether a tweet is organic or synthetic, i.e., whether it has been written by a Twitter user or by the AI model GPT-3. The results of our preregistered study, including 697 participants, show that GPT-3 is a double-edge sword: In comparison with humans, it can produce accurate information that is easier to understand, but it can also produce more compelling disinformation. We also show that humans cannot distinguish between tweets generated by GPT-3 and written by real Twitter users. Starting from our results, we reflect on the dangers of AI for disinformation and on how information campaigns can be improved to benefit global health.
Spitale, G., Biller-Andorno, N., & Germani, F. (2023). AI model GPT-3 (dis)informs us better than humans. Science Advances, 9(26). https://doi.org/10.1126/sciadv.adh1850
@article{spitale2023ai,
author = {Spitale, G. and Biller-Andorno, N. and Germani, F.},
title = {{AI model GPT-3 (dis)informs us better than humans}},
journal = {Science Advances},
year = {2023},
volume = {9},
number = {26},
doi = {10.1126/sciadv.adh1850}
}
