Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media

Grimme, Britta; Pohl, Janina; Winkelmann, Hendrik; Stampe, Lucas; Grimme, Christian

Abstract

This paper addresses new challenges of detecting campaigns in social media, which emerged with the rise of Large Language Models (LLMs). LLMs particularly challenge algorithms focused on the tempo- ral analysis of topical clusters. Simple similarity measures can no longer capture and map campaigns that were previously broadly similar in con- tent. Herein, we analyze whether the classification of messages over time can be profitably used to rediscover poorly detectable campaigns at the content level. Thus, we evaluate classical classifiers and a new method based on siamese neural networks. Our results show that campaigns can be detected despite the limited reliability of the classifiers as long as they are based on a large amount of simultaneously spread artificial content.