Between Argument and Influence – How Persuasive Can AI Be?

AI systems argue politely, in a structured manner, without errors and usually without emotional bias. They do not contradict, they package complex topics in simple language, and they are available around the clock. These characteristics ensure that many users perceive AI as particularly competent, trustworthy – and also convincing.

Two recent examples from research and the media show the risks that arise when this persuasiveness is combined with deliberate manipulation – and the role that UX plays in this.

What happened?

A study by the École polytechnique fédérale de Lausanne (EPFL) shows that ChatGPT was able to convince test subjects more than real people in 64% of cases. This effect was particularly pronounced when the AI had personalised information about the target person, such as their age, profession or political views. This made the arguments more targeted, more relatable and more successful.

At the same time, Der Spiegel reported on a Russian disinformation campaign in which content was deliberately fed into the digital space via a network called ‘Pravda’ with the aim of infiltrating it into the training or behaviour of Western AI systems. In random samples, around a third of chatbot responses contained narratives from this network without identifying them as such.

Persuasiveness as a risk

Psychological impact

Large language models (LLMs) such as ChatGPT are not only capable of providing accurate information – they can also present it in a particularly engaging way. The combination of calm language, logical reasoning and perceived objectivity means that statements made by AI systems are often less scrutinised than those made by real people.

An additional amplifying effect occurs when AI provides individualised arguments. This form of personalisation exploits psychological patterns such as confirmation bias – the tendency to accept only arguments that fit one's own worldview.

In combination with disinformation

When this persuasiveness is combined with systematically introduced false information, a highly problematic scenario arises. In the case of the Russian network ‘Pravda’, over 150 websites with SEO-optimised propaganda were set up. The aim was to feed this content into AI systems through sheer volume – a practice known as ‘LLM grooming’.

This is not about classic hacks, but rather subtle influence via content. Further details on this phenomenon and countermeasures can be found in the linked blog article:

👉 What is LLM grooming – and how can AI systems be protected against it?

Significance for UX

UX design is increasingly more than just interface design – it is the space where persuasion, trust and control are negotiated.

When AI-based systems are more persuasive than humans, UX professionals face new challenges:

• How is argumentation designed?

• What role does personalisation play?

• How visible are sources and uncertainties?

• How can users be enabled to think critically?

UX thus becomes a designer of impact – and shares responsibility for whether users experience AI systems as supportive or manipulative.

Recommendations for action and outlook

In order to adequately address the growing persuasiveness of AI, clear design principles and new standards for dealing with language models are needed. These include, among other things:

• Improve source attribution: Information should be verifiable and contextualised, not just appear as continuous text.

• Make personalisation transparent: Users should be able to recognise whether and how their data influences the argumentation.

• Allow ambiguity: AI should not only provide clear answers, but also be able to deal with uncertainties.

• Identify responsibility: Who is speaking here – and with what intention?

• Adapting UX research: New methods for measuring the impact of AI responses, for example in terms of persuasion, trust or critical distance, are needed.

  
  

Conclusion: The persuasiveness of AI is not a minor issue. It is central – and it must be considered, designed and critically monitored. UX design can and should play an active role here.

💌 Not enough? Then read on – in our newsletter. It comes four times a year. Sticks in your mind longer. To subscribe: https://www.uintent.com/newsletter