The check says: everything is fine. The check is wrong.  

You have automated an analysis. You even thought ahead and put a second agent behind it to check the result. It reports back: plausible, complete, no anomalies. You breathe a sigh of relief and pass the result on.  

What you don't know: The checking agent is wearing the same glasses as the one being checked. It shares its blind spots. It catches careless mistakes – but not the systematic misreading that both have. Your checkpoint was not a checkpoint. It was an echo.  

Here is the problem: Most debates about automation revolve around the question "How can I automate even more?" That is the wrong question. The right one is: Which part of this task am I even allowed to hand over – and how hard must the checkpoint be that I keep?  

Time to change that. In this workshop, you will not build a pipeline that you take home and that is outdated by the next product release. You will develop a judgment: the ability to decide for yourself for every future research task where the automation boundary lies and how stable the checkpoint at this boundary must be. This is more uncomfortable than a recipe. But it lasts longer. 

Why You Should Attend

• Automating without compromising quality: You learn to derive the boundary from the task – not from what your tool currently offers.  

• Building checkpoints that hold: A rule in an instruction file is a request, not a command. You learn where you need hard gates and how to get them.  

• Avoiding the verification trap: A checking apparatus can certify a reliably distorted pipeline – with a seal of approval. You learn to recognize this.  

• Four runs instead of one recipe: You make the same cut on four different tasks. Only repetition turns it into a skill.  

• Knowledge that survives a tool change: If your company changes platforms tomorrow, the map moves with you.  

What Awaits You

What changes when the human steps out of the loop?

• Why your manually built analysis process has actually been an agent at its core for a long time – and the step to real agents is smaller than you think  

• How a wrong input propagates through an automatic chain: unobserved, smoothed out, and ultimately more convincing than before  

• Why there is no magic pipeline – control does not disappear, it relocates  

Gate Audit: Where was the control located until now?

• Putting your familiar analysis process on the table and marking every gate  

• Two questions per gate: What happens to it – and how hard must it be?  

• The uncomfortable discovery: Several gates that you considered guarantees are, upon honest reflection, only soft requests  

The 2×2 Map: Where is the boundary?

• Is the task specifiable? Can the result be verified affordably? That is all it takes  

• Four quadrants, four answers – from "fully automate" to "leave with the human"  

• Why the boundary follows from the task and not from the maturity of your setup  

The Hardness Scale: How stable is your checkpoint really?

• Hard, semi-hard, soft, circular – four degrees of hardness and what they are good for  

• Why instructions to a language model are interpreted and not executed  

• The only truly hard checkpoint in your system is ultimately you – and you are only as hard as your methodological competence  

The Architecture of Agentic Systems – in every tool

• Six layers: Data access, permanent rules, processes, decomposition, hard gates, connectors  

• The translation table: What the same layers are called in Claude, ChatGPT, and Microsoft Copilot  

• Deliberately kept short – this is not tool training  

Run A: Your first own automation cut

• Building a clearly automatable step as a reusable process  

• PII redaction and counting – and why a hard check is absolutely necessary here  

• The click: I can automate and know that it's correct – because the task allows it  

Run B: When the check says "all good" and is still wrong

• Automating thematic clustering and integrating an AI-supported reliability check  

• Comparing the result against a manually created reference result  

• Experiencing circularity firsthand: An AI checker inherits the blindness of what it is supposed to check  

Run C: When a task fights back

• Wanting to automate a real question of interpretation and prioritization – with little help  

• Two honest outcomes, and both are a result  

• Why the result looks finished long before it is correct  

Verification that holds up

• Data test sets: a strong tool – and still no proof of correctness  

• Self-catches, forced stops, risk-oriented spot checks, deterministic checking steps – each with an honest hardness degree label  

• Why the answer key itself is the locus of competence: Whoever writes the reference result needs research methodology  

Multi-Agent: Lever or Overkill?

• Where parallel sub-agents raise quality – and where they create false precision  

• Three agents that agree are not three independent opinions 

   

What We Work With

A continuous scenario runs as a common thread – a fictional startup that wants to improve the customer journey around dentist visits. Fictional data, real problems. There are two reasons for this: An agent that independently works its way through a folder of real transcripts significantly increases the attack surface. And customer sandboxes are often simply not designed for agent access. You will still learn how to handle personal data – using the fictional material. Making mistakes without them hurting. And if you have an environment that allows it – sandbox-ready, data protection clarified: Then we are happy to work on your own cases in the in-house format. Let us know in good time, and we will clarify in advance whether it is feasible.  

In Which Tool?

In yours. The workshop runs with Claude, with ChatGPT, and with Microsoft Copilot – we will determine the reference tool together when booking. This is not a concession to your IT, but the whole point: What you learn here are properties of tasks, not properties of tools. Whether a task is specifiable does not change if you switch products. A workshop that runs everywhere proves exactly that.  

Who Is the Workshop For?

For UX Researchers who have already taken the first step.

This workshop requires foundational craft: You know how an LLM works. You know why hallucinations are not a bug, but part of the design. And you have already worked with prompt chains instead of just asking questions.

In concrete terms, this means: You have either attended our workshop "Efficient Use of AI and Advanced Prompting for UX Professionals" – or you are clearly advanced in the use of LLMs and bring the knowledge from elsewhere. Both are equally welcome.

Why we are saying this so clearly: Because we want to spare you a day of catching up on basics while the group moves on. This workshop picks up where the other one leaves off. If you are unsure whether you fit in – just drop us a line. We will tell you honestly what we think.

And in case you are afraid of being underchallenged because you have already built agents: The question of which of them you would have been better off not building is also new for technically savvy people. Having built ten skills does not mean having spent an hour deciding which of them should not have been created.