AI-Powered Discussion Facilitation

Collaborators: Mirta Galesic, Joshua Becker

Tools & Languages: Python, R, DeliberateLab, oTree (Django)

Methods: Experiment Design, Survey Design, Large Language Models, Linear and Logistic Mixed Effects Regressions

🔭 About

Online discussions frequently devolve into unproductive conflict. Most platforms respond through moderation (removing or flagging content) but moderation is reactive and doesn’t repair the conversation. Facilitation offers a different approach: a third party intervenes not to remove content, but to redirect the exchange. While facilitation is well-established in negotiation and mediation, two key questions remain open: which specific interventions actually work online, and whether people would accept an AI as a legitimate facilitator at all?

The second question is particularly interesting because AI facilitation sits at the intersection of two competing tendencies in the literature. People tend to resist algorithmic decision-making in emotionally and socially charged domains (algorithmic aversion). But AI can also be preferred precisely for the consistency and impartiality it offers (algorithmic appreciation). Online discussions seem to demand both, which makes it unclear which tendency wins. This project addresses both questions through two pre-registered studies.

👾 Methods

Study 1: Survey Experiment

To isolate perceptions of AI versus human facilitation, we generated high-conflict conversations using LLMs and followed each with a facilitator message labeled as either AI or Human while keeping the actual message text identical across conditions. This gives clean experimental control over the label while holding content constant. We tested 10 intervention types drawn from the literature: emotional validation, perspective-taking, bridging, finding common ground, descriptive norms, and others. Participants rated facilitators on legitimacy, fairness, effectiveness, trust (warmth and competence), and willingness to engage.

Study 2: Live Conversation Experiment

In Study 2, participants are paired with someone who genuinely disagrees with them on a political or non-political topic and have a live text-based conversation, with real-time AI facilitation. Unlike Study 1, where people observe facilitation from the outside, here they experience it directly giving us access to richer behavioral outcomes.

The facilitation system is built around two independent AI agents:

Facilitator

This agent monitors the conversation for signs of unproductive exchange and decides in real time whether and how to intervene. It fires only when a triggering condition is met: a rebuttal or counter-argument, signs of frustration or dismissiveness, an unsupported or one-sided claim, or a pattern where both participants are restating their own views without engaging the other’s. It stays silent if the conversation has already reached genuine resolution. To avoid being intrusive, it is capped at a small number of interventions per session and requires a minimum cooldown of 8 participant messages between each response.

Each intervention type is implemented as a separate LLM prompt with explicit content rules governing tone, symmetry, and prohibited moves (e.g., implying one participant is more justified, repeating sentence structures, using em-dashes). The prompts are structured to minimize stylistic variance across conversations, since the focus is on what is said, not how it is phrased.

Assistant

A second agent independently monitors the balance of participation across the conversation, tracking whether one participant is consistently dominating the exchange. When a meaningful imbalance is detected (e.g. one participant producing less than roughly half the words of the other across a rolling window) the assistant gently invites the quieter participant to share their perspective. It also handles conversation stalls: if no messages arrive within a set silence window, it re-engages both participants without referencing the gap. To prevent the two agents from sending messages back-to-back, the participation assistant checks whether the intervention facilitator has just spoken before deciding to respond.

Both agents use carefully tuned settings (minimum messages before eligibility, word count thresholds, cooldown periods, and response caps) to balance responsiveness with restraint.

🐳 Results

Study 1 revealed that AI facilitators face a penalty — but not where we expected. Rather than the classic “cold but capable” trade-off, AI was rated lower on both warmth and competence, suggesting people specifically doubt AI’s social judgment in emotionally charged settings. Among intervention types, emotional validation, bridging, and perspective-taking were consistently preferred, while normative interventions — telling people what others do or what the community expects — performed worst. Importantly, participants who had been exposed to the AI facilitator showed substantially reduced aversion to it compared to those who had not, suggesting that direct experience with AI facilitation shifts attitudes.

Study 2 is ongoing and will test whether these interventions actually improve conversations, not just how they appear from the outside.

💻 Code

Adaptation of DeliberateLab Platform for Conversation Experiment