AG Heinzelmann, 31.10.2025 Many LLMs Are More Utilitarian Than One

Moral judgment is integral to large language model (LLM) alignment and social reasoning. As multi-agent systems gain prominence, it becomes crucial to understand how LLMs function collectively during collaboration, compared to individual agents. In human moral judgment, group deliberation leads to a utilitarian boost: a tendency to endorse norm violations that maximize benefits for the greatest number of people despite harms. We study whether a similar dynamic emerges in multi-agent LLM systems. We tested six models on well-established sets of moral dilemmas across two conditions: (1) Solo, where models reasoned independently, and (2) Group, where they engaged in multi-turn discussions in pairs or triads. In personal moral dilemmas, where agents must decide to directly harm one individual to maximize the utility for others, all models found moral violations to be more acceptable when part of a group than individually, similar to human experiments. Some models endorsed actions that maximized overall well-being, even if they benefited strangers over familiar individuals. Others became more willing to violate moral norms in groups. However, while human groups show a similar action bias, the mechanism for their utilitarian boost differs from LLMs. Whereas the human shift comes from heightened sensitivity to decision outcomes, LLM groups show either reduced norm sensitivity or enhanced impartiality. This suggests that while the surface behavior of LLM collectives mimics human group reasoning, the underlying drivers differ. We discuss the implications for AI alignment, multi-agent design, and artificial moral reasoning.

Anita Keshmirian is Head of Data Science and Assistant Professor in Psychology at the Forward College in Berlin. She holds a bachelor’s degree in computer engineering, a master’s in cognitive science from Iran, and completed a second master’s in neuroscience through the Neurasmus program across Berlin, Bordeaux, and Boston. She received her PhD in 2021 with a dissertation titled “Moral Decisions in (and for) Groups: A Collective Approach”. At LMU she held a postdoctoral position and was also a research assistant at Reasoned AI Decisions Fraunhofer Institute for Cognitive Systems IKS. Dr. Keshmirian’s research bridges neuroscience, cognitive modeling, and group morality: she investigates how factors like diffusion of responsibility, collective decision-making, shared agency, and moral cognition shape judgments in social contexts.