Shared Cooperative Activity

Bratman’s (1992) philosophical analysis of joint cooperative action: SCA requires (1) mutual responsiveness to the other’s intentions and actions, (2) commitment to the joint activity itself, (3) commitment to mutual support should the other be unable to perform their part. Each agent’s intentions are recursively nested in the other’s. Distinguishes genuine cooperation from coincident individually-rational action; foundation of joint-intention models in MAS.

In this vault

Backlinks

Linked Pages

Why Do Multi-Agent LLM Systems Fail

Why Do Multi-Agent LLM Systems Fail?

Reference: Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica (2025). arXiv:2503.13657v2 (UC Berkeley). Source file: 2503.13657v2.pdf. URL

Summary

First empirically grounded taxonomy of failure modes in Multi-Agent LLM Systems (MAS). The authors analyse 200+ execution traces from seven popular MAS frameworks (MetaGPT, ChatDev, HyperAgent, AppWorld, AG2, Magentic-One, OpenManus), annotated by six human experts via grounded theory and reaching Cohen’s κ ≈ 0.88, and distil 14 fine-grained failure modes grouped into three categories: Specification Issues (42%), Inter-Agent Misalignment (37%), and Task Verification (21%).

They release MAST (Multi-Agent System failure Taxonomy), a validated LLM-as-judge pipeline for automated failure diagnosis, and two intervention case studies showing that architectural/prompt fixes inspired by MAST improve success rates modestly — demonstrating that MAS failures are system-design problems, not merely model-capability problems.

Key Ideas

Three failure categories: specification, inter-agent misalignment, verification
14 fine-grained failure modes including step repetition, information withholding, task derailment
Grounded-theory methodology with rigorous inter-annotator agreement (κ=0.88)
LLM-as-judge pipeline (MAST) achieves κ=0.77 vs humans for scalable evaluation
Insight: better specifications and verification beat bigger models

Connections

Conceptual Contribution

Claim: Multi-Agent LLM System (MAS) failures are predominantly system-design problems — specification, coordination, and verification — rather than base-model capability problems; and these failures have an empirically discoverable, reproducible taxonomy.
Mechanism: Grounded-theory analysis of 200+ execution traces across seven MAS frameworks (MetaGPT, ChatDev, HyperAgent, AppWorld, AG2, Magentic-One, OpenManus) with six human expert annotators; iterative refinement to Cohen’s κ≈0.88; yield the 14-mode MAST taxonomy grouped into Specification Issues (42%), Inter-Agent Misalignment (37%), Task Verification (21%); validate an LLM-as-judge annotator (κ≈0.77); intervention case studies showing prompt/architecture fixes provide only modest gains, motivating deeper redesign.
Concepts introduced/used: MAST Taxonomy, Grounded Theory, Inter-Agent Misalignment, LLM-as-judge, Specification Issues, Task Verification, Multi-Agent Systems, LLM Agents, Cohen’s Kappa, Standard Operating Procedures (SOPs)
Stance: empirical / evaluative
Relates to: Supplies empirical grounding for the design-quality concerns in Agents Framework - Zhou et al (SOPs attempt to mitigate FC1 specification issues) and Multi-Agent Collaboration in AI - Wasif Tunkel. Inter-agent misalignment mirrors the formal pathologies in Are Multiagent Systems Resilient to Communication Failures. Motivates richer communication protocols like A Scalable Communication Protocol for Networks of LLMs and commitment-style ACLs (ACL Rethinking Principles).

Tags

We-intentions

Reference: Tuomela, R. & Miller, K. (1988). We-intentions. Philosophical Studies, 53(3), pp. 367–389. (Extended in Tuomela, R. (2007). The Philosophy of Sociality: The Shared Point of View. Oxford University Press.) DOI/Springer · Open access PDF (Academia.edu)

Summary

Tuomela and Miller introduce the analysis of we-intentions — intentions of the form “we will do X” — that became the central rival to Bratman’s reductive analysis of joint action. Their headline thesis: a we-intention is not reducible to any combination of individual I-intentions of the form “I will do my part.” A genuine we-intention has the structure: I intend that we (you and I) do X together; we mutually believe that we have this intention; this mutual belief is in part the reason we have the intention. The key features that resist reduction are (i) the content of the intention names the group as the agent — “we will” — and (ii) the intention is partly constituted by the mutual belief that the participants share it, in a way that no individual-intention combination can match. Tuomela’s later work elaborates this into a comprehensive social ontology distinguishing I-mode action (acting as a private individual, even when in a group) from we-mode action (acting as a member of a group with group-level reasons). The we-intention analysis is the main philosophical alternative to the Bratman reductive position; in MAS the dispute manifests as the choice between (a) implementing joint action via richly nested individual intentions (Cohen & Levesque joint-intention models, SharedPlans) and (b) treating the group as a primitive locus of intention (DAI multi-agent organisations, electronic-institution frameworks). The dispute has practical consequences for which kinds of failure modes a multi-agent system is susceptible to: reductive systems can fail through individual rationality undermining group action, while we-mode systems can fail through group-level reasoning detached from individual responsibility.

Key Ideas

We-intention structure: I intend that we do X together; we mutually believe that we have this intention; the mutual belief is part of the reason we have the intention. The reflexive grounding in mutual belief is the key feature.
Irreducibility thesis: no combination of individual I-intentions of the form “I will do my part of X” captures the content of “we will do X.” The we-intention’s content names the group as the agent.
I-mode vs we-mode (Tuomela 2007): even within a group, an agent may act in I-mode (treating others as instrumental) or we-mode (treating the group’s reasons as genuinely group-level). The distinction is normative as well as descriptive.
Mutual belief as constitutive: we-intentions are partly constituted by the mutual belief that they are shared — change the mutual belief and the we-intention itself changes, in a way that has no analogue in individual intentions.
Group as agent: the analysis takes seriously the locution of group agency. We won the game; we signed the treaty. The locutions are not (Tuomela argues) elliptical for individual contributions; they ascribe action to the group as such.
Connection to social facts: we-intentions are the building blocks of social institutions — money, marriage, governments — which exist only because their participants mutually intend that they exist (cf. Searle 1995, The Construction of Social Reality).

Connections

Conceptual Contribution

Claim: A we-intention — an intention of the form “we will do X” — is irreducible to any combination of individual intentions of the form “I will do my part.” The content names the group as the agent, and the intention is partly constituted by the mutual belief that the participants share it. Joint action is genuinely group-level; the reduction programme (Bratman, Cohen-Levesque, Grosz-Kraus) misses what is distinctive about it.
Mechanism: Conceptual analysis of the locution “we intend”; specification of the reflexive structure (mutual belief as part of the reason); arguments against various reductive proposals; later development (Tuomela 2007) into the I-mode / we-mode distinction.
Concepts introduced/used: We-Intention, We-mode vs I-mode, Group Agency, Mutual Belief as constitutive, Social Institutional Facts.
Stance: foundational philosophical paper; the canonical alternative to Bratman’s reductive programme.
Relates to: Direct philosophical opposition to Bratman 1992 (and the computational descendants Cohen & Levesque 1991, Grosz & Kraus 1996) — Bratman holds that joint action reduces to recursively nested individual intentions, Tuomela holds that a primitive group intention is needed. The dispute is partly definitional but has practical consequences: implementations adopting the reductive position end up with rich individual-intention machinery (MAS frameworks following BDI / Cohen-Levesque), while implementations adopting we-mode primitives tend toward organisation-level constructs (electronic institutions, Agent Communication And Institutional Reality). Norms and Obligations and the institutional-fact tradition (Searle 1995) are conceptual cousins. In LLM-agent settings the dispute manifests as the choice between (a) individual LLM agents with theory-of-mind reasoning about each other’s intentions and (b) higher-level group-policy reasoning that treats the multi-agent system as a single decision-making locus — both approaches appear in the literature, often without explicit acknowledgement of the underlying philosophical commitment.

Tags

#joint-action #we-intentions #tuomela #philosophy-of-action #foundations #social-ontology

SharedPlans

Grosz & Kraus’s (1996) computational framework for collaborative joint action: a SharedPlan is a group-level structure carrying mutual beliefs about who is doing what, why, and how, with two intention modalities (Int.To for own actions, Int.Th for actions of others or the group). Most real collaboration involves partial SharedPlans, refined incrementally through meta-intentions to complete the plan, help others, and communicate. The canonical computational realisation of Bratman’s Shared Cooperative Activity.

In this vault

Cohen and Levesque

Philip R. Cohen and Hector J. Levesque, whose joint and individual work in the 1980s–1990s formalised intention, joint intention, and the rational-balance constraints on BDI-style agents. Their 1990 paper Intention Is Choice with Commitment (AIJ 42) is the canonical formal model of intention as a persistent goal — a commitment to maintain a goal until it is believed achieved, believed unachievable, or its motivating reason dissolves. Cohen & Levesque (1991) Teamwork extends the analysis to joint action and joint intentions in multi-agent settings.

In this vault

We-Intention

Tuomela & Miller’s (1988) concept: an intention to do something as part of a group — operationally I intend that we φ rather than I intend that I do my part of φ. Tuomela argued that we-intentions are irreducible to the individual intentions of the group’s members; the alternative reductive position is held by Bratman (1992, Shared Cooperative Activity) and Cohen & Levesque (1991), who show that a sufficiently rich structure of recursively-nested individual intentions can capture joint action without invoking a primitive group subject. The dispute is partly empirical, partly definitional, and partly programmatic — the reductive position better fits computational MAS implementations; the irreducibility position better fits descriptive social ontology.

In this vault

Mutual Responsiveness

The first of Bratman’s three conditions for Shared Cooperative Activity: each participant in joint action is responsive to the intentions and actions of the other(s) — adjusting their own behaviour as the other’s behaviour unfolds rather than executing a fixed sub-plan in isolation. Operationally distinguishes cooperative joint action from parallel-but-non-interactive execution.

In this vault

Meshing Subplans

Bratman’s (1992) construction in Shared Cooperative Activity: in genuine cooperative joint action, each participant forms a subplan for their own contribution, and the subplans must mesh — be positively coordinated rather than merely consistent. Stronger than non-interference: the subplans must positively support each other and not inadvertently undermine each other’s success conditions. Computational realisation in SharedPlans (Grosz & Kraus 1996).

In this vault

Shared Cooperative Activity

In this vault

Multi-Agent Collaboration in AI - Wasif Tunkel

Multi-Agent Collaboration in AI: Enhancing Software Development with Autonomous LLMs

Reference: Mubeen Wasif and David Tunkel (2025). ResearchGate preprint. Source file: 16.pdf

Summary

A short survey-style paper arguing that multi-agent LLM systems can improve software-development workflows by distributing tasks (requirements, code generation, testing, documentation) across specialised autonomous agents that communicate via structured dialogue. The authors report qualitative experimental findings that division of labour and iterative refinement among agents produce higher-quality outputs than single-agent baselines.

The paper also surveys open challenges: coordination overhead, response consistency, bias propagation, and governance/security concerns. It advocates human-in-the-loop validation and explainability (XAI) as mitigations, and points to future integration with IDEs, CI/CD, and RAG.

Key Ideas

Specialised agent roles (coder, tester, documenter) mirror human dev teams
Structured inter-agent dialogue enables iterative code refinement
Hybrid human-AI teams recommended for reliability
Coordination cost, bias propagation, accountability are unsolved
RAG and adaptive prompting as future contextual-awareness tools

Connections

Conceptual Contribution

Claim: Multi-agent LLM systems, with role-specialised agents (coder, tester, documenter) communicating through structured dialogue, outperform single-agent LLMs on software-engineering tasks by mirroring human team division-of-labour.
Mechanism: Survey-plus-experiment discussion: assign distinct agents to requirement analysis, code generation, debugging, documentation; use iterative feedback loops and structured prompts; report efficiency/accuracy gains while flagging coordination overhead, bias propagation, and security concerns; advocate human-in-the-loop governance.
Concepts introduced/used: LLM Agents, Role-specialised Agents, Human-in-the-loop, Agent Coordination Overhead, Explainable AI, Multi-Agent Systems, Retrieval-Augmented Generation, Division of Labour
Stance: engineering / survey
Relates to: A light-weight practitioner view of the same problem space that Why Do Multi-Agent LLM Systems Fail tackles rigorously, and that Agents Framework - Zhou et al operationalises as an open-source library. Shares the communication-protocol concern with A Scalable Communication Protocol for Networks of LLMs.

Tags

Common Knowledge (Convention)

The infinite hierarchy of mutual expectation introduced by Lewis (1969) and rediscovered by Aumann (1976) in game theory and Halpern–Moses (1990) in distributed systems: A knows p, A knows that B knows p, A knows that B knows that A knows p, … all the way up. A convention is sustained by common knowledge of the regularity. The construction has the same mathematical content in three independent fields — Lewis-style social philosophy, game theory, and distributed-systems epistemology — and is unattainable in fully asynchronous distributed systems (Halpern & Moses), with deep consequences for what coordination protocols can achieve.

In this vault

Modeling Rational Agents within a BDI-Architecture

Reference: Rao, A. S., & Georgeff, M. P. (1991). In J. Allen, R. Fikes, & E. Sandewall (Eds.), Proceedings of the Second International Conference on Principles of Knowledge Representation and Reasoning (KR’91), 473–484. Morgan Kaufmann. Also AAII Technical Note 14. Source file: rao_georgeff_bdi_architecture.pdf. URL

Summary

Rao and Georgeff give the canonical formalization of the Belief–Desire–Intention architecture as a modal/temporal logic over a branching-time possible-worlds model (a first-order extension of CTL*). Beliefs, goals (chosen, consistent, believed-achievable desires), and intentions are each modelled as accessibility relations over time trees, with the crucial innovation that belief-, goal-, and intention-accessible worlds are themselves time trees rather than static worlds — so an agent’s mental state simultaneously ranges over uncertainty about the world (branches across belief-worlds) and choice among options (branches within each world).

The paper realises the central tenets of Bratman’s philosophical theory (intentions as partial plans, irreducible to belief+desire) in a tractable semantic framework and introduces the distinction between choice (what the agent picks) and possibility (what the environment affords). A key technical contribution is the family of commitment strategies — blind, single-minded, and open-minded — which parametrically define how intentions persist under changing beliefs, yielding a taxonomy of rational agent types from fanatically committed to fully reconsidering. The formalism relates intentions to actions via sub-world inclusion (goals ⊆ beliefs; intentions ⊆ goals), avoiding classical side-effect problems of earlier intention logics.

Key Ideas

First-order CTL*-based modal logic with Bel, Goal, Intend accessibility.
Belief-, goal-, intention-accessible worlds are themselves branching time trees.
Optional vs. inevitable path modalities; distinguishes choice from possibility.
Strong realism constraint: goal-worlds are sub-worlds of belief-worlds.
Commitment strategies: blind, single-minded, open-minded.
Intentions treated as first-class, non-reducible to beliefs and desires.
Comparison with Cohen–Levesque’s reductive intention formalism.

Connections

Conceptual Contribution

Claim: Rational agent behaviour is best modelled by treating beliefs, goals and intentions as equally fundamental modal attitudes over branching time, with agent types differentiated by commitment strategy rather than by reduction of intention to other attitudes.
Mechanism: (i) Adopt CTL*-style branching time where each situation is a time point in a time tree, with path modalities optional and inevitable; (ii) associate each situation with sets of belief-, goal-, and intention-accessible worlds, each itself a time tree, imposing sub-world containment (strong realism); (iii) define rationality postulates relating the three attitudes; (iv) axiomatise commitment policies (blind = drop intention only on achievement, single-minded = also drop when no longer believed achievable, open-minded = also drop when no longer desired), yielding distinct agent classes.
Concepts introduced/used: BDI Logic, Commitment Strategies, Strong Realism, Branching Time, Intention, Practical Reasoning
Stance: formal framework
Relates to: Formalises the philosophical picture of Two Faces of Intention and operationalises the stance of Intentional Stance; provides the semantic bedrock for BDI Architecture, Agent-Oriented Programming and the theory side of Intelligent Agents Theory and Practice; differs from and complements Intention Is Choice with Commitment (Cohen & Levesque) by treating intention as primitive rather than derived.

Tags

BDI

Belief-Desire-Intention

Dominant agent-architecture paradigm: agents maintain beliefs, desires, and intentions and deliberate over them.

In this vault

Intention Is Choice with Commitment

Reference: Cohen, P.R. & Levesque, H.J. (1990). Artificial Intelligence 42(3): 213–261. Source file: cohen-levesque-intention-is-choice-with-commitment.pdf. URL

Summary

Cohen and Levesque formalise the “rational balance” that holds between an autonomous agent’s beliefs, goals, actions, and intentions. They define intention as a composite mental attitude: a persistent goal that the agent is committed to pursuing until it is achieved, known to be impossible, or no longer relevant. This yields a modal logic whose axioms capture Bratman’s three functional roles for intention — posing problems for means-end reasoning, acting as a screen of admissibility for future intentions, and prompting the agent to track (and re-try) the success of its actions.

The paper opens with the now-classic “Willie the robot” vignette (uncommitted, overcommitted, then homicidally committed), framing intention as what an agent must reason about to be both reliable and responsive. By relativising intentions to beliefs about another agent’s intentions the authors also sketch a theory of interpersonal commitment that anticipates speech-act and multi-agent communication semantics.

Key Ideas

Persistent goal (P-GOAL): a goal an agent will not drop until it is believed achieved, believed impossible, or its relevance condition fails.
Intention defined as a P-GOAL to have done an action while believing one is about to do it — closing Bratman’s gap between intending and acting intentionally.
Avoids the “side-effect problem”: agents don’t intend foreseen side-effects of their actions.
Mutual/relativised intentions as a foundation for interpersonal commitments and speech acts.
Desiderata list (beliefs, feasibility, non-futility, tracking) becomes a standard benchmark for intention logics.

Connections

Agent-Oriented Programming
Belief-Desire-Intention
BDI Logic
BDI Architecture
Joint Intentions
Commitment
Mental State
Rational Action Semantics
Speech Act Theory
Intentional Stance
Flexible Protocol Specification and Execution
A Commitment-Based Approach to Agent Communication
Communicative Actions for Artificial Agents - Cohen Levesque — the 1995 ICMAS sequel that turns the 1990 framework into an explicit speech-act / ACL semantics; direct ancestor of FIPA-ACL.
FIPA-ACL Specifications
Mentalistic Semantics
ACL Rethinking Principles — Singh’s critique of the BDI-based ACL semantics built on this foundation.

Conceptual Contribution

Claim: Intention is not a primitive mental state but a choice with commitment — a goal plus a meta-policy for when to drop it.
Mechanism: Modal logic over beliefs and goals plus a temporal operator for persistence; intention defined compositionally so its functional roles (planning, filtering, tracking) are theorems rather than axioms.
Concepts introduced/used: Persistent Goal, Commitment, Rational Balance, Mental State, Nested Beliefs, Joint Intentions.
Stance: foundational/formal.
Relates to: Shoham’s Agent-Oriented Programming cites this paper as the prime candidate for grounding AOP’s mental-state semantics; Shoham deliberately chooses a simpler obligation/commitment primitive for AGENT-0 but frames C&L as the rigorous theory AOP languages should eventually implement. The paper also underwrites FIPA-style Mentalistic Semantics and Rational Action Semantics for ACLs.

Tags

Two Faces of Intention

Reference: Bratman, M. E. (1984). The Philosophical Review, 93(3), 375–405. Source file: bratman_two_faces_of_intention.pdf. URL

Summary

Bratman’s canonical statement of the planning theory of intention. He argues that commonsense psychology uses “intention” to characterise both actions (done intentionally) and mental states (intending to act), and that neither the desire-belief reductionist model (Davidson) nor the Simple View (to A intentionally I must intend to A) correctly captures the two faces. Instead, intentions are distinctive, irreducible states of mind that function as elements of partial, future-directed plans which coordinate conduct over time and with other agents.

Because intentions serve as inputs to further means-end reasoning and as filters on admissible options, they must satisfy internal and means-end consistency constraints; but they need not entail all their foreseen consequences as further intentions. Bratman develops the distinction between what one intends and the motivational potential of an intention (foreseen but unintended side-effects that one is nonetheless prepared to bring about), which dissolves classic puzzles about double effect without collapsing into the Simple View. The methodological priority of future-directed intention — intentions as ingredients of plans — became the philosophical foundation for the BDI architecture in AI.

Key Ideas

Two faces: intentional action vs. intending (present- vs. future-directed).
Rejection of the Desire-Belief reductive model.
Rejection of the Simple View (intentional A-ing requires intending to A).
Intentions as partial plans coordinating activity over time.
Plan-consistency, means-end coherence, non-reconsideration norms.
Distinction: what one intends vs. motivational potential (side effects).
Future-directed intention has methodological priority.

Connections

Conceptual Contribution

Claim: Intentions cannot be reduced to configurations of belief and desire; they are distinctive planning states whose function is to stabilise and coordinate future action, and the two uses of “intention” (for acts and for mental states) are united through the common role of such planning states rather than through an identity of content.
Mechanism: Bratman argues against the Desire-Belief Model by showing planning creatures settle in advance on options that beliefs+desires underdetermine; against the Simple View by exhibiting cases where A is done intentionally without intending-to-A (e.g., foreseen side effects one is willing to accept). He replaces both with the Single Phenomenon View refined by a plan-theoretic account: intentions are inputs to further reasoning, demand consistency and means-end coherence, resist easy reconsideration, and possess motivational potential beyond strict intended content.
Concepts introduced/used: Intention, Planning Theory of Intention, Practical Reasoning, Motivational Potential, Future-Directed Intention, Simple View, Single Phenomenon View
Stance: philosophical theory
Relates to: Philosophical foundation cited and formalised by Intention Is Choice with Commitment and Modeling Rational Agents within a BDI-Architecture; supplies the mentalistic vocabulary surveyed in Intelligent Agents Theory and Practice and operationalised in Agent-Oriented Programming; complements the external ascription view of Intentional Stance.

Tags

Joint Intentions

Cohen & Levesque’s theory of teamwork: joint persistent goals with mutual belief commitments.