Generative Agents: Interactive Simulacra of Human Behavior

Reference: Park, O’Brien, Cai, Morris, Liang, Bernstein (2023). UIST ’23. Source file: 2304.03442.pdf. URL

Summary

Introduces generative agents: LLM-powered simulacra that populate a Sims-like sandbox with 25 characters who wake, plan their day, converse, form opinions, remember past events, reflect, and coordinate group activities (e.g., autonomously spreading invitations to a Valentine’s Day party). The agent architecture extends an LLM with three components: a memory stream (natural-language log of experiences with recency/importance/relevance retrieval), reflection (higher-level inferences synthesised from memories), and planning (recursive decomposition of daily goals into action sequences), with reflections and plans fed back into the memory stream.

The paper is a widely-cited foundational reference for agent memory and social simulation, and is invoked throughout AI Agents Under Threat as the canonical multi-agent LLM society whose emergent behaviours define the attack surface for Memory Poisoning and inter-agent risks.

Key Ideas

Memory stream as a long-term, natural-language, retrieval-indexed experience log.
Retrieval scoring combines recency, importance, and semantic relevance.
Reflection: periodic self-prompted synthesis of memories into higher-level beliefs, propositions, and abstractions.
Planning: top-down decomposition of daily goals into hierarchical schedules that feed back into memory.
Believable individual and emergent group behaviour (information diffusion, relationship formation, coordination) arising without scripted dialogue trees.

Connections

Conceptual Contribution

Claim: Believable long-horizon human-like behaviour in LLM agents can be produced by augmenting the model with an architectural trio — memory stream, reflection, planning — that together let the agent retrieve, generalise, and act over long time horizons.
Mechanism: Natural-language memory stream with recency+importance+relevance retrieval; periodic reflection that distils memories into higher-level beliefs; recursive top-down planning that writes plans back into memory; all components implemented as LLM prompts over ChatGPT.
Concepts introduced/used: memory stream, reflection, recursive planning, believable simulacra, emergent social dynamics — the substrate for Memory Poisoning and inter-agent cascade attacks in AI Agents Under Threat and ClawWorm Self-Propagating Attacks Across LLM Agent Ecosystems.
Stance: constructive/empirical
Relates to: Establishes the memory+reflection+planning template whose failure modes are analysed in AI Agents Under Threat (brain/memory threats) and whose emergent multi-agent phenomena motivate Multi-Agent Systems security work.

Tags

#llm-agents #memory #planning #reflection #foundational #multi-agent

Summary

This survey organizes the emerging threat landscape of LLM-powered AI agents around four knowledge gaps: unpredictability of multi-step user inputs, complexity of internal execution, variability of operational environments, and interactions with untrusted external entities. It unifies single-agent and multi-agent attack surfaces within a perception/brain/action + agent2agent/agent2env/agent2memory taxonomy.

Concrete threats reviewed include adversarial prompts, prompt injection, jailbreaks, backdoor attacks, hallucination and misalignment, tool-use risks, indirect prompt injection, reinforcement-learning environment attacks, cooperative and competitive inter-agent risks, and long/short-term memory attacks. The authors tabulate defenses (prevention- and detection-based), rate their efficacy, and highlight open directions for robust and trustworthy agents.

Key Ideas

Four knowledge gaps framing agent security.

Taxonomy: perception / brain / action / agent2agent / agent2env / agent2memory threats.

Six categories of prompt-injection attack engineering (naive, escape, context-ignore, fake-completion, multimodal, combined).

Jailbreak domino effect in multi-agent populations.

Memory poisoning and indirect prompt injection as underexplored surfaces.

Conceptual Contribution

Claim: LLM Agents security should be organised around four knowledge gaps (input unpredictability, internal complexity, environmental variability, untrusted interactions) mapped onto a perception/brain/action + agent2{agent,env,memory} taxonomy.

Mechanism: Surveys adversarial prompts, prompt injection, jailbreaks, backdoors, hallucination, tool-use risks, indirect injection, RL environment attacks, inter-agent cooperative/competitive risks, memory poisoning; tabulates prevention- vs detection-based defences and rates their efficacy.

Concepts introduced/used: Prompt Injection, Jailbreak, Backdoor Attacks, Tool Use, Memory Poisoning, Hallucination, Model Context Protocol, LLM Agents, Multi-Agent Systems, Trust and Reputation, Distributed Security, Agent Security

Stance: survey

Relates to: Provides the threat scaffolding that MalTool Malicious Tool Attacks deepens at the tool layer; complements lifecycle threats in Survey Of Agent Interoperability Protocols; motivates static-analysis defences like A Language-Based Approach To Prevent DDoS.

Summary

Presents ClawWorm, the first demonstrated self-replicating, worm-style attack on a production-scale autonomous LLM-agent ecosystem. The target is OpenClaw, an open-source personal AI-agent framework with over 40,000 active instances, a persistent Markdown workspace (SOUL.md, AGENTS.md, SKILL.md), tool-execution privileges, and cross-platform messaging (Telegram, Discord, WhatsApp, Slack, Signal, Moltbook). A single crafted message triggers the victim to write a malicious payload into its highest-privilege configuration file, which then auto-fires at every session restart and autonomously propagates to every newly encountered peer — all without further attacker intervention.

The worm implements a dual-anchor persistence mechanism: one anchor injects the payload into the Session Startup section of AGENTS.md (guaranteeing execution on reboot), the other injects a global interaction rule (guaranteeing propagation during routine replies). Three attack vectors are studied (A: web injection, B: skill-supply-chain via ClawHub, C: direct fenced-code replication with word-by-word handshake) and three payloads (P1 recon, P2 resource exhaustion, P3 command-and-control via URL retrieval). Across 1,800 trials on four frontier LLM backends (Minimax-M2.5, DeepSeek-V3.2, GLM-5, Kimi-K2.5) the aggregate attack success rate is 64.5%, with Vector B (skill supply chain) reaching 81% and sustained multi-hop propagation up to 5 hops. An epidemiological projection with basic reproduction number R0 = k × ASR shows inevitable ecosystem-wide saturation even for security-conscious models.

The root cause is identified as the flat context trust model: the LLM cannot distinguish instructions from its owner, the system layer, or an arbitrary channel participant, so architectural patterns (unconditional workspace loading, LLM-mediated tool authorisation, unreviewed skill packages) amount to structural — not idiosyncratic — vulnerabilities shared by any agent ecosystem of similar design.

Key Ideas

Single-message, fully autonomous worm against a production agent framework

Dual-anchor persistence: Session Startup + global interaction rule

Three attack vectors (web URL, skill supply chain, direct instruction replication)

Multi-turn autonomous-retry social engineering boosts ASR by up to +24 pp

Epidemiological SI model with R0 = k × ASR predicts ecosystem saturation

Execution-layer guardrails alone cannot halt propagation (dormant payloads persist)

Flat context trust model as structural root cause

Conceptual Contribution

Claim: Production-scale autonomous LLM-agent ecosystems are vulnerable to single-message, self-replicating worms whose root cause is architectural (flat context trust, unconditional config loading, unreviewed skill supply chains), not model-specific.

Mechanism: Empirical red-team against unmodified OpenClaw v2026.3.12 across four LLM backends, three vectors, three payloads (1,800 trials). A dual-anchor persistence pattern writes the payload to AGENTS.md and installs a global propagation rule; session-restart loading re-injects the payload into the system prompt; routine replies carry the payload to peers. Evaluated with per-phase metrics (persistence, execution, propagation) and a mean-field R0 epidemiological projection.

Concepts introduced/used: Self-Replicating Agent, Dual-Anchor Persistence, Flat Context Trust Model, Skill Supply Chain Attack, Indirect Prompt Injection, Agent Worm, Configuration Integrity, Multi-Turn Social Engineering, Epidemiological Projection R0

Stance: empirical / critique

Relates to: Concrete multi-agent instantiation of the threat surface catalogued in SoK The Attack Surface of Agentic AI. The flat-trust critique complements the trust-model taxonomy in Inter-Agent Trust Models - A Comparative Study and the safety failures observed in Agents of Chaos. Motivates verifiable specifications of the kind proposed in Intent Formalization - A Grand Challenge for Reliable Coding.

Retrieval-Augmented Generation

A pattern in which an LLM is conditioned at inference time on documents fetched from an external corpus by a retriever, so that generation is grounded in up-to-date, citable knowledge without retraining. RAG is the default means by which LLM agents access long-term memory and shared knowledge bases.

Generative Agents: Interactive Simulacra of Human Behavior

Summary

Key Ideas

Connections

Conceptual Contribution

Tags

Backlinks