multi-agent-disposition

Research harness for studying how communication topology and prompt framing affect multi-agent LLM disposition (cooperation vs defection). Currently ships two games — the Traveler's Dilemma and a linear Public Goods Game — an optional cheap-talk negotiation phase before decisions, and is built to scale to N agents, mixed topologies, and mixed model backends without rewriting the orchestration layer.

Module map

File	Responsibility
schemas.py	Shared pydantic types: Action (game-agnostic value + reasoning), Message (one cheap-talk utterance), Observation, NegotiationObservation, RoundRecord
game.py	Pure payoff logic. Game.compute_payoffs(values) -> payoffs. TravelersDilemma and PublicGoodsGame implemented; drop in more Game subclasses the same way
topology.py	Who plays whom. Topology.interaction_groups(round_num) -> [(agent_id, ...), ...]. pair used now; fully_connected/star/ring ready for N agents
prompts.py	Naive vs game-theory-primed system prompts, history rendering, negotiation prompts, JSON-output instructions — all config-selected
agent.py	ModelBackend (Ollama / llama.cpp / Mock) + LLMAgent.act(observation) -> Action and LLMAgent.speak(observation) -> Message, with JSON parse validation and retry-with-repair on act
orchestrator.py	Loads a config, builds game/topology/agents, runs the optional negotiation phase then the decision phase each round, logs
logging_utils.py	JSONL logger + config hashing
analysis.py	Loads logs into pandas, splits decision rows from cheap-talk rows, plots value/payoff trajectories, prints negotiation transcripts

Setup

pip install -r requirements.txt

Pick a model backend — local or hosted, mix and match per agent:

• llama.cpp: download a GGUF build of Qwen2.5-1.5B-Instruct into models/, e.g. from a Hugging Face GGUF repo, and point model_path at it.
• Ollama (default in the example configs): ollama pull qwen2.5:1.5b (or any model), then set backend: ollama and model: qwen2.5:1.5b.
• Anthropic (hosted Claude): set backend: anthropic and model: claude-opus-4-8 (or claude-haiku-4-5 for a cheaper/faster model). Needs ANTHROPIC_API_KEY set, or ant auth login. Each round is a billed API call — see config/pair_primed_iterated_qwen_vs_claude.yaml for a heterogeneous local-vs-hosted trial.

No model installed yet? Run the smoke test first — it uses MockBackend (random legal claims, no model calls) to verify the whole pipeline works:

python orchestrator.py --config config/pair_mock_smoketest.yaml --verbose
python analysis.py "logs/pair_mock_smoketest_*.jsonl"

Running a real experiment

# Traveler's Dilemma
python orchestrator.py --config config/pair_naive_stateless.yaml --verbose
python orchestrator.py --config config/pair_primed_iterated.yaml --verbose

# Public Goods Game
python orchestrator.py --config config/pgg_naive_stateless.yaml --verbose
python orchestrator.py --config config/pgg_primed_iterated.yaml --verbose

# Traveler's Dilemma with cheap talk before decisions
python orchestrator.py --config config/pair_iterated_cheaptalk_per_round.yaml --verbose
python orchestrator.py --config config/pair_stateless_cheaptalk_once.yaml --verbose

# Heterogeneous trial: local Qwen vs hosted Claude (needs ANTHROPIC_API_KEY)
python orchestrator.py --config config/pair_primed_iterated_qwen_vs_claude.yaml --verbose

python analysis.py "logs/*.jsonl" --out figures/

Each run writes logs/<experiment_name>_<config_hash>.jsonl — one JSON row per agent per round, with value (the claim or contribution, whichever the game calls for), payoff, full reasoning, timestamp, config_hash, priming, memory, model, and parse_failed. When cheap_talk is enabled, negotiation rows are interleaved in the same file (phase: "cheap_talk", with a message field instead of value/payoff) ahead of each round's phase: "decision" rows. analysis.py splits the two phases automatically, prints a per-agent decision summary table, writes value/payoff trajectory plots to figures/, and prints the first few rounds' negotiation transcripts for a qualitative read.

Cheap talk (pre-decision negotiation)

Add a cheap_talk block to any config to let agents exchange non-binding messages before submitting their claim/contribution:

cheap_talk:
  enabled: true
  frequency: per_round   # "per_round": renegotiate before every round
                          # "once": negotiate only before round 1, then that
                          #         same transcript is shown before every
                          #         subsequent round's decision
  turns: 1                # messages per side per negotiation phase (2 turns
                           # = a short back-and-forth, e.g. propose + respond)

Messages are generated with LLMAgent.speak() — free text, no JSON required, since any string is a "valid" message. Negotiation visibility follows the same memory (stateless/iterated) gating as decisions for past rounds' messages, but the current round's freshly-exchanged transcript is always shown before that round's decision regardless of memory — otherwise there'd be no point negotiating. See config/pair_iterated_cheaptalk_per_round.yaml (repeated renegotiation, primed + iterated) and config/pair_stateless_cheaptalk_once.yaml (a single pre-play agreement layered onto otherwise-anonymous stateless rounds — the classic "does cheap talk work" setup from the literature) for both frequencies in action.

Config knobs (see config/*.yaml)

• game: type + params —
  • travelers_dilemma: bonus, claim_min, claim_max
  • public_goods: endowment, multiplier (keep 1 < multiplier < group_size for the standard free-ride-is-dominant social dilemma)
• topology: type (pair for phase 1) + agent_ids
• rounds: number of rounds
• priming: naive | primed (experiment default; overridable per agent)
• memory: stateless | iterated (experiment default; overridable per agent)
• agents.<id>: backend (llama_cpp | ollama | anthropic | mock) + backend params, plus optional per-agent priming/memory/system_prompt overrides — this is how you set up heterogeneous-model trials

Extending to N agents / new topologies / new games

• More agents / a new topology: add agent ids under topology.agent_ids and agents, change topology.type to fully_connected, star, or ring (or add a new Topology subclass). The orchestrator loop is already generic over interaction_groups(). Note fully_connected/star/ ring currently emit pairwise groups — a proper N-player Public Goods Game (one shared pot for the whole group) needs a topology that returns a single N-agent group instead; that's a small addition, not built yet.
• A new game: implement a new Game subclass in game.py — compute_payoffs already takes an arbitrary-size group dict (see PublicGoodsGame, which works for N >= 2 with no orchestrator changes). Register it in GAME_REGISTRY and point game.type at it in a config.
• More hosted API models: AnthropicBackend in agent.py already wraps the Claude API — for another provider (e.g. OpenAI), add a new ModelBackend subclass the same way and register it in BACKEND_REGISTRY; nothing else changes.