library / lib2fa3c2b34bb93821
Build a Multi-Agent System (From Scratch)
Manning (MEAP) · 2025
In a sentence
A hands-on guide to building LLM agents and multi-agent systems from scratch by developing your own educational framework with support for tools, MCP, Skills, memory, human-in-the-loop, and the A2A protocol.
Multi-agent systems are among the most discussed and worked-on topics in AI today, but most engineers reach for frameworks like CrewAI or LangGraph without understanding what happens under the hood. This book takes you under the hood by having you build your own LLM agent framework—called llm-agents-from-scratch—from the ground up. Starting with the core insight that LLMs can only express intent but cannot act, you'll build the orchestration infrastructure that turns intentions into actions: interfaces for tools and LLMs, a processing loop, memory systems, and support for the core protocols MCP, Agent Skills, and A2A. Assuming basic familiarity with LLMs and Python, you'll code along chapter by chapter, gaining the deep understanding needed to work confidently with any existing framework or to build specialized solutions of your own. By the end, you'll be able to assemble focused agents into collaborating multi-agent systems and know exactly why and how each component works.
The four lenses
- Science
- Statistics
- Systems
- Strategy
The model
A framework model expressing how design levers (backbone LLM capabilities, equipped tools, protocol integrations, processing loop design, and multi-agent composition) drive psychological/behavioral states of the agent system (planning quality, tool-use behavior, reusable-workflow adherence) that in turn produce task performance outcomes such as accuracy, efficiency, and versatility.
Backbone LLM Capabilitiesdesign lever
The planning and tool-calling capabilities of the LLM that serves as the reasoning core of an agent, including its ability to formulate plans, make sensible next-action choices, and generate structured tool-call requests.
Equipped Toolsdesign lever
The set of tools (functions, code interpreters, web search, etc.) provided to the agent's backbone LLM via textual descriptions, expanding the range of actions the agent can take beyond text generation.
Protocol Integrationdesign lever
The degree to which the agent integrates external standards—MCP for third-party tools, Agent Skills for reusable documented workflows, and A2A for agent-to-agent communication—to extend capabilities and connectivity.
Processing Loop Designdesign lever
The orchestration logic that executes tasks through discrete sub-steps, repeatedly invoking planning and tool-calling and adapting plans based on prior results until completion or a stopping condition is met.
Agent Planning Qualitybehavioral pattern
The behavioral state in which the agent formulates correct initial plans and adaptively course-corrects mid-task, synthesizing results of prior sub-steps to determine the next action or task completion.
Tool-Use Behaviorbehavioral pattern
The behavioral pattern of the agent correctly selecting appropriate tools, generating valid tool-call requests, and using tool results rather than relying solely on parametric knowledge.
Multi-Agent Compositiondesign lever
The design choice to combine multiple focused, specialized LLM agents into a collaborating system whose individual processing loop outcomes are combined to complete an overarching decomposable task.
Task Decomposabilitycontextual condition
The contextual condition describing the extent to which a complex task can be broken into smaller sub-tasks where focused agents can outperform general-purpose ones.
Task Performanceoutcome metric
The outcome capturing how accurately, efficiently, and versatilely the agent or multi-agent system completes user-submitted tasks, including correct results, avoidance of cascading errors, and reasonable execution times.
Human-in-the-Loop Oversightdesign lever
The design pattern of inserting human review and approval of plans, sub-steps, or final results into the processing loop to mitigate cascading errors at the cost of increased execution time.
Agent Memorydesign lever
The capability enabling agents to save key information from task executions (trajectories, tool results) and load relevant portions back into context for future tasks, improving efficiency and continuity.
How they connect
- backbone llm capabilities → predicts agent planning quality
- backbone llm capabilities → predicts tool use behavior
- equipped tools → influences tool use behavior
- protocol integration → influences equipped tools
- processing loop design → influences agent planning quality
- agent planning quality → predicts task performance
- tool use behavior → predicts task performance
- protocol integration → influences agent planning quality
- multi agent composition → moderates task performance
- task decomposability → moderates task performance
- human in the loop → moderates task performance
- agent memory → influences task performance
The story
The reader An AI engineer or developer familiar with LLMs and Python who wants to deeply understand and confidently build LLM agents and multi-agent systems.
External problem
They can use existing agent frameworks but don't understand how agents, tools, protocols, and processing loops actually work under the hood.
Internal problem
They feel like a mere user of black-box abstractions and lack the confidence to debug, extend, or build specialized agent solutions.
Philosophical problem
Relying on frameworks you don't understand is a shallow way to work with a technology poised to reshape how tasks are done.
The plan
- Build a foundational LLM agent from scratch: define tool and LLM interfaces plus the LLMAgent class and processing loop.
- Enhance the agent with MCP tools, Agent Skills, memory, and human-in-the-loop capabilities.
- Assemble multiple agents into multi-agent systems using the A2A protocol.
- Reinforce learning by coding along in Jupyter notebooks and completing exercises and capstone projects.
Success
- The reader deeply understands how LLM agents and multi-agent systems work internally.
- They can use any existing agent framework confidently and efficiently.
- They can build specialized, custom agent and multi-agent solutions for their own needs.
At stake
- The reader remains a surface-level user of frameworks, unable to debug or extend them.
- They cannot adapt to the rapidly evolving agent ecosystem or build tailored solutions.
- They miss the potential of agent-to-agent collaboration to reshape workflows.
Questions this book answers
- What is an LLM agent and why are LLMs alone insufficient to act?
- How does a processing loop turn LLM plans and tool-call requests into completed tasks?
- How do tools, MCP, Skills, and A2A protocols extend and connect LLM agents?
- When do multi-agent systems outperform single agents?
- How do you build the internal components of an agent framework from scratch?
Glossary
- Backbone LLM Capabilities
- The planning and tool-calling competencies of the LLM at the core of an agent, determining how well it formulates plans and generates valid tool-call requests.
- Equipped Tools
- The collection of tools made available to the agent's backbone LLM to expand its actionable capabilities.
- Protocol Integration
- The extent to which the agent adopts external standards (MCP, Agent Skills, A2A) to access tools, reusable workflows, and other agents.
- Processing Loop Design
- The orchestration architecture that drives task execution through repeated planning and tool-calling sub-steps until completion.
- Agent Planning Quality
- The degree to which the agent forms correct plans and adaptively revises them based on prior sub-step results.
- Tool-Use Behavior
- The agent's pattern of correctly selecting tools, forming valid requests, and using returned results instead of relying on parametric knowledge.
- Multi-Agent Composition
- The design of combining multiple specialized agents into a collaborating system for an overarching task.
- Task Decomposability
- The extent to which a complex task can be split into subtasks amenable to focused agents.
Related in the library