Building AI Agent Architecture From Scratch

Most AI agent frameworks start simple: one prompt, one model call, one output. Then requirements grow. You add tools, then tool routing, then multi-step reasoning, then memory. Before you know it, you have a 2,300-line orchestrator file that nobody wants to touch.

That's exactly where I found myself six months ago. Here's how I rebuilt it.

The Problem

The original system was a single orchestrator.ts file that handled everything:

• Agent selection based on user input
• Prompt construction with context injection
• Tool routing and execution
• Memory management
• Error recovery

Adding a new agent type meant modifying 5+ files, and the test coverage sat at 12% because everything was tightly coupled.

The Architecture

The new system uses three core concepts:

1. Agent Definitions as Data

Each agent is defined in a YAML file with its prompt, available tools, and validation rules. No code changes needed to add a new agent — just drop a YAML file.

id: code-reviewer
name: Code Reviewer
model: sonnet
triggers:
  - review
  - lint
  - code review
tools:
  - read-file
  - grep-search
  - glob-find

2. Factory Pattern for Instantiation

A single factory reads agent definitions and produces configured instances. The factory handles dependency injection, tool binding, and prompt compilation.

const agent = AgentFactory.create('code-reviewer', {
  context: conversationHistory,
  tools: toolRegistry.getTools(['read-file', 'grep-search']),
})

3. Discriminated Unions for Type Safety

Each agent action is a discriminated union, so TypeScript catches routing errors at compile time rather than runtime.

type AgentAction =
  | { type: 'tool_call'; tool: string; params: Record<string, unknown> }
  | { type: 'response'; content: string }
  | { type: 'delegation'; targetAgent: string; context: string }

Results

The orchestrator dropped from 2,300 lines to 50. Adding a new agent takes 15 minutes instead of 4 hours. Test coverage jumped to 94%.

More importantly, the team can now iterate on individual agents without fear of breaking the system. Each agent is isolated, testable, and deployable independently.

Key Takeaways

1. Data-driven configuration beats code-driven logic. YAML definitions let non-engineers modify agent behavior.
2. TypeScript discriminated unions are perfect for agent routing. The compiler catches mistakes before they reach production.
3. Small, composable modules always win. Even when the initial refactor takes longer, the velocity gain compounds.

The full architecture is documented in our internal playbook, which covers patterns for tool binding, memory strategies, and error recovery across multi-agent workflows.