peopleanalyst

library / lib62a8f8d13dab550e

Architecture Patterns with Python

Harry J. W. Percival & Bob Gregory · 2020

In a sentence

A practical guide to applying Domain-Driven Design, Test-Driven Development, and Event-Driven Architecture patterns in Python to build maintainable, testable, and loosely coupled systems.

Architecture Patterns with Python teaches Python developers how to manage growing complexity in real-world applications by systematically applying battle-tested architectural patterns drawn from the DDD, TDD, and microservices traditions. Starting from a concrete e-commerce allocation domain, the book walks step-by-step through building a layered architecture—Domain Model, Repository, Service Layer, Unit of Work, Aggregates—and then extends it into an event-driven system using Domain Events, a Message Bus, Commands, CQRS, and Dependency Injection. Every pattern is introduced test-first, with working Python code, explicit trade-off tables, and honest discussion of costs. By the end, readers have a blueprint for writing Python applications whose complexity grows slowly relative to their size, whose core business logic is fully decoupled from infrastructure, and whose test suites are fast, meaningful, and pyramidal.

The four lenses

  • Science
  • Statistics
  • Systems
  • Strategy

Tags

f1-systems

The model

A causal model describing how architectural design levers (patterns, principles, and practices) shape intermediate code-quality and team-behavioral states, which in turn drive system-level outcomes such as maintainability, testability, and delivery speed.

Dependency Inversion Principle Applicationdesign lever

The degree to which high-level modules (domain, service layer) depend on abstractions rather than on concrete low-level infrastructure modules such as ORMs, email libraries, or HTTP frameworks. Operationalized as explicit abstract base classes or duck-typed interfaces between layers.

Domain Model Puritydesign lever

The extent to which domain model classes contain no direct imports of or dependencies on infrastructure concerns such as ORMs, web frameworks, email clients, or databases. A pure domain model consists solely of plain Python objects expressing business rules and concepts.

Repository Pattern Adoptiondesign lever

The use of a repository abstraction that hides persistent storage details behind a simple add/get interface, allowing the domain and service layers to be tested without a real database by substituting an in-memory fake repository.

Service Layer Adoptiondesign lever

The presence of a dedicated orchestration layer that defines use-case entry points, coordinates repositories and domain services, handles transactions via the Unit of Work, and presents a technology-agnostic API to entrypoints such as Flask or CLI tools.

Unit of Work Pattern Adoptiondesign lever

The use of a Unit of Work abstraction that manages atomic database transactions, provides access to repositories within a single context, and collects domain events for publishing, decoupling service functions from session management.

Aggregate and Consistency Boundary Designdesign lever

The degree to which the domain model is organized around well-chosen aggregates that act as single entrypoints for modifying related objects, enforce invariants within their boundary, and constrain the scope of each database transaction to one aggregate.

Event-Driven Design Adoptiondesign lever

The extent to which the system models significant business occurrences as explicit Domain Event objects, uses a Message Bus to route events to handlers, and treats both internal workflows and external integrations as event-processing pipelines rather than direct method calls.

CQRS Adoptiondesign lever

The degree to which read operations are served by separate, simplified read models or raw SQL views rather than forcing the write-optimized domain model to serve query needs, reducing complexity and enabling independent scaling of reads and writes.

Dependency Injection and Bootstrappingdesign lever

The practice of explicitly declaring all external dependencies (UoW, email, message publisher) as function or class parameters and wiring them together in a single bootstrap script, enabling easy substitution of real and fake implementations across entrypoints and tests.

Test Pyramid Shapebehavioral pattern

The distribution of automated tests across unit, integration, and end-to-end levels, with a healthy pyramid having many fast unit tests, fewer integration tests, and very few end-to-end tests. An inverted pyramid (ice-cream cone) indicates over-reliance on slow, brittle tests.

Infrastructure Couplingpsychological state

The degree to which business logic modules are tightly coupled to specific infrastructure choices such as a particular ORM, web framework, database engine, or messaging library, making it expensive to change any infrastructure component without modifying business logic.

Global Code Coupling (Big Ball of Mud)contextual condition

The overall degree of entanglement among modules in the codebase, characterized by homogeneity of function across layers, business logic scattered across web handlers and model classes, and dependencies that make changing any component risky and expensive.

Domain Model Expressivenesspsychological state

The degree to which the domain model code reads in the language of the business domain, uses ubiquitous language for class and method names, and serves as living documentation that non-technical stakeholders could validate, making the model a faithful representation of business rules.

Testabilitypsychological state

The ease with which individual components of the system can be exercised in isolation with fast, deterministic tests that do not require real databases, email servers, or HTTP services, enabled by well-defined abstractions and injectable fake dependencies.

Consistency and Invariant Enforcementbehavioral pattern

The degree to which the system reliably prevents invalid states—such as over-allocation of stock or double-booking—through a combination of aggregate boundaries, version numbers for optimistic locking, and transaction isolation, even under concurrent load.

Inter-Service Couplingcontextual condition

The degree to which separate services or subsystems depend on one another synchronously and directly (high coupling, e.g., synchronous HTTP RPC chains) versus communicating through asynchronous events with no direct dependency on each other's availability (low coupling).

System Maintainabilityoutcome metric

The long-term ease of modifying, extending, and refactoring the codebase in response to changing business requirements without introducing regressions, measured by the cost and risk of making changes as the system grows in size and complexity.

Delivery Speed and Agilityoutcome metric

The rate at which the team can safely ship new features and bug fixes, influenced by test suite speed, confidence from automated tests, and the ease of reasoning about the impact of a change in a well-structured codebase.

System Reliability and Resilienceoutcome metric

The ability of the system to continue functioning correctly and to recover gracefully when individual components fail, enabled by small atomic transactions, independent failure of event handlers, retry logic, and eventual consistency between aggregates.

Read Performance and Scalabilityoutcome metric

The speed and horizontal scalability of query operations, improved by separating read models from the write-optimized domain model, using denormalized views or caches, and avoiding SELECT N+1 patterns inherent in ORM-based domain traversal for reads.

How they connect

  • dependency inversion influences infrastructure coupling
  • dependency inversion influences testability
  • domain model purity influences domain model expressiveness
  • domain model purity influences testability
  • repository pattern influences infrastructure coupling
  • repository pattern influences testability
  • service layer influences test pyramid shape
  • service layer influences global code coupling
  • unit of work pattern influences consistency enforcement
  • unit of work pattern influences testability
  • aggregate design influences consistency enforcement
  • aggregate design influences infrastructure coupling
  • event driven design influences service coupling between systems
  • event driven design influences global code coupling
  • event driven design influences system reliability
  • cqrs adoption influences read performance
  • cqrs adoption influences global code coupling
  • dependency injection influences testability
  • dependency injection influences infrastructure coupling
  • infrastructure coupling influences maintainability
  • global code coupling influences maintainability
  • global code coupling influences delivery speed
  • testability influences test pyramid shape
  • test pyramid shape influences delivery speed
  • test pyramid shape influences maintainability
  • consistency enforcement influences system reliability
  • service coupling between systems influences system reliability
  • service coupling between systems influences maintainability
  • domain model expressiveness influences maintainability
  • read performance influences delivery speed

The process

The book's overall operating playbook guides a developer in building a robust, maintainable, and scalable Python application by progressively applying a series of architectural patterns. The journey begins with establishing a 'pure' domain model using Test-Driven Development (TDD) and Domain-Driven Design (DDD), ensuring the core business logic is isolated from infrastructure concerns. This core is then protected by applying the Dependency Inversion Principle through patterns like the Repository, Service Layer, and Unit of Work, which create a clean separation between the domain and external systems like databases and web frameworks. Once this solid, decoupled foundation is in place, the playbook evolves the architecture into an event-driven system. By introducing Domain Events, Commands, and a Message Bus, the application is transformed into a message processor. This decouples components even further, allowing for complex, asynchronous workflows that can span multiple parts of the system or even integrate with other microservices. This approach enhances resilience and flexibility, as different parts of a business process can execute independently. Finally, the playbook introduces advanced patterns for optimization and cleanup. Command-Query Responsibility Segregation (CQRS) is presented as a method to scale read and write operations independently, improving performance for read-heavy applications. To manage the growing number of dependencies cleanly, the book advocates for a centralized Dependency Injection mechanism using a bootstrap script. The cumulative effect of these processes is a well-structured, highly-testable, and adaptable system capable of managing complex business requirements while avoiding the common 'Big Ball of Mud' anti-pattern.

Develop the Domain Model with TDD

To create a pure, infrastructure-agnostic representation of the core business logic and rules, ensuring it is correct, easy to understand, and easy to change.

When to use: At the beginning of a new project or when carving out a new domain from an existing system.

  1. Step 1Explore the domain language with business experts.

    Entry: Access to domain experts and a basic understanding of the business problem.

    Exit: A glossary of terms and a set of concrete examples illustrating business rules.

    In: Business requirements, Conversations with domain experts · Out: Ubiquitous language glossary, Example scenarios

  2. Step 2Write a failing unit test for a specific business rule.

    Entry: A specific business rule has been identified.

    Exit: A single failing unit test exists in the test suite.

    In: A business rule · Out: A failing unit test

  3. Step 3Implement the simplest domain model code to make the test pass.

    Entry: A failing unit test exists.

    Exit: All unit tests are passing.

    In: Failing unit test · Out: Minimal domain model code

  4. Step 4Distinguish between Entities, Value Objects, and Domain Services.

    Entry: The domain model is growing and needs more structure.

    Exit: Domain concepts are clearly implemented as either Entities, Value Objects, or Domain Services.

    • Does this concept have a persistent identity? (Entity vs. Value Object)
    • Is this operation a standalone process? (Domain Service)

    In: Business concepts · Out: Entity classes, Value Object classes, Domain Service functions

  5. Step 5Refactor the code and tests for clarity.

    Entry: All tests are passing.

    Exit: The code is clean, expressive, and easy to understand.

    In: Passing test suite, Working domain code · Out: Refactored domain code and tests

  6. Step 6Repeat the cycle for the next business rule.

    Entry: The previous feature is complete and refactored.

    Exit: The domain model correctly implements all required business logic.

Decouple the Domain from Infrastructure

To isolate the pure domain model from external concerns like databases, file systems, and web frameworks, making the system more flexible, testable, and maintainable.

When to use: After establishing a basic domain model and needing to connect it to persistence and user-facing entrypoints.

  1. Step 1Implement the Repository pattern to abstract persistence.

    Entry: A domain model exists and needs to be persisted.

    Exit: The application can save and retrieve domain objects via a repository abstraction, and can be tested without a real database.

    In: Domain model (specifically, aggregates) · Out: AbstractRepository interface, Concrete repository implementation, FakeRepository for tests

  2. Step 2Introduce a Service Layer to orchestrate use cases.

    Entry: The application needs a clear entrypoint for its business workflows.

    Exit: Business logic orchestration is contained within a service layer, separate from both the domain model and the UI/API layer.

    In: User actions/requests, Repository abstractions · Out: Service layer functions

  3. Step 3Implement the Unit of Work (UoW) pattern for atomic operations.

    Entry: Service layer functions need to perform multiple database operations that must succeed or fail together.

    Exit: Service layer operations are atomic, and the service layer is decoupled from the specifics of session/transaction management.

    In: Service layer functions, Repository implementations · Out: AbstractUnitOfWork interface, Concrete UoW implementation, FakeUnitOfWork for tests

  4. Step 4Define Aggregates to enforce consistency boundaries.

    Entry: The domain model has multiple related objects, and there are business rules (invariants) that span them.

    Exit: The domain model is organized into aggregates, and repositories operate at the aggregate level.

    • Which objects must be consistent with each other at all times?

    In: Domain model · Out: Aggregate root classes

  5. Step 5Implement concurrency control on aggregates.

    Entry: The application will have concurrent users modifying the same data.

    Exit: Concurrent updates to the same aggregate are safely handled, preventing data corruption.

    • Optimistic vs. Pessimistic locking (e.g., SELECT FOR UPDATE).

    In: Aggregate root classes · Out: Concurrency control mechanism

Evolve to an Event-Driven Architecture

To further decouple application components, handle complex workflows that cross consistency boundaries, and enable asynchronous processing by transforming the application into a message processor.

When to use: When a use case needs to trigger multiple, independent follow-up actions, or when integrating with other microservices.

  1. Step 1Introduce Domain Events and a Message Bus.

    Entry: A use case has side effects (e.g., sending an email) that should be decoupled from the core transaction.

    Exit: Side effects are handled by event handlers, triggered via the message bus after the main transaction completes.

    In: Domain model aggregates · Out: Event classes, Message Bus, Event handlers

  2. Step 2Refactor the application to be message-driven.

    Entry: The application has multiple use cases and internal workflows that could be unified under a single message-passing paradigm.

    Exit: All application logic is triggered by handling either a command or an event via the message bus.

    In: Service layer functions, Event classes · Out: Command classes, Command handlers, A unified message bus

  3. Step 3Implement cross-aggregate workflows using events.

    Entry: A business process requires modifying multiple aggregates, but they don't need to be updated in a single atomic transaction.

    Exit: Complex workflows are broken down into a series of independent handlers linked by events.

    In: Complex business process requirement · Out: A chain of command and event handlers

  4. Step 4Integrate with external systems via an external message broker.

    Entry: The application needs to communicate asynchronously with other microservices.

    Exit: The application is integrated into a larger microservices ecosystem, consuming and producing events.

    In: Internal events and commands · Out: Event consumer adapter, Event publisher adapter

Implement Command-Query Responsibility Segregation (CQRS)

To optimize read and write operations independently, improving performance, scalability, and simplicity for each path.

When to use: When the domain model, which is optimized for writes and enforcing invariants, becomes inefficient or awkward for querying.

  1. Step 1Separate command and query entrypoints.

    Entry: The application has endpoints that both change state and return data.

    Exit: API endpoints are clearly separated into commands (writes) and queries (reads).

    In: API design · Out: Separate command and query endpoints

  2. Step 2Create a dedicated, optimized read model.

    Entry: Queries against the write model are becoming slow or complex.

    Exit: A separate data schema for the read model is defined and created.

    • Should the read model be in the same database or a different technology (e.g., Redis, Elasticsearch)?

    In: Query requirements · Out: Read model schema

  3. Step 3Create event handlers to update the read model.

    Entry: A read model exists and needs to be kept up-to-date.

    Exit: The read model is updated automatically whenever the write model changes.

    In: Domain events · Out: Event handlers for read model updates

  4. Step 4Implement query endpoints to use the read model.

    Entry: The read model is being populated by event handlers.

    Exit: All read operations are served quickly and efficiently from the dedicated read model.

    In: Read model · Out: Fast query endpoints

Centralize Dependency Management with a Bootstrap Script

To cleanly manage and inject dependencies into handlers and other components, removing setup and configuration logic from application entrypoints and tests.

When to use: When you find yourself repeating dependency setup code in multiple places, such as in your Flask app, your Redis consumer, and your test fixtures.

  1. Step 1Make dependencies explicit in handler signatures.

    Entry: Handlers have implicit, hardcoded dependencies on concrete implementations.

    Exit: Handlers depend on abstract interfaces passed in as arguments.

    In: Command and event handlers · Out: Refactored handlers with explicit dependencies

  2. Step 2Create a bootstrap script as the composition root.

    Entry: Dependencies are instantiated in multiple places.

    Exit: A single `bootstrap()` function is responsible for creating all dependencies.

    Out: A `bootstrap.py` module

  3. Step 3Inject dependencies into handlers within the bootstrap script.

    Entry: Handlers require dependencies to be passed in manually.

    Exit: The bootstrap script produces a set of handlers that are ready to be called with their dependencies already injected.

    • Use function inspection for automatic injection or manually wire up each handler with `partial`.

    In: Handlers with explicit dependencies, Concrete dependency instances · Out: A dictionary of dependency-injected handlers

  4. Step 4Configure the message bus with the injected handlers.

    Entry: The message bus has a static, global mapping of handlers.

    Exit: The message bus is configured dynamically at startup by the bootstrap script.

    In: Dependency-injected handlers · Out: A configured MessageBus instance

  5. Step 5Update entrypoints and tests to use the bootstrap script.

    Entry: Entrypoints and tests perform their own dependency setup.

    Exit: All dependency setup is centralized in the bootstrap script, and entrypoints/tests are simplified.

    Out: Simplified entrypoint and test code

The story

The reader A Python developer working on a moderately complex application who wants clean, maintainable code but keeps ending up with a tangled ball of mud that is hard to test, change, or reason about.

External problem

Business logic is spread across web handlers, ORM models, and utility modules; tests are slow, fragile, or absent; adding features breaks existing behavior.

Internal problem

The developer feels frustrated, embarrassed, and stuck—unsure whether their architecture is fundamentally broken or whether they are just missing the right vocabulary and tools.

Philosophical problem

It is wrong that growing a codebase should make it progressively harder to change; software should remain malleable as it matures.

The plan

  1. Model the business domain in pure Python objects (Entities, Value Objects, Domain Services) with no infrastructure dependencies, driven by unit tests.
  2. Introduce the Repository pattern to abstract persistent storage, enabling trivial in-memory fakes for testing.
  3. Add a Service Layer to define use-case boundaries, keep controllers thin, and make the bulk of tests fast and decoupled.
  4. Apply the Unit of Work pattern to group operations atomically and give tests a single seam for controlling database state.
  5. Introduce Aggregates to enforce consistency boundaries and manage concurrency safely.
  6. Adopt Domain Events and a Message Bus to decouple side effects from core use cases.
  7. Distinguish Commands from Events to clarify error-handling semantics.
  8. Integrate microservices via asynchronous external events using Redis pub/sub.
  9. Apply CQRS to use simple read models for queries while preserving a rich domain model for writes.
  10. Wire everything together with a bootstrap script and explicit dependency injection, making production and test configurations easy to swap.

Success

  • A test pyramid dominated by fast, dependency-free unit tests with a small number of integration and E2E tests.
  • A domain model that can be changed or refactored without touching infrastructure or rewriting tests.
  • New features that fit cleanly into existing architecture without increasing global complexity.
  • Microservices that communicate through well-defined events and can evolve independently.
  • A codebase new developers can onboard to quickly by reading living-documentation tests written in domain language.

At stake

  • The application becomes an unmaintainable 'big ball of mud' where every change risks breaking something else.
  • Test suites take hours to run, providing little confidence and slowing delivery.
  • Business logic is scattered across web handlers, ORM models, and manager classes, invisible to domain experts.
  • The team becomes afraid to refactor and accumulates technical debt until the system must be rewritten.
  • Microservices become a 'distributed big ball of mud' with tight temporal coupling through synchronous HTTP chains.

Questions this book answers

How do you keep business logic clean and free of infrastructure concerns as a Python application grows?
How do you structure automated tests so that fast unit tests dominate and slow integration/E2E tests are minimized?
How do you model a business domain in code using Entities, Value Objects, Aggregates, and Domain Services?
How do you decouple persistence from the domain model using the Repository and Unit of Work patterns?
How do you implement event-driven workflows inside a single service and across multiple microservices?

Related in the library

Tools these methods power