The Paradox of Complexity

When we start building systems, adding features seems free. A microservice here, a message queue there, a cache layer to speed things up. Each decision seems reasonable in isolation.

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

But complexity isn’t additive; it’s multiplicative. Each new component doesn’t just add its own complexity—it multiplies the possible interactions with all other components.²

The Hidden Costs

Complexity has costs that rarely appear in budgets:

1. Diagnosis time — Every incident takes longer to investigate
2. Onboarding — New engineers need months to understand the system
3. Design decisions — Every change requires considering more implications
4. Testing — The matrix of possible interactions grows exponentially

A Real Example

Years ago, I worked on a payment processing system that had grown organically over a decade. It had:

• 47 microservices
• 12 different databases
• 8 message queues
• 3 caching systems
• Approximately 150 external integrations

┌─────────────────────────────────────────────────────┐
│                    COMPLEXITY                        │
├─────────────────────────────────────────────────────┤
│  Components: O(n)                                    │
│  Interactions: O(n²)                                 │
│  Failure modes: O(2ⁿ)                                │
└─────────────────────────────────────────────────────┘

The mean time to resolution for incidents had gone from minutes to hours. Not because the engineers were less capable, but because understanding what was happening required navigating an increasingly dense web of dependencies.

Deliberate Simplicity

Simplicity is not the absence of functionality; it’s the absence of unnecessary complexity. It’s the result of conscious and often painful decisions.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

Practical Principles

I’ve learned to apply some principles:

1. Prefer monolith first

Start with the simplest system that can work. Only split when you have clear evidence that splitting will solve a real problem.³

2. Measure before optimizing

Complexity added for “performance” often doesn’t provide measurable improvements. Even worse, it can add latency due to the overhead of communication between components.

3. Document decisions, not just code

Every architectural decision should have an ADR (Architecture Decision Record) that explains:

• The context
• The alternatives considered
• The decision made
• The expected consequences

4. Review complexity periodically

What made sense two years ago might not make sense now. Schedule regular architecture reviews with the question: “Is this still providing value?”

Conclusion

Simplicity is a discipline, not a natural state. It requires constant effort, difficult decisions, and the willingness to say “no” to features that seem attractive but add unnecessary complexity.

The best systems I’ve seen aren’t the most sophisticated; they’re the ones that do one thing, do it well, and are simple enough that any team member can understand them in an afternoon.

This is the first entry in a series about lessons learned after nearly two decades in SRE. The next entry will cover observability and why “more metrics” isn’t always better.