Software Architecture Principles Every Senior Engineer Should Know

Santheepkumar

Santheepkumar / May 20, 2024

6 min read––– views

What Architecture Actually Is

Architecture is not the boxes-and-arrows diagram you draw before a project starts. It is the set of decisions in your system that are expensive to change later. The database engine, the service boundaries, the authentication model, the communication patterns between components - these are architectural decisions because reversing them means rewriting large parts of the system.

Good architecture is not about predicting the future. It is about preserving your ability to change your mind. Every architectural choice is a bet: "we believe this constraint will matter more than that one." The goal is to make the high-stakes bets carefully and keep the low-stakes ones flexible.

Principle 1: Separate What Changes from What Does Not

Every system has parts that change frequently and parts that are stable. The business logic of an e-commerce checkout changes constantly. The fact that HTTP requests come in and responses go out does not. Good architecture keeps these separate.

This is the core insight behind layered architectures, hexagonal architecture, and clean architecture - they all try to protect the stable core (domain logic) from the volatile edges (framework, database, UI). When your domain model does not know what HTTP is, you can swap your web framework without touching your business rules.

A practical test: if your framework changed tomorrow, what would break? If the answer is "everything," your architecture has coupled the stable and the volatile together.

Principle 2: Design for the 10x, Not the 1000x

Over-engineering is the most common architectural mistake senior engineers make. You design for a million users when you have a thousand. You add a message queue when a function call would do. You split into microservices when a modular monolith would ship faster and be easier to operate.

A useful rule: optimize for the 10x growth scenario, not the 1000x. If you have 1,000 users today, design for 10,000 - not 1,000,000. This usually means: a single well-structured service, a managed relational database, aggressive caching at the edges, and clear module boundaries inside the monolith. You can extract services when the seams become obvious from real usage, not from speculation.

The cost of premature distribution is real: distributed systems are harder to debug, harder to operate, and require coordination overhead that slows down small teams dramatically.

Principle 3: Make Failure a First-Class Concern

Most architecture discussions focus on the happy path. The system works, the database is up, the third-party API responds. Real production systems fail constantly, and the systems that survive are designed around failure, not despite it.

Questions every design should answer:

  • What happens when the database is unavailable for 30 seconds?
  • What happens when this API call takes 10 seconds instead of 200ms?
  • What happens when this job fails halfway through and leaves partial data?
  • What happens when two instances of this service run simultaneously?

The answers drive real design decisions: circuit breakers, idempotency, timeouts and retries with backoff, dead letter queues, optimistic locking. These are not nice-to-haves. They are what separates a system that handles a 3am incident gracefully from one that pages you every weekend.

Principle 4: Data is the Hardest Part

Code is easy to change. Schema is not. The decisions you make about your data model early in a project will follow you for years. Migrations on large tables are slow and risky. Denormalized data that made sense for an early access pattern becomes a liability when requirements change.

Invest heavily in getting your domain model right before you persist it. The questions that matter:

  • What are the natural aggregates in this domain? (These should usually map to tables or document collections.)
  • Where are the boundaries of consistency? (Everything in one transaction, or eventually consistent across services?)
  • What are the access patterns? (A normalized model that requires 12 joins for every API response will cause problems at scale.)
  • How will the data grow? (Time-series data, event logs, and user-generated content have very different growth profiles.)

The most common mistake is designing the database to reflect the code structure rather than the business domain.

Principle 5: APIs Are Forever

Once an API is consumed by clients or other services, changing it is expensive. Clients break. You have to version. You carry the old behavior as technical debt. Treat API design with the same care you give database schema.

Good API design principles:

  • Name things from the caller's perspective, not the implementation's. A POST /users/send-verification-email is worse than POST /users/{id}/verify.
  • Be conservative in what you accept, liberal in what you return. Adding new fields to responses is safe. Removing them is not.
  • Version from day one if you have external consumers. /api/v1/ is cheap to add upfront and expensive to retrofit.
  • Pagination, filtering, and sorting are architectural decisions. How you handle these affects every consumer and is hard to change without breaking them.

Principle 6: Observability Is Not Optional

A system you cannot observe is a system you cannot debug, and a system you cannot debug is one you cannot operate confidently. Observability is not a feature to add later - it is a design constraint from the start.

Three pillars:

  • Metrics: What is the system doing right now? Request rate, error rate, latency at p50/p95/p99, queue depth. These are your operational dashboard.
  • Logs: What happened? Structured logs (JSON, not strings) with consistent fields (request ID, user ID, service name, duration) let you correlate events across services.
  • Traces: Why is this request slow? Distributed tracing (OpenTelemetry, Jaeger) shows you the full call tree across service boundaries.

Design your services to emit all three from day one. Adding them retroactively, in the middle of an incident, is the worst time to learn that your logs have no correlation IDs.

Conclusion

Architecture principles are not rules to follow blindly. They are heuristics built from the accumulated pain of systems that failed in predictable ways. Understand the why behind each one - what failure mode it prevents, what tradeoff it makes explicit - and you will know when to apply it and when the specific constraints of your system call for a different approach.

The best architects are not the ones who know the most patterns. They are the ones who can look at a system and accurately identify which decisions are expensive to reverse, which constraints actually matter for this product, and what the team can realistically operate and maintain. That judgment, more than any diagram, is what good architecture is.

Subscribe to the newsletter

Get emails from me about web development, tech, and early access to new articles.

- subscribers