Platforms & Distributed Systems
When a product grows, the problem stops being a feature and becomes how the services talk to each other. That is where service boundaries, event delivery and flows that cannot be lost along the way come in.
The symptom is usually the same: two teams can no longer deploy without coordinating, a multi-step process stalls halfway and nobody knows where, or the same piece of data tells different stories depending on which service you ask.
Splitting a monolith is not the default answer. First comes finding where the business boundaries actually are, and only then does anything get separated, with the events and contracts that requires.
Where the work goes
Where to cut
A badly cut microservice is worse than the monolith it came from: same coupling, more latency, more deployments. Before anything is split, the map says which part of the business writes which data, and the cut follows that, not the technical layers.
Events you can reprocess
An event is a contract with the future: someone will read it a year from now, and last week’s may need reprocessing. That is why they ship with a versioned schema, a deliberate partition key and idempotent consumers.
Processes that do not get lost
An onboarding, a charge or a reconciliation are several steps with waits and failures in between. With Temporal the process state lives outside the service, so a deploy or a crash does not leave anyone stranded, and you can see which step each case is on.
Actually multitenant
Isolating clients is not just filtering by tenant_id. It is deciding what is shared and what is not across data, configuration, usage limits and deployment, and putting that somewhere it cannot be forgotten when the next endpoint is written.
How it fits together
One event, end to end
Order matters here: if the event is published before the write is committed, sooner or later you publish something that never happened.
Service A
Outbox
Kafka
Service B
Writes the change and the event in the same transaction
A publisher reads the table and publishes, at least once
Consumes its partition, in order per key
If it already processed that id, it drops it
Commits the offset only once it is done
A workflow that can fail halfway
Reserve, charge and confirm are three steps with three different owners. Temporal keeps track of where it stopped, so a restart does not erase the progress.
Scheduled
Something triggers it and it is recorded with its business id.
Running
Each step is an activity with its own timeout.
Waiting to retry
The provider answered badly or not at all. It waits and tries again.
Completed
The result is written once, even if the step ran more than once.
Compensated
If there is no way through, the work is undone and the case is closed rather than left hanging.
Retries are part of the design, not an exception: backoff, a cap on attempts and, at the end, an explicit compensation.
What it includes
- Microservice design: where to cut, who owns which data and what contract sits between them.
- Event-driven architecture on Kafka: topics, partitions, schemas and backward compatibility.
- Durable workflows with Temporal for multi-step processes that must survive a restart.
- Service-to-service communication with gRPC and Protobuf when latency matters, REST when it does not.
- Consistency patterns: saga, transactional outbox, idempotency and retries with backoff.
- Multitenant design: data, configuration and deployment isolation per client.
- API gateway and BFF so each client gets what it needs without coupling to every service.
- OLAP analytics (ClickHouse) kept separate from the transactional database.
- A migration strategy: extract one service at a time without freezing the product.
How the project runs
- 01The current flow gets mapped with your team, marking where data is lost today.
- 02Service boundaries and events, with their schemas, are settled before a line of code moves.
- 03One full end-to-end flow migrates before the rest is touched, so the risk shows up small.
- 04The system is left observable: what was queued, what retried and what failed.
What you get
- The migrated flows, with versioned event schemas.
- A service map with data ownership per service.
- Dashboards to see queue and workflow state.
- A decision record: what was chosen and why.
Usual stack
- Messaging
- Apache KafkaRabbitMQRedis StreamsPub/Sub
- Orchestration
- TemporalApache AirflowSidekiqCelery
- Contracts
- gRPCProtobufOpenAPIJSON Schema
- Services
- PythonNestJSRuby on RailsGo
- Data
- PostgreSQLClickHouseRedisDynamoDB
- Patterns
- SagaOutboxIdempotencyEvent sourcingCQRS
Where this has been done
Frequently asked questions
Does the monolith have to be split?
Almost never entirely, and never as the first move. A badly cut microservice is worse than the monolith it came from. The first step is seeing which part of the business writes which data; sometimes the answer is extracting two services and leaving the rest alone.
How does a migration happen without freezing the product?
By migrating one full end-to-end flow before touching anything else. The risk shows up small, the team learns the pattern on a real case, and the product keeps delivering meanwhile.
Is Kafka needed, or is a queue enough?
It depends on whether the past needs rereading. A queue hands out work and forgets it; Kafka keeps the log, so a new consumer can reprocess last week. If nobody is going to reprocess anything, a plain queue is cheaper to run.
When is a durable workflow better than a queue?
When the process has several steps with waits and different owners: an onboarding, a charge, a reconciliation. With Temporal the state lives outside the service, so a deploy or a crash leaves no case half done, and you can see which step each one is on.