Design a Chat System
Practice a chat-system design with persistent connections, conversation ordering, delivery acknowledgements, presence, offline sync, and real-time scaling.
What you'll learn
Run the architecture drill
- How to separate durable message writes from real-time fanout.
- How to scope ordering, delivery, and read receipts to a conversation.
- How reconnect and offline sync change a chat architecture.
A chat system combines a durable messaging record with a low-latency delivery experience. It must make conversation ordering, connection ownership, delivery state, offline synchronization, media handling, and presence semantics explicit without assuming a user's network connection is always trustworthy.
Start with the architecture drill, then follow the request path through scale, failure, and a defensible interview answer.
Make your assumptions and decision path easy to inspect.
ATOFF architecture reasoning canvas
Design a Chat System: the architecture drill
Short answer
A chat system combines durable conversation records with low-latency delivery, making ordering, acknowledgement, reconnects, and offline sync explicit.
10× evolution
Partition by conversation and decouple socket fanout from durable delivery.
01 Decision checkpoint
Design brief
Assign a server-side sequence or durable append position per conversation so clients can detect gaps without pretending global message order exists.
02 Decision checkpoint
First request path
Track acceptance, recipient delivery, and read receipt as different states because a connected device and a human reader are not the same proof.
03 Decision checkpoint
Failure drill
Use a durable cursor and missed-message sync so socket ownership changes or mobile network loss do not silently discard the conversation history.
Say it in the interview
I would define per-conversation ordering and delivery semantics, then show how a reconnecting client fills gaps from durable storage.
- 1
Sender device
persistent connection: Connection gateway
- 2
Connection gateway
send: Message service sequence + auth
- 3
Message service sequence + auth
durable write first: Conversation log
- 4
Conversation log
message event: Realtime fanout stream · replay on reconnect: Offline sync from history
- 5
Realtime fanout stream
live delivery: Recipient gateway
- 6
Recipient gateway
- 7
Offline sync from history
catch up: Recipient gateway
- Sender device flows to Connection gateway via persistent connection.
- Connection gateway flows to Message service sequence + auth via send.
- Message service sequence + auth flows to Conversation log via durable write first.
- Conversation log flows to Realtime fanout stream via message event.
- Realtime fanout stream flows to Recipient gateway via live delivery.
- Conversation log flows to Offline sync from history via replay on reconnect.
- Offline sync from history flows to Recipient gateway via catch up.
Walk the design under pressure
Clarify the conversation contract
Ask whether this is one-to-one chat, groups, channels, or broadcast; whether messages can be edited or deleted; whether attachments and search are in scope; and what delivery and read states mean. Ordering is usually required within a conversation, not globally across every user.
Assign a stable conversation ID and message ID. The sender needs an idempotency key because a mobile client may retry after losing its response even though the service already persisted the message.
Persist before claiming a message exists
The message service should validate membership, persist a durable record with a conversation-local ordering key, and return a state the sender can reconcile. Real-time gateways can deliver the persisted message afterward. This avoids treating a volatile socket send as the source of truth.
A single conversation can be routed to one ordering owner or partition so concurrent writers get a well-defined sequence. That sacrifices some global flexibility in exchange for a simple guarantee users can understand.
Use real-time connections for delivery, not history
Connection gateways keep track of which devices are currently reachable, but they cannot guarantee a device received or displayed a message. On reconnect, a client requests messages after its last durable cursor. Delivery and read receipts are separate events with their own privacy and ordering semantics.
Fanout strategy changes with group size. Small groups can push directly to connected members. Large channels may need a broadcast or pull model so one message does not force an unbounded write or connection burst.
Staff-level insight: make connection and conversation failure visible
At staff level, distinguish message-write failures from fanout lag, regional connection loss, notification fallback, and client synchronization bugs. Track message age, per-conversation ordering errors, reconnect storms, and gateway capacity. Preserve privacy by limiting what presence and read state reveal.
In an interview, explain what changes at ten times scale: partition conversations more deliberately, separate media, add regional gateway routing, and keep the durable history path recoverable even when real-time delivery is degraded.
Keep this with you
Key takeaways
- A socket is a delivery channel, not the authoritative record of a message.
- Scope ordering to the conversation or key that truly needs it.
- Reconnect and offline sync require durable cursors and idempotent writes.
Practice aloud
Interview questions to explore
- 1.How do you ensure a retried mobile send does not create two messages?
- 2.Where is message order guaranteed, and where is it not?
- 3.How does an offline recipient catch up safely?
Common follow-ups
Frequently asked questions
Do chat systems need WebSockets?
They are useful for low-latency delivery and presence, but the durable message and history APIs remain necessary for reconnect, offline sync, search, and recovery.
Can every chat message be globally ordered?
It is possible but costly and usually unnecessary. Per-conversation ordering matches the user expectation while allowing the system to scale across many conversations.
Already an Elite member? Open the complete walkthrough.
Need the broader preparation context? Go back to Interview Preparation for behavioral readiness, question practice, and the larger AceTheOffer preparation framework.