Retries are inevitable
Over a network, a client that times out cannot know what happened. So it will retry — and the system must be built to survive that.
A timeout tells the client only that no reply arrived in time. It is consistent with at least three different histories:
- the request never reached the server;
- the server received it and failed before acting;
- the server acted, and the reply was lost or delayed.
Waiting longer doesn’t settle the question in general, because a slow reply and a lost one look the same until the slow one arrives. Practical clients therefore retry, and correct systems are designed on the assumption that any request may arrive more than once.
Two consequences:
- Operations need to be safe to repeat — see Idempotency, and Why Retries Need It.
- Retries need pacing, or many clients retrying together can overload a recovering server — see the backoff playground.