Why TCP Congestion Control Makes Networks Slower After Loss

Reading time: 8 minute Word count: 1602

Networking TCP Congestion Control Retransmission Throughput

When a network is slow, bandwidth is often the first thing people check.

Bandwidth looks sufficient, but HTTPS downloads are slow. Cellular throughput jumps up and down. Wi-Fi signal looks acceptable, but API calls occasionally time out. Packet captures show retransmissions, duplicate ACKs, RTOs, and then the sending rate drops. The server CPU is fine and the application did not change, but TCP suddenly becomes conservative.

Congestion control is often the reason.

The safest first model is this: TCP cannot send only according to what the application wants. It must estimate how much unacknowledged data the network path can currently carry, and it must reduce its sending rhythm when loss, rising delay, or retransmission signals suggest congestion.

application wants to send
-> TCP is limited by receive window rwnd
-> TCP is also limited by congestion window cwnd
-> actual in-flight data <= min(rwnd, cwnd)

So throughput debugging is not only about link bandwidth. Ask how large RTT is, how much loss exists, whether the congestion window was reduced, and whether retransmissions or timeouts are changing the sending rhythm.

Congestion Control Is Not Flow Control

TCP has two windows that are easy to mix up.

The receive window rwnd is advertised by the receiver. It says how much more data the receiver can accept. It answers: can the peer receive fast enough?

The congestion window cwnd is maintained by the sender. It says how much unacknowledged data the sender believes the network path can carry. It answers: can the network carry this much?

The actual amount of in-flight data is limited by both:

in-flight data <= min(rwnd, cwnd)

If rwnd is small, the receiver or its application is slow. If cwnd is small, the sender believes the path is congested.

If a capture shows Zero Window, look at the receiver first. If throughput falls after retransmissions, duplicate ACKs, or RTOs, the issue is more likely path quality and congestion control.

Why TCP Does Not Start at Full Speed

At connection start, TCP does not know the path capacity.

The sender does not know the bottleneck bandwidth, queue depth, RTT, wireless loss, or whether a congested gateway exists. If it sends at local maximum speed immediately, it may fill intermediate queues and create more delay and loss.

So TCP begins with a relatively conservative initial congestion window. Early in the connection, slow start is common: each ACK lets the sender increase how much data may be in flight.

The name is misleading. Slow start is not simply slow. It starts from a small window and quickly probes capacity.

small cwnd
-> ACKs return
-> cwnd grows quickly
-> until threshold, loss, or algorithm transition

This matters for short connections. If the object is small, the connection may finish before the sender has probed a large enough window. The larger RTT is, the more visible this becomes for short requests.

Why Loss Makes the Sender Conservative

In traditional TCP congestion control, packet loss is often treated as a congestion signal.

The sender becomes concerned when it sees signals such as:

repeated duplicate ACKs suggesting a missing segment
no ACK before timeout, causing RTO
SACK showing some data arrived while some is missing
rising RTT suggesting queue buildup

The cost of loss is not just “send the packet again.” It creates a chain reaction:

missing data must be retransmitted
later in-order delivery may be blocked
congestion window may shrink
the sender must probe capacity again
RTO recovery is slower

Small amounts of loss can therefore significantly reduce TCP throughput, especially on high-RTT paths.

Duplicate ACK and RTO Have Different Costs

TCP commonly discovers loss through two paths.

The first is duplicate ACKs. The receiver keeps acknowledging the same sequence number, which suggests that later data arrived but a middle segment is missing. The sender can retransmit the missing segment relatively quickly and enter recovery.

The second is RTO. The sender waits until the retransmission timeout fires. This is usually more expensive because the sender did not receive useful feedback for a while and must recover more conservatively.

In engineering terms, RTO is usually more damaging to throughput and latency than ordinary fast retransmit.

Dup ACK / SACK -> faster detection of localized loss
RTO            -> feedback stalled, slower recovery

A capture with a few fast retransmissions is very different from one with repeated RTOs. The latter is more likely to create visible application stalls.

Why High-Bandwidth, High-RTT Paths Do Not Fill Easily

TCP throughput depends on bandwidth, but not only on bandwidth.

To fill a path, enough data must be in flight. Roughly, bandwidth multiplied by RTT tells you how much data the pipe needs to hold. This is often called the bandwidth-delay product.

bandwidth-delay product = bandwidth * RTT

If the path has high bandwidth and high RTT, the sender needs a large enough window to keep the pipe full. If the window is too small, the sender waits for ACKs and the link sits partially empty.

This explains several symptoms:

cross-region transfers have high bandwidth but one TCP connection does not fill it
satellite, cellular, and international paths show unstable throughput
small windows, loss, or receiver limits have larger impact
multiple download connections may appear faster than one

So throughput debugging cannot stop at outbound bandwidth. RTT, windows, loss, and congestion algorithm behavior all matter.

Bufferbloat Makes Latency Bad Before Loss

Congestion does not always show up as immediate packet loss. Some devices have large queues and buffer packets first.

As queues grow, packets may not drop, but RTT rises significantly. The application sees slow requests, poor interactivity, MQTT heartbeat jitter, or delayed video control.

This is commonly called bufferbloat. The problem is not “no buffering.” The problem is queues so deep that congestion signals arrive too late. By the time loss appears, latency is already bad.

For TCP, queue delay and slower ACK return change the sending rhythm. For real-time behavior, queueing delay alone can be enough to hurt the experience even without massive loss.

How to Think About CUBIC and BBR

Engineering discussions often mention congestion control algorithms such as CUBIC and BBR.

Do not start with formulas. Start with their different instincts:

CUBIC-like algorithms mainly adjust window growth using loss feedback
BBR-like algorithms try to estimate bottleneck bandwidth and minimum RTT, then control in-flight data using that model

This means they can behave differently on different paths. Loss-based links, deep queues, wireless links, data-center links, and high-RTT links provide different feedback, so algorithm reactions differ.

But the algorithm name is not a magic answer. Real behavior also depends on:

kernel version and implementation details
queue management
whether pacing works well
receiver and middleboxes
wireless link-layer retransmission
fairness among multiple connections

For embedded and IoT debugging, looking at RTT, retransmissions, RTOs, windows, and queues is usually more useful than arguing algorithm names first.

Slow TCP on Weak Networks Is Not Always a Slow Server

IoT devices often use Wi-Fi, cellular, Ethernet-to-wireless paths, VPNs, NATs, proxies, and public cloud paths. TCP congestion control can turn lower-layer quality changes into application-level symptoms.

Common symptoms include:

MQTT publish occasionally stalls
HTTPS request sometimes times out
OTA download rate jumps up and down
the same device is fast on LAN but slow through the public Internet
cellular signal looks acceptable, but RTT and retransmissions fluctuate
Wi-Fi link-layer retransmission already happened before TCP sees delay and loss

Do not inspect only application logs. A slow request may come from link-layer retries, queueing, congestion-window reduction, and RTO below the application.

What to Look At in Captures

For congestion-control-related TCP issues, inspect in this order.

First, check whether RTT is stable. Gradually rising RTT usually suggests queueing.

Second, classify retransmissions. Fast retransmit, duplicate ACKs, SACK, and RTO do not mean the same thing.

Third, inspect windows. A small receive window points to receiver pressure. Congestion-window reduction points to the sender becoming conservative about the path. Captures may not directly show cwnd, but in-flight data, ACK rhythm, and send gaps provide clues.

Fourth, check for Zero Window. That is usually not network congestion; it is receiver or application consumption pressure.

Fifth, inspect lower-layer signals. Wireless retransmission, rate fallback, cellular SINR fluctuation, and queue buildup can all appear later as slow TCP.

Sixth, consider request size. Short requests are more affected by handshake, slow start, and RTT. Long-lived bulk flows expose windows, loss, and algorithm behavior more clearly.

What to Remember

TCP congestion control answers “how much in-flight data can the network path carry,” not “how much can the receiver accept.” Receive window rwnd is about the receiver. Congestion window cwnd is about the path.

Loss, duplicate ACKs, RTO, rising RTT, and queue buildup all change how the sender estimates path capacity. After loss, the cost is not just one retransmission. It also includes congestion-window reduction, in-order delivery waiting, and capacity probing.

When debugging slow TCP, do not look only at bandwidth and server time. Put RTT, retransmissions, RTOs, windows, queues, and lower-layer link quality together. Many “occasionally slow application” problems belong at this layer first.