TIME_WAIT sockets in Debian GNU/Linux

Applications that handle a huge number of TCP connections, either as a server, or as a client, will often see a large number of sockets in TIME_WAIT state. The sockets in TIME_WAIT can be seen with the 'netstat -t' command. If the number of TIME_WAIT sockets gets too large, your address space will be exhausted, causing a disruption of TCP traffic. This page describes the TIME_WAIT issue in a Linux context. For a generic source of information, see this FAQ entry on UNIX socket programming.

Only one of the two partners communicating with TCP will get the TIME_WAIT status. This happens to the side that initiated the closing of the connection.

GNU/Linux seems to be lacking a proper configuration option for the TIME_WAIT status (like BSD offers). For linux, it is hard-coded to 60 seconds in the kernel in include/net/tcp.h as:

#define TCP_TIMEWAIT_LEN (60*HZ)
Do not be confused by the /proc/sys/net/ipv4/tcp_fin_timeout config item. The FIN TIMEOUT is not the same as the TIMEWAIT length.

Also, I find the /proc/sys/net/ipv4/tcp_max_tw_buckets confusing. Although the default is set at 180000, I see a TCP disruption when I have 32K TIME_WAIT sockets on my system, regardless of the max tw buckets.

You can force the kernel to reuse the TIME_WAIT buckets though, by setting the /proc/sys/net/ipv4/tcp_tw_recycle entry to 1. I have tested this, and it works: very few TIME_WAIT entries will be present when using this setting. It could be however, that using this could affect the reliability of TCP connections. I believe problems may arrise in NAT environments.

An application like a client of the linuxtuples system, easily exhausts the available socket address space. If this happens, I see a 'connect failed: Cannot assign requested address'.


Visit the sponsor: the little tank that could