Why Your IPsec Tunnel Keeps Dropping at 2 AM
IPsec is the backbone of most site-to-site VPN deployments and remote access setups. It's not sexy. It works reliably enough that nobody thinks about it until it breaks. I've spent years debugging tunnels that mysteriously die, peers that randomly rekey into nowhere, and firewalls that silently drop traffic that should have been allowed. Here's what actually goes wrong. NAT-traversal was supposed to solve the problem of IPsec traversing NAT devices, but it's basically a hack layer built on UDP encapsulation. The real issue is that not every piece of middleware between two sites properly forwards UDP. You'll hit a corporate firewall or ISP middlebox that sees UDP port 4500 and decides to do something with it. Your tunnel drops. Recovery depends on whether the NAT device sends a keepalive, which many don't. I once spent six hours tracing a tunnel that kept dropping during business hours only. The root cause was a load balancer in the path that was doing persistence table lookups on TCP but dropping UDP entirely after idle timeout. The workaround was moving IKE to port 4500 and forcing ESP through a dedicated passthrough rule on that exact device. Took three configuration changes and a ticket that should never have existed.
Key Management Doesn't Scale
Manual keying is dead and anyone still using it is running a liability. IKEv2 improved things over IKEv1, but the fundamental problem remains: every peer pair needs valid certificates or pre-shared keys, and both sides need to agree on the proposal parameters exactly. The moment one device in a fleet gets a certificate that doesn't match the other's trust anchor, the phase one negotiation fails silently. No error message, just no tunnel. The common pitfall is assuming that if your certificates are valid, everything will work. They need to match the peer identity type you configured. If you set ID_RFC822_NAME on one end and ID_FQDN on the other, they'll reject each other regardless of certificate validity. I've seen this in production when someone migrates a server from its old hostname to a new domain and updates the certificate but forgets to update the VPN peer config on the opposite side.
Performance Overhead Is Real
ESP in tunnel mode adds headers. Not much, maybe 50 to 60 bytes per packet depending on your cipher and authentication algorithm. That sounds negligible until you're pushing multi-gigabit throughput across a long fiber link with AES-GCM or AES-CBC plus HMAC-SHA256. You'll lose roughly 10 to 15 percent of available bandwidth to encryption overhead and the additional round trips required for IKE rekeying. On constrained links, that margin disappears fast. Hardware offloading solves this on enterprise routers, but on commodity hardware or virtualized instances, you're at the mercy of CPU cycles. A dual-Xeon box with QuickAssist can handle gigabit+ IPsec easily. A standard VM without crypto acceleration will struggle past 200 Mbps before CPU becomes the bottleneck. This isn't theoretical. I deployed a pair of cloud-based IPsec gateways on standard compute instances and had to downgrade to 100 Mbps effective throughput per tunnel to avoid saturation.
Get the Full Details
Fragmentation Kills Throughput
When encrypted packets exceed the path MTU, fragmentation becomes a serious problem. Many middleboxes drop fragmented IP packets outright. Some do because of security policy. Others do because the fragment reassembly state eats into limited buffers. The result is TCP sessions stalling, intermittent connectivity, and no obvious error to point at. The fix is PMTUD with DF bit set, but IPsec complicates this because the encrypted packet's effective MTU is smaller than the outer packet. You need to account for the IPsec header size when calculating the adjusted MTU. If your tunnel MTU is set incorrectly, you'll get blackhole routing for anything larger than the actual negotiated MSS. I tuned tunnel MTU down to 1350 bytes on a problematic link and saw instant improvement in sustained transfer speeds. It cost about 3 percent overhead but eliminated the fragmentation drops entirely.
Stateful Tracking Creates Single Points of Failure
IPsec tunnels maintain state. Session tables, SA counters, rekey timers. When one gateway in an HA pair fails over, the new active node may not have the full Security Association state unless you've configured SA persistence or stateful failover correctly. Standard VRRP or HSRP don't sync IPsec SAs by default. The standby unit comes online with empty SA tables and traffic resets while it renegotiates from scratch. This means brief but real outages during failover. Two to five seconds typically, longer under load. I designed a deployment where we accepted this and added UDP flood protection on the downstream device to absorb the burst of rekeying traffic. The alternative is using a vendor solution with proprietary SA replication, which removes the gap but locks you into that vendor's hardware.
Firewall Rule Complexity Grows Exponentially
Every new tunnel means new ACLs, new permit rules, new zones. In small environments this is manageable. In enterprise networks with dozens of tunnels connecting branches, the ACL matrix becomes unwieldy. You need SPD (Security Policy Database) entries on both ends for every destination pair, and any misconfiguration leaves either the phase one or phase two negotiation hanging. The debug logs help, but reading Phase 1 and Phase 2 output line by line is tedious and error-prone when you're juggling twelve tunnels simultaneously. The workaround is grouping your policies by topology rather than by individual tunnel. Use address groups for entire subnets instead of host-specific rules. It reduces configuration from hundreds of lines to roughly a dozen and makes audits tractable. I reduced a 400-line IPsec config on a Juniper to about 45 lines using this approach and cut our change review time from an hour to fifteen minutes.
It Doesn't Solve Everything
IPsec encrypts the transport, not the application. If your application does its own DNS resolution through a resolver that leaks, IPsec doesn't protect that. Same with application-layer protocol leaks. It also doesn't provide authentication of the endpoint beyond the IKE peer. You can have a perfectly valid IPsec tunnel to a device that's running compromised software. For zero-trust architectures, IPsec alone is insufficient. WireGuard offers a cleaner model for some of these concerns, though it has its own tradeoffs around enterprise key management and auditability. For environments where you need granular identity-based policies, per-user encryption, and easy certificate rotation at scale, consider supplementing IPsec with a proper ZTNA solution rather than trying to make IPsec do work it wasn't designed for.
