Intermediate · Packets That Never Come Back · section 5 of 7

Scenario 2: asymmetric routing

✓ Lab-tested 17 October 2026 on VyOS 2026.10.16-0024-rolling, Debian 12

Symptom: with NAT fixed, someone adds a route on the client sending traffic for the server via rtr2 ("it's a shorter path"). Now TCP connections to the server time out. R1 still works for everything else.

Investigation:

ip route get on the client shows the server is now reached via 10.0.0.2 (rtr2).
On the server, tcpdump -e shows SYNs arriving from rtr2's MAC, and SYN-ACKs leaving toward R1's MAC (52:54:00:00:01:01). The two directions take different paths.
On R1, the SYN-ACKs arrive on eth1, and nothing leaves eth0.
R1's firewall counters show why: forwarded packets with conntrack state invalid are dropped.

Why: a stateful firewall only accepts replies to connections it saw being set up. R1 never saw the SYN (it went via rtr2), so the SYN-ACK matches no connection and is invalid. The same happens with any stateful device that sees only one direction of a flow: firewalls, NAT gateways, load balancers. It also happens with reverse-path filtering (VyOS source-validation, Linux rp_filter), which drops packets arriving on an interface that isn't the best route back to their source.

Fix: make the path symmetric, here by removing the client's special route. If two paths must exist, keep state in sync (firewall clusters) or put the stateful device where both directions pass through it.

Diagram: Scenario 2: asymmetric path — The SYN goes one way, the SYN-ACK comes back another, and the firewall on the return path has never seen the connection.

The SYN goes via rtr2. The client has a more specific route to the server via 10.0.0.2. rtr2 just forwards; it has no firewall.
The SYN-ACK comes back via R1. The server's default gateway is R1, so its reply to 10.0.0.10 goes there instead.
R1 has never seen this connection. R1's conntrack has no SYN for this flow, so the SYN-ACK is ct state invalid, and the stateful firewall drops it. Nothing leaves eth0 toward the client.
Make the path symmetric. Remove the client's special route so both directions go through R1. Stateful devices must see both directions of every flow.

Symptom on the client (real output)

root@client:~# ip route get 203.0.113.20
203.0.113.20 via 10.0.0.2 dev eth0 src 10.0.0.10 uid 0
    cache
root@client:~# nc -vz -w 3 203.0.113.20 80
nc: connect to 203.0.113.20 port 80 (tcp) timed out: Operation now in progress

Lines 1–3: The new, more specific host route sends traffic for the server via rtr2.
Lines 4–5: A timeout, not a refusal: something is silently dropping.

On the server, with -e (real output)

root@server:~# timeout 4 tcpdump -l -nn -e -i eth0 tcp port 40000
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
21:38:38.232691 4a:5a:d9:0f:75:fe > 3a:bf:71:a0:af:0a, ethertype IPv4 (0x0800), length 74: 10.0.0.10.40000 > 203.0.113.20.80: Flags [S], seq 2317000030, win 64240, options [mss 1460,sackOK,TS val 1192142810 ecr 0,nop,wscale 10], length 0
21:38:38.232703 3a:bf:71:a0:af:0a > 52:54:00:00:01:01, ethertype IPv4 (0x0800), length 74: 203.0.113.20.80 > 10.0.0.10.40000: Flags [S.], seq 1185013057, ack 2317000031, win 65160, options [mss 1460,sackOK,TS val 2911596986 ecr 1192142810,nop,wscale 10], length 0
21:38:39.262199 3a:bf:71:a0:af:0a > 52:54:00:00:01:01, ethertype IPv4 (0x0800), length 74: 203.0.113.20.80 > 10.0.0.10.40000: Flags [S.], seq 1185013057, ack 2317000031, win 65160, options [mss 1460,sackOK,TS val 2911598015 ecr 1192142810,nop,wscale 10], length 0
21:38:39.262616 4a:5a:d9:0f:75:fe > 3a:bf:71:a0:af:0a, ethertype IPv4 (0x0800), length 74: 10.0.0.10.40000 > 203.0.113.20.80: Flags [S], seq 2317000030, win 64240, options [mss 1460,sackOK,TS val 1192143839 ecr 0,nop,wscale 10], length 0
21:38:39.262645 3a:bf:71:a0:af:0a > 52:54:00:00:01:01, ethertype IPv4 (0x0800), length 74: 203.0.113.20.80 > 10.0.0.10.40000: Flags [S.], seq 1185013057, ack 2317000031, win 65160, options [mss 1460,sackOK,TS val 2911598016 ecr 1192142810,nop,wscale 10], length 0
21:38:40.288731 4a:5a:d9:0f:75:fe > 3a:bf:71:a0:af:0a, ethertype IPv4 (0x0800), length 74: 10.0.0.10.40000 > 203.0.113.20.80: Flags [S], seq 2317000030, win 64240, options [mss 1460,sackOK,TS val 1192144865 ecr 0,nop,wscale 10], length 0
21:38:40.288784 3a:bf:71:a0:af:0a > 52:54:00:00:01:01, ethertype IPv4 (0x0800), length 74: 203.0.113.20.80 > 10.0.0.10.40000: Flags [S.], seq 1185013057, ack 2317000031, win 65160, options [mss 1460,sackOK,TS val 2911599042 ecr 1192142810,nop,wscale 10], length 0

7 packets captured
7 packets received by filter
0 packets dropped by kernel

Line 1: The client used a pinned source port (nc -p 40000), so the filter shows only this test flow.
Line 4: The SYN arrives from 4a:5a:d9:0f:75:fe, which is rtr2's eth1.
Line 5: The SYN-ACK leaves toward 52:54:00:00:01:01: R1's eth1. The return path is different from the forward path.
Lines 6–10: Both sides retransmit: the client never got the SYN-ACK, so it resends the SYN; the server resends its SYN-ACK.

On R1: arrives on WAN… (real output)

vyos@R1:~$ monitor traffic interface eth1 filter "tcp port 40001"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
21:21:56.972549 IP 203.0.113.20.http > 10.0.0.10.40001: Flags [S.], seq 1314620064, ack 1675870629, win 65160, options [mss 1460,sackOK,TS val 1994872165 ecr 205116854,nop,wscale 10], length 0
21:21:57.979486 IP 203.0.113.20.http > 10.0.0.10.40001: Flags [S.], seq 1314620064, ack 1675870629, win 65160, options [mss 1460,sackOK,TS val 1994873172 ecr 205116854,nop,wscale 10], length 0
21:21:57.979493 IP 203.0.113.20.http > 10.0.0.10.40001: Flags [S.], seq 1314620064, ack 1675870629, win 65160, options [mss 1460,sackOK,TS val 1994873172 ecr 205116854,nop,wscale 10], length 0
21:21:59.001622 IP 203.0.113.20.http > 10.0.0.10.40001: Flags [S.], seq 1314620064, ack 1675870629, win 65160, options [mss 1460,sackOK,TS val 1994874194 ecr 205116854,nop,wscale 10], length 0
21:22:01.051369 IP 203.0.113.20.http > 10.0.0.10.40001: Flags [S.], seq 1314620064, ack 1675870629, win 65160, options [mss 1460,sackOK,TS val 1994876244 ecr 205116854,nop,wscale 10], length 0
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel

Lines 4–8: The SYN-ACKs reach R1. VyOS's monitor resolves port names, hence .http.

…never leaves on LAN (real output)

vyos@R1:~$ monitor traffic interface eth0 filter "tcp port 40002"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

Lines 4–7: Nothing toward the client. R1 is dropping them.

Which rule? (real output)

vyos@R1:~$ show firewall ipv4 forward filter
Ruleset Information

---------------------------------
ipv4 Firewall "forward filter"

Rule     Action    Protocol      Packets    Bytes  Conditions
-------  --------  ----------  ---------  -------  ------------------------------------
10       accept    all                 2      144  ct state new iifname "eth0"  accept
default  drop      all                 0        0  comment "FWD-filter default-action "

vyos@R1:~$ sudo nft list chain ip vyos_filter VYOS_STATE_POLICY_FORWARD
table ip vyos_filter {
	chain VYOS_STATE_POLICY_FORWARD {
		ct state established counter packets 7 bytes 404 accept
		ct state invalid counter packets 63 bytes 3780 drop
		ct state related counter packets 0 bytes 0 accept
		return
	}
}

Lines 9–10: The forward filter's own rules show no drops at all: the default-action counter is 0.
Line 12: Forwarded traffic first passes through this state-policy chain, created from firewall global-options state-policy.
Line 16: There they are: 63 packets dropped as ct state invalid. These are the SYN-ACKs (plus retransmissions) for connections R1 never saw start.

Fix: make the path symmetric (real output)

root@client:~# ip route del 203.0.113.20/32 via 10.0.0.2
root@client:~# nc -vz -w 3 203.0.113.20 80
Connection to 203.0.113.20 80 port [tcp/http] succeeded!

Line 1: Remove the shortcut. Both directions now go through R1, so its conntrack sees the whole connection.
Line 3: Works immediately.

Check your understanding

A server's tcpdump -e shows requests arriving from router A's MAC and replies leaving toward router B's MAC. Only router B has a stateful firewall. What happens?

Everything works, because both routers forward
Router B drops the replies as invalid, so connections time out
Router A drops the requests
The server rejects the connection with RST