Connection marks
For forwarded traffic, the router has to remember which ISP each connection arrived on, and route every reply packet accordingly. conntrack already follows each connection; nftables can attach a number to it, the connection mark (ct mark).
The extra file /etc/nftables.d/dualwan.nft, loaded by the main file's include:
- When a new connection arrives on ISP 1's interface, set its ct mark to 1; on ISP 2's, to 2. Connections started in the office stay at 0.
- On every packet, copy the connection's mark to the packet's own mark (
meta mark), which is whatip rule fwmarkmatches.
The chain runs on the prerouting hook at priority mangle (−150), before the routing decision, so the reply from the office server is already marked 2 when the kernel chooses its route: the fwmark rule sends it to ISP 2's table. The rule at preference 900 comes first, so incoming packets for the office LAN still go straight there, marked or not.
Both port forwards now work, and conntrack shows the mark on each connection: 1 for the one that came in through ISP 1, 2 for ISP 2. The attempt from before the marks existed is still listed, half-open, with mark 0.
Marks are just numbers; pick values that don't clash with other software using them (WireGuard's wg-quick, for example, uses 51820).
# /etc/nftables.d/dualwan.nft: remember which ISP a connection came in on,
# so its replies can be routed back out the same way (ip rule fwmark 1 / 2).
table inet dualwan {
chain prerouting {
type filter hook prerouting priority mangle;
ct state new iifname "eth0" ct mark set 1
ct state new iifname "eth1" ct mark set 2
meta mark set ct mark comment "copy the connection's mark to every packet, for routing"
}
}- Line 5: Prerouting at mangle priority: before routing, and before NAT.
- Lines 6–7: Label each new incoming connection with its ISP.
- Line 8: Copy the label to every packet of the connection, both directions, for ip rule fwmark.
root@gw:~# cp dualwan.nft /etc/nftables.d/ && nft -c -f /etc/nftables.conf && systemctl reload nftables && nft list table inet dualwan
table inet dualwan {
chain prerouting {
type filter hook prerouting priority mangle; policy accept;
ct state new iifname "eth0" ct mark set 0x00000001
ct state new iifname "eth1" ct mark set 0x00000002
meta mark set ct mark comment "copy the connection's mark to every packet, for routing"
}
}
root@web:~# curl -s -o /dev/null -w "%{http_code}\n" -m 3 http://198.51.100.2:8080/; curl -s -o /dev/null -w "%{http_code}\n" -m 3 http://203.0.113.2:8080/
200
200
root@pc:~# tail -2 /tmp/web.log
192.0.2.80 - - [21/Oct/2026 08:07:18] "GET / HTTP/1.1" 200 -
192.0.2.80 - - [21/Oct/2026 08:07:18] "GET / HTTP/1.1" 200 -
root@gw:~# conntrack -L -p tcp --orig-port-dst 8080 2>/dev/null
tcp 6 119 TIME_WAIT src=192.0.2.80 dst=198.51.100.2 sport=41904 dport=8080 src=10.20.0.10 dst=192.0.2.80 sport=80 dport=41904 [ASSURED] mark=1 use=1
tcp 6 6 SYN_RECV src=192.0.2.80 dst=203.0.113.2 sport=56060 dport=8080 src=10.20.0.10 dst=192.0.2.80 sport=80 dport=56060 mark=0 use=1
tcp 6 119 TIME_WAIT src=192.0.2.80 dst=203.0.113.2 sport=52472 dport=8080 src=10.20.0.10 dst=192.0.2.80 sport=80 dport=52472 [ASSURED] mark=2 use=1- Lines 1–9: Reload: the main file includes the new one.
- Lines 11–13: Both port forwards work.
- Lines 15–17: Two visits, one through each ISP.
- Line 20: In through ISP 1: mark 1.
- Line 21: The failed attempt from before the marks existed: mark 0, never completed.
- Line 22: In through ISP 2: mark 2, and its replies went back through ISP 2.
Check your understanding
Why can't a from 203.0.113.2 rule route the replies of a port forward through ISP 2?
- The office server's replies come from its LAN address; the router only knows which ISP to use from the connection's mark.
- Because DNAT removes the source address.
- Because ip rule can't match forwarded traffic.
- It can; the marks are just faster.