IP forwarding performance
sepherosa at gmail.com
Fri Dec 14 01:47:27 PST 2012
This email serves as the base performance measurement for further
network stack optimization (as of git 107282b).
mobo ASUS P867H-M
4x4G DDR3 memory
CPU i7-2600 (w/ HT and Turbo Boost enabled, 4C/8T)
Forwarding NIC Intel 82576EB dual copper
Packet generator NICs Intel 82571EB dual copper
A emx1 <---> igb0 forwarder igb1 <---> emx1 B
A and "forwarder", B and "forwarder" are directly connected using CAT6 cables.
Polling(4) is enabled on igb1 and igb0 on "forwarder". Following
tunables are in /boot/loader.conf:
Following sysctl is changed before putting igb1 into polling mode:
First for the users that are only interested in the bulk forwarding
performance: The 32 netperf TCP_STREAMs running on A could do
Now the tiny packets forwarding performance:
A and B generate 18 bytes UDP datagrams using
tools/tools/netrate/pktgen. The destination addresses of the UDP
datagrams are selected that the generated UDP datagrams will be evenly
distributed the to the 8 RX queues, which should be common in the
Bidirectional normal IP forwarding:
1.42Mpps in each direction, so total 2.84Mpps are forwarded.
On CPUs that are doing TX in addition to RX: 85% ~ 90% (max allowed by
On CPUs that are only doing RX: 40% ~ 50%
Bidirectional fast IP forwarding: (net.inet.ip.fastforwarding=1)
1.48Mpps in each direction, so total 2.96Mpps are forwarded.
On CPUs that are doing TX in addition to RX: 65% ~ 70%
On CPUs that are doing RX: 30% ~ 40%
Tomorrow Will Never Die
More information about the Users