git: udp: Dispatch UDP datagrams to the correct netisr to perform ip_output()
sephe at crater.dragonflybsd.org
Sat Jun 7 06:23:43 PDT 2014
Author: Sepherosa Ziehau <sephe at dragonflybsd.org>
Date: Sat Jun 7 21:17:08 2014 +0800
udp: Dispatch UDP datagrams to the correct netisr to perform ip_output()
Redispatch UDP datagrams to the netisr, whose owner CPU matches the
UDP datagrams hash, to do the ip_output(). As measured bt KTR, the
udp_send() spends most of its time in ip_output().
To properly support this, following stuffs are added/changed:
- Add a network private lwkt message flag to indicate the so_port of
the netmsg should not be matched against the current netisr's msgport.
This flag is set when we redispatch the UDP datagrams to other netisrs.
- Don't use priority messages for netisr barrier, so at UDP socket close
time, UDP datagrams pending on other netisr msgport could be properly
The UDP datagrams redispatch itself:
- If IP options are ever configured (supposely none), a copy of it will
be carried along w/ the UDP datagram to the target netisr. The copy
is made mainly because it is not safe to access the IP options of the
inpcb in netisr which does not own the inpcb.
(*) On the other hand accessing inpcb's multicast options is safe since
multicast UDP datagrams output and multicast options configuration all
happen in netisr0.
- Add nm_priv into netmsg_pru_send, which saves flags to be passed to
ip_output() for the UDP datagram. This does not changes the size of
netmsg_pru_send on x86_64 due to the implicit 4bytes padding.
- udp_addrcpu_pkt() is added to calculate the "real" CPU for the UDP
- Don't use inpcb route cache for redispatched UDP datagrams. Since:
o The cached route is usually not for the UDP datagrams' destination.
o Accessing inpcb route cache in the netisr, which is not the owner
of the inpcb is not safe.
On i7-3770 w/ 82599ES, this increases 18bytes UDP request/response
performance by ~19% (1.12M trasactions/s ---> 1.34M transactions/s)
This commit also makes lockless firewall state table doable, since input
and output of UDP datagrams, which have same hash, are running in the
same netisr now!
Summary of changes:
sys/net/netisr.c | 17 ++++++--
sys/net/netmsg.h | 3 ++
sys/netinet/ip_demux.c | 11 +++++
sys/netinet/udp_usrreq.c | 109 ++++++++++++++++++++++++++++++++++++++++++++---
sys/netinet/udp_var.h | 2 +
5 files changed, 132 insertions(+), 10 deletions(-)
DragonFly BSD source repository
More information about the Commits