- dev - passt

[PATCH 0/8] Improve flow table based dispatch of timers and other handlers
by David Gibson 14 Dec '23

14 Dec '23

There are still some thing dispatched explicitly in tcp.c and tcp_splice.c that make more sense to dispatch at the flow level, even with the current "stub" version of the flow table. This should be independent of my other outstanding series. David Gibson (8): flow: Make flow_table.h #include the protocol specific headers it needs treewide: Standardise on 'now' for current timestamp variables tcp, tcp_splice: Remove redundant handling from tcp_timer() tcp, tcp_splice: Move per-type cleanup logic into per-type helpers flow, tcp: Add flow-centric dispatch for deferred flow handling flow, tcp: Add handling for per-flow timers epoll: Better handling of number of epoll types tcp, tcp_splice: Avoid double layered dispatch for connected TCP sockets flow.c | 33 +++++++++++++++++++- flow.h | 3 ++ flow_table.h | 2 ++ icmp.c | 12 ++++---- icmp.h | 2 +- log.c | 34 ++++++++++----------- passt.c | 18 ++++++++--- passt.h | 8 +++-- tcp.c | 86 +++++++++++----------------------------------------- tcp.h | 2 +- tcp_conn.h | 5 +-- tcp_splice.c | 33 ++++++++++---------- tcp_splice.h | 4 +-- udp.c | 16 +++++----- udp.h | 2 +- 15 files changed, 129 insertions(+), 131 deletions(-) -- 2.43.0

1 8

test: make passt.mbuto script more robust
by Jon Maloy 12 Dec '23

12 Dec '23

From: Jon Paul Maloy <jmaloy(a)redhat.com> Creation of a symbolic link from /sbin to /usr/sbin fails if /sbin exists and is non-empty. This is the case on Ubuntu-23.04. We fix this by removing /sbin before creating the link. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- test/passt.mbuto | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test/passt.mbuto b/test/passt.mbuto index 90816d2..12d7f50 100755 --- a/test/passt.mbuto +++ b/test/passt.mbuto @@ -26,6 +26,8 @@ DIRS="${DIRS} /tmp /sbin /usr/share /var/log /var/lib /etc/ssh /run/sshd /root/. COPIES="${COPIES} small.bin,/root/small.bin medium.bin,/root/medium.bin big.bin,/root/big.bin" FIXUP="${FIXUP}"' + rm -rf /sbin + ln -s /usr/sbin /sbin cat > /sbin/dhclient-script << EOF #!/bin/sh LOG=/var/log/dhclient-script.log @@ -47,7 +49,6 @@ set >> \$LOG exit 0 EOF chmod 755 /sbin/dhclient-script - ln -s /sbin /usr/sbin ln -s /bin /usr/bin ln -s /run /var/run :> /etc/fstab -- 2.39.2

2 3

[PATCH] test: Select first reported IPv6 address for guest/host comparison
by Stefano Brivio 11 Dec '23

11 Dec '23

If we run passt nested (a guest connected via passt to a guest connected via passt to the host), the first guest (L1) typically has two IPv6 addresses: one formed from the prefix assigned via SLAAC, and another one assigned via DHCPv6 (to match the address on the host). When we select addresses for comparison, in this case, we have multiple global unicast addresses. Selecting the first reported one on both host and guest is not entirely correct (in theory, the order might differ), but works reasonably well. Use the trick from 5beef085978e ("test: Only select a single interface or gateway in tests") to ask jq(1) for the first address returned by the query. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/passt/dhcp | 8 ++++---- test/passt/ndp | 4 ++-- test/pasta/dhcp | 8 ++++---- test/pasta/ndp | 4 ++-- test/two_guests/basic | 6 +++--- 5 files changed, 15 insertions(+), 15 deletions(-) diff --git a/test/passt/dhcp b/test/passt/dhcp index 7272755..b428064 100644 --- a/test/passt/dhcp +++ b/test/passt/dhcp @@ -22,8 +22,8 @@ check [ -n "__IFNAME__" ] test DHCP: address guest /sbin/dhclient -4 __IFNAME__ -gout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' -hout HOST_ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__HOST_IFNAME__").addr_info[0].local' +gout ADDR ip -j -4 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[0].local] | .[0]' +hout HOST_ADDR ip -j -4 addr show|jq -rM '[.[] | select(.ifname == "__HOST_IFNAME__").addr_info[0].local] | .[0]' check [ "__ADDR__" = "__HOST_ADDR__" ] test DHCP: route @@ -49,8 +49,8 @@ check [ "__SEARCH__" = "__HOST_SEARCH__" ] test DHCPv6: address guest /sbin/dhclient -6 __IFNAME__ -gout ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.prefixlen == 128).local' -hout HOST_ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local' +gout ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.prefixlen == 128).local] | .[0]' +hout HOST_ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local] | .[0]' check [ "__ADDR6__" = "__HOST_ADDR6__" ] test DHCPv6: route diff --git a/test/passt/ndp b/test/passt/ndp index 6de4081..7b2dbfe 100644 --- a/test/passt/ndp +++ b/test/passt/ndp @@ -21,9 +21,9 @@ hout HOST_IFNAME6 ip -j -6 route show|jq -rM '[.[] | select(.dst == "default").d check [ -n "__IFNAME__" ] test SLAAC: prefix -gout ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global" and .prefixlen == 64).local' +gout ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global" and .prefixlen == 64).local] | .[0]' gout PREFIX6 sipcalc __ADDR6__/64 | grep prefix | cut -d' ' -f4 -hout HOST_ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local' +hout HOST_ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local] | .[0]' hout HOST_PREFIX6 sipcalc __HOST_ADDR6__/64 | grep prefix | cut -d' ' -f4 check [ "__PREFIX6__" = "__HOST_PREFIX6__" ] diff --git a/test/pasta/dhcp b/test/pasta/dhcp index 309001b..b4f6f60 100644 --- a/test/pasta/dhcp +++ b/test/pasta/dhcp @@ -20,8 +20,8 @@ check [ -n "__IFNAME__" ] test DHCP: address ns /sbin/dhclient -4 --no-pid __IFNAME__ -nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' -hout HOST_ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' +nsout ADDR ip -j -4 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[0].local] | .[0]' +hout HOST_ADDR ip -j -4 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[0].local] | .[0]' check [ __ADDR__ = __HOST_ADDR__ ] test DHCP: route @@ -36,8 +36,8 @@ check [ __MTU__ = 65520 ] test DHCPv6: address ns /sbin/dhclient -6 --no-pid __IFNAME__ hout HOST_IFNAME6 ip -j -6 route show|jq -rM '[.[] | select(.dst == "default").dev] | .[0]' -nsout ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.prefixlen == 128).local' -hout HOST_ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local' +nsout ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.prefixlen == 128).local] | .[0]' +hout HOST_ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local] | .[0]' check [ __ADDR6__ = __HOST_ADDR6__ ] test DHCPv6: route diff --git a/test/pasta/ndp b/test/pasta/ndp index bb33110..2a8afe6 100644 --- a/test/pasta/ndp +++ b/test/pasta/ndp @@ -21,9 +21,9 @@ ns ip link set dev __IFNAME__ up sleep 2 test SLAAC: prefix -nsout ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global" and .prefixlen == 64).local' +nsout ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global" and .prefixlen == 64).local] | .[0]' nsout PREFIX6 sipcalc __ADDR6__/64 | grep prefix | cut -d' ' -f4 -hout HOST_ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global").local' +hout HOST_ADDR6 ip -j -6 addr show|jq -rM ['.[] | select(.ifname == "__IFNAME__").addr_info[] | select(.scope == "global").local] | .[0]' hout HOST_PREFIX6 sipcalc __HOST_ADDR6__/64 | grep prefix | cut -d' ' -f4 check [ "__PREFIX6__" = "__HOST_PREFIX6__" ] diff --git a/test/two_guests/basic b/test/two_guests/basic index 09fbd3e..fa0608b 100644 --- a/test/two_guests/basic +++ b/test/two_guests/basic @@ -39,9 +39,9 @@ test DHCPv6: addresses sleep 2 guest1 /sbin/dhclient -6 __IFNAME1__ guest2 /sbin/dhclient -6 __IFNAME2__ -g1out ADDR1_6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME1__").addr_info[] | select(.prefixlen == 128).local' -g2out ADDR2_6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME2__").addr_info[] | select(.prefixlen == 128).local' -hout HOST_ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local' +g1out ADDR1_6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME1__").addr_info[] | select(.prefixlen == 128).local] | .[0]' +g2out ADDR2_6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__IFNAME2__").addr_info[] | select(.prefixlen == 128).local] | .[0]' +hout HOST_ADDR6 ip -j -6 addr show|jq -rM '[.[] | select(.ifname == "__HOST_IFNAME6__").addr_info[] | select(.scope == "global").local] | .[0]' check [ "__ADDR1_6__" = "__HOST_ADDR6__" ] check [ "__ADDR2_6__" = "__HOST_ADDR6__" ] -- 2.39.2

2 3

[PATCH 0/3] RFC: TCP hash change changes, in preparation for flow table
by David Gibson 11 Dec '23

11 Dec '23

I now have an in-progress draft of a unified hash table to go with the unified flow table. This turns out to be easier if we first make some preliminary changes to the structure of the TCP hash table. So, here those are for review. This is based on my already posted series introducing the rudimentary version of the unified flow table. David Gibson (3): tcp: Switch hash table to linear probing instead of chaining tcp: Implement hash table with indices rather than pointers tcp: Don't account for hash table size in tcp_hash() flow.h | 11 ++++ tcp.c | 144 ++++++++++++++++++++++++++++------------------------- tcp_conn.h | 2 - util.h | 13 +++++ 4 files changed, 101 insertions(+), 69 deletions(-) -- 2.43.0

2 10

[PATCH] ndp: Extend lifetime of prefix, router, RDNSS and search list
by Stefano Brivio 11 Dec '23

11 Dec '23

Currently, we have no mechanism to dynamically update IPv6 addressing, routing or DNS information (which should eventually be implemented via netlink monitor), so it makes no sense to limit lifetimes of NDP information to any particular value. If we do, with common configurations of systemd-networkd in a guest, we can end up in a situation where we have a /128 address assigned via DHCPv6, the NDP-assigned prefix expires, and the default route also expires. However, as there's a valid address, the prefix is not renewed. As a result, the default route becomes invalid and we lose it altogether, which implies that the guest loses IPv6 connectivity except for link-local communication. Set the router lifetime to the maximum allowed by RFC 8319, that is, 65535 seconds (about 18 hours). RFC 4861 limited this value to 9000 seconds, but RFC 8319 later updated this limit. Set prefix and DNS information lifetime to infinity. This is allowed by RFC 4861 and RFC 8319. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- ndp.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ndp.c b/ndp.c index adab1d2..4c85ab8 100644 --- a/ndp.c +++ b/ndp.c @@ -94,7 +94,7 @@ int ndp(struct ctx *c, const struct icmp6hdr *ih, const struct in6_addr *saddr) ihr->icmp6_type = RA; ihr->icmp6_code = 0; ihr->icmp6_hop_limit = 255; - ihr->icmp6_rt_lifetime = htons(9000); + ihr->icmp6_rt_lifetime = htons(65535); /* RFC 8319 */ ihr->icmp6_addrconf_managed = 1; p = (unsigned char *)(ihr + 1); @@ -103,9 +103,9 @@ int ndp(struct ctx *c, const struct icmp6hdr *ih, const struct in6_addr *saddr) *p++ = 4; /* length */ *p++ = 64; /* prefix length */ *p++ = 0xc0; /* prefix flags: L, A */ - *(uint32_t *)p = htonl(3600); /* lifetime */ + *(uint32_t *)p = (uint32_t)~0U; /* lifetime */ p += 4; - *(uint32_t *)p = htonl(3600); /* preferred lifetime */ + *(uint32_t *)p = (uint32_t)~0U; /* preferred lifetime */ p += 8; memcpy(p, &c->ip6.addr, 8); /* prefix */ p += 16; @@ -126,7 +126,7 @@ int ndp(struct ctx *c, const struct icmp6hdr *ih, const struct in6_addr *saddr) *p++ = 25; /* RDNSS */ *p++ = 1 + 2 * n; /* length */ p += 2; /* reserved */ - *(uint32_t *)p = htonl(60); /* lifetime */ + *(uint32_t *)p = (uint32_t)~0U; /* lifetime */ p += 4; for (i = 0; i < n; i++) { @@ -142,7 +142,7 @@ int ndp(struct ctx *c, const struct icmp6hdr *ih, const struct in6_addr *saddr) *p++ = 31; /* DNSSL */ *p++ = (dns_s_len + 8 - 1) / 8 + 1; /* length */ p += 2; /* reserved */ - *(uint32_t *)p = htonl(60); /* lifetime */ + *(uint32_t *)p = (uint32_t)~0U; /* lifetime */ p += 4; for (i = 0; i < n; i++) { -- 2.39.2

2 1

[PATCH] test: Make handling of shell prompts with escapes a little more reliable
by David Gibson 07 Dec '23

07 Dec '23

When using the old-style "pane" methods of executing commands during the tests, we need to scan the shell output for prompts in order to tell when commands have finished. This is inherently unreliable because commands could output things that look like prompts, and prompts might not look like we expect them to. The only way to really fix this is to use a better way of dispatching commands, like the newer "context" system. However, it's awkward to convert everything to "context" right at the moment, so we're still relying on some tests that do work most of the time. It is, however, particularly sensitive to fancy coloured prompts using escape sequences. Currently we try to handle this by stripping actual ESC characters with tr, then looking for some common variants. We can do a bit better: instead strip all escape sequences using sed before looking for our prompt. Or, at least, any one using [a-zA-Z] as the terminating character. Strictly speaking ANSI escapes can be terminated by any character in 0x40..0x7e, which isn't easily expressed in a regexp. This should capture all common ones, though. With this transformation we can simplify the list of patterns we then look for as a prompt, removing some redundant variants. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- test/lib/term | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/test/lib/term b/test/lib/term index aa05bf1..262937e 100755 --- a/test/lib/term +++ b/test/lib/term @@ -203,11 +203,9 @@ pane_wait() { __done=0 while - __l="$(tail -1 ${LOGDIR}/pane_${__lc}.log | tr -d [:cntrl:])" + __l="$(tail -1 ${LOGDIR}/pane_${__lc}.log | sed 's/[[][^a-zA-Z]*[a-zA-Z]//g')" case ${__l} in - '$ ' | '# ' | '# # ' | *"$ " | *"# ") return ;; - *" #[m " | *" #[m [K" | *"]# ["*) return ;; - *' $ [6n' | *' # [6n' ) return ;; + *"$ " | *"# ") return ;; esac do sleep 0.1 || sleep 1; done } -- 2.42.0

2 3

[RFC net-next v2] tcp: add support for read with offset when using MSG_PEEK
by Jon Maloy 06 Dec '23

06 Dec '23

When reading received messages with MSG_PEEK, we sometines have to read the leading bytes of the stream several times, only to reach the bytes we really want. This is clearly non-optimal. What we would want is something similar to pread/preadv(), but working even for tcp sockets. At the same time, we don't want to add any new arguments to the recv/recvmsg() calls. In this commit, we allow the user to set iovec.iov_base in the first vector entry to NULL. This tells the socket to skip the first entry, hence letting the iov_len field of that entry indicate the offset value. This way, there is no need to add any new arguments or flags. In the iperf3 logs examples shown below, we can observe a throughput improvement of ~20 % in the direction host->namespace when using the protocol splicer 'passt'. This is a consistent result. $ ./passt/passt/pasta --config-net -f MSG_PEEK with offset not supported. [root@fedora37 ~]# perf record iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 60344 [ 6] local 192.168.122.163 port 5201 connected to 192.168.122.1 port 60360 [ ID] Interval Transfer Bitrate {...] [ 6] 13.00-14.00 sec 2.54 GBytes 21.8 Gbits/sec [ 6] 14.00-15.00 sec 2.52 GBytes 21.7 Gbits/sec [ 6] 15.00-16.00 sec 2.50 GBytes 21.5 Gbits/sec [ 6] 16.00-17.00 sec 2.49 GBytes 21.4 Gbits/sec [ 6] 17.00-18.00 sec 2.51 GBytes 21.6 Gbits/sec [ 6] 18.00-19.00 sec 2.48 GBytes 21.3 Gbits/sec [ 6] 19.00-20.00 sec 2.49 GBytes 21.4 Gbits/sec [ 6] 20.00-20.04 sec 87.4 MBytes 19.2 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 6] 0.00-20.04 sec 48.9 GBytes 21.0 Gbits/sec receiver ----------------------------------------------------------- [jmaloy@fedora37 ~]$ ./passt/passt/pasta --config-net -f MSG_PEEK with offset supported. [root@fedora37 ~]# perf record iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 46362 [ 6] local 192.168.122.163 port 5201 connected to 192.168.122.1 port 46374 [ ID] Interval Transfer Bitrate [...] [ 6] 12.00-13.00 sec 3.18 GBytes 27.3 Gbits/sec [ 6] 13.00-14.00 sec 3.17 GBytes 27.3 Gbits/sec [ 6] 14.00-15.00 sec 3.13 GBytes 26.9 Gbits/sec [ 6] 15.00-16.00 sec 3.17 GBytes 27.3 Gbits/sec [ 6] 16.00-17.00 sec 3.17 GBytes 27.2 Gbits/sec [ 6] 17.00-18.00 sec 3.14 GBytes 27.0 Gbits/sec [ 6] 18.00-19.00 sec 3.17 GBytes 27.2 Gbits/sec [ 6] 19.00-20.00 sec 3.12 GBytes 26.8 Gbits/sec [ 6] 20.00-20.04 sec 119 MBytes 25.5 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 6] 0.00-20.04 sec 59.4 GBytes 25.4 Gbits/sec receiver ----------------------------------------------------------- Passt is used to support VMs in containers, such as KubeVirt, and is also generally supported in libvirt/QEMU since release 9.2 / 7.2. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- net/ipv4/tcp.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 53bcc17c91e4..e9d3b5bf2f66 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -2310,6 +2310,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, int *cmsg_flags) { struct tcp_sock *tp = tcp_sk(sk); + size_t peek_offset; int copied = 0; u32 peek_seq; u32 *seq; @@ -2353,6 +2354,20 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, if (flags & MSG_PEEK) { peek_seq = tp->copied_seq; seq = &peek_seq; + if (!msg->msg_iter.__iov[0].iov_base) { + peek_offset = msg->msg_iter.__iov[0].iov_len; + msg->msg_iter.__iov = &msg->msg_iter.__iov[1]; + if (msg->msg_iter.nr_segs <= 1) + goto out; + msg->msg_iter.nr_segs -= 1; + if (msg->msg_iter.count <= peek_offset) + goto out; + msg->msg_iter.count -= peek_offset; + if (len <= peek_offset) + goto out; + len -= peek_offset; + *seq += peek_offset; + } } target = sock_rcvlowat(sk, flags & MSG_WAITALL, len); -- 2.39.0

2 2

tcp.c: leverage MSG_PEEK with offset kernel capability when available
by Jon Maloy 06 Dec '23

06 Dec '23

The kernel may support recvmsg(MSG_PEEK), starting from a given offset. This makes it possible to avoid repeated reading of already read initial bytes of a received message, hence saving us read cycles when forwarding TCP messages in the host->name space direction. When this feature is supported, iov_sock[0].iov_base can be set to NULL. The kernel code will interpret this as an instruction to skip reading of the first iov_sock[0].iov_len bytes of the message. Since iov_sock[0].iov_base is set to point to tcp_buf_discard, we do this by simply not allocating that buffer, letting the pointer remain NULL, when we find that we have this kernel support. There is no macro or function indicating kernel support for this feature. We therefore have to probe for it by reading from an established TCP connection. The traffic connection cannot be used for this purpose, because it will be broken if the probe reading fails. We therefore have to create a temporary local connection for this task. Because of this, we also add a new function, tcp_probe_msg_peek_offset_cap(), which creates this temporary connection and performs the probe read on it. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- tcp.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 108 insertions(+), 2 deletions(-) diff --git a/tcp.c b/tcp.c index f506cfd..ab5168e 100644 --- a/tcp.c +++ b/tcp.c @@ -402,6 +402,8 @@ struct tcp6_l2_head { /* For MSS6 macro: keep in sync with tcp6_l2_buf_t */ (conn->events & (SOCK_FIN_RCVD | TAP_FIN_RCVD))) #define CONN_HAS(conn, set) ((conn->events & (set)) == (set)) +static bool tcp_probe_msg_peek_offset_cap(); + static const char *tcp_event_str[] __attribute((__unused__)) = { "SOCK_ACCEPTED", "TAP_SYN_RCVD", "ESTABLISHED", "TAP_SYN_ACK_SENT", @@ -506,7 +508,8 @@ static struct tcp_buf_seq_update tcp6_l2_buf_seq_update[TCP_FRAMES_MEM]; static unsigned int tcp6_l2_buf_used; /* recvmsg()/sendmsg() data for tap */ -static char tcp_buf_discard [MAX_WINDOW]; +static char *tcp_buf_discard = NULL; + static struct iovec iov_sock [TCP_FRAMES_MEM + 1]; static struct iovec tcp4_l2_iov [TCP_FRAMES_MEM]; @@ -573,6 +576,15 @@ static unsigned int tcp6_l2_flags_buf_used; #define CONN(idx) (&(FLOW(idx)->tcp)) + +/** msg_peek_offset_cap() - Does the kernel support recvmsg(MSG_PEEK) with offset? + */ +static inline bool msg_peek_offset_cap() +{ + return !tcp_buf_discard; +} + + /** conn_at_idx() - Find a connection by index, if present * @idx: Index of connection to lookup * @@ -2224,7 +2236,9 @@ static int tcp_data_from_sock(struct ctx *c, struct tcp_tap_conn *conn) return 0; } - sendlen = len - already_sent; + sendlen = len; + if (!msg_peek_offset_cap()) + sendlen -= already_sent; if (sendlen <= 0) { conn_flag(c, conn, STALLED); return 0; @@ -3107,6 +3121,15 @@ int tcp_init(struct ctx *c) NS_CALL(tcp_ns_socks_init, c); } + /* Only allocate discard buffer if needed */ + if (!tcp_probe_msg_peek_offset_cap()) { + tcp_buf_discard = malloc(MAX_WINDOW); + if (!tcp_buf_discard) { + perror("failed to allocate discard buffer\n"); + exit(EXIT_FAILURE); + } + debug("allocated discard buffer of size %i\n", MAX_WINDOW); + } return 0; } @@ -3213,3 +3236,86 @@ void tcp_timer(struct ctx *c, const struct timespec *ts) if (c->mode == MODE_PASTA) tcp_splice_refill(c); } + +/** tcp_probe_msg_peek_offset_cap() - Probe kernel for MSG_PEEK with offset support + */ +static bool tcp_probe_msg_peek_offset_cap() +{ + int listenerfd, fd; + ssize_t len; + char buf[8] = { 0 }; + struct msghdr msg; + struct iovec iov[2]; + struct sockaddr_in addr = { 0, }; + socklen_t addrlen = sizeof(addr); + int option = 1; + bool ret = true; + + memset(buf, 0, sizeof(buf)); + addr.sin_family = AF_INET; + addr.sin_addr.s_addr = INADDR_ANY; + addr.sin_port = 0; + + if ( (listenerfd = socket(AF_INET, SOCK_STREAM, 0)) < 0 ) { + perror("temporary listener socket creation failed"); + exit(EXIT_FAILURE); + } + setsockopt(listenerfd, SOL_SOCKET, SO_REUSEADDR, &option, sizeof(option)); + + if (bind(listenerfd, (const struct sockaddr *)&addr, sizeof(addr)) < 0 ) { + perror("temporary bind() failed"); + exit(EXIT_FAILURE); + } + if (0 > listen(listenerfd, 100)) { + perror("temporary listen failed"); + exit(EXIT_FAILURE); + } + if (0 != getsockname(listenerfd, (struct sockaddr *)&addr, &addrlen)) { + perror("temporary getsockname() failed"); + exit(EXIT_FAILURE); + } + + /* Spawn off a child process and let it send a message here: */ + if (fork() == 0) { + if ( (fd = socket(AF_INET, SOCK_STREAM, 0)) < 0 ) { + perror("temporary socket creation failed"); + exit(EXIT_FAILURE); + } + if (0 > connect(fd, (struct sockaddr *)&addr, sizeof(addr))) { + perror("temporary connect() failed"); + exit(EXIT_FAILURE); + } + if (sizeof(buf) != send(fd, buf, sizeof(buf), 0)) + perror("send on temporary server socket failed"); + shutdown(fd, SHUT_RD | SHUT_WR); + close(fd); + exit(0); + } + /* Receive the message in mother process: */ + fd = accept(listenerfd, NULL, NULL); + if (fd <= 0) { + perror("accept on temporary listener socket failed"); + exit(EXIT_FAILURE); + } + memset(&msg, 0, sizeof(msg)); + msg.msg_iov = iov; + msg.msg_iovlen = 2; + iov[0].iov_base = NULL; + iov[0].iov_len = 4; + iov[1].iov_base = &buf[3]; + iov[1].iov_len = 4; + len = recvmsg(fd, &msg, MSG_PEEK); + if (len <= 0) { + if (errno != EFAULT) { + perror("temporary recvmsg() failed"); + exit(EXIT_FAILURE); + } + ret = false; + } else if ((size_t)len != iov[1].iov_len) { + exit(EXIT_FAILURE); + } + shutdown(fd, SHUT_RD | SHUT_WR); + close(fd); + info("MSG_PEEK with offset %ssupported by kernel.\n", ret ? "" : "not "); + return ret; +} -- 2.39.2

2 4

tcp: add support for read with offset when using MSG_PEEK
by Jon Maloy 06 Dec '23

06 Dec '23

When reading received messages with MSG_PEEK, we sometines have to read the leading bytes of the stream several times, only to reach the bytes we really want. This is clearly non-optimal. What we would want is something similar to pread/preadv(), but working even for tcp sockets. At the same time, we don't want to add any new arguments to the recv/recvmsg() calls. In this commit, we allow the user to set iovec.iov_base in the first vector entry to NULL. This tells the socket to skip the first entry, hence letting the iov_len field of that entry indicate the offset value. This way, there is no need to add any new arguments or flags. In the iperf3 logs examples shown below, we can observe a throughput improvement of ~20 % in the direction host->namespace when using the protocol splicer 'passt'. This is a consistent result. $ ./passt/passt/pasta --config-net -f MSG_PEEK with offset not supported. [root@fedora37 ~]# perf record iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 60344 [ 6] local 192.168.122.163 port 5201 connected to 192.168.122.1 port 60360 [ ID] Interval Transfer Bitrate {...] [ 6] 13.00-14.00 sec 2.54 GBytes 21.8 Gbits/sec [ 6] 14.00-15.00 sec 2.52 GBytes 21.7 Gbits/sec [ 6] 15.00-16.00 sec 2.50 GBytes 21.5 Gbits/sec [ 6] 16.00-17.00 sec 2.49 GBytes 21.4 Gbits/sec [ 6] 17.00-18.00 sec 2.51 GBytes 21.6 Gbits/sec [ 6] 18.00-19.00 sec 2.48 GBytes 21.3 Gbits/sec [ 6] 19.00-20.00 sec 2.49 GBytes 21.4 Gbits/sec [ 6] 20.00-20.04 sec 87.4 MBytes 19.2 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 6] 0.00-20.04 sec 48.9 GBytes 21.0 Gbits/sec receiver ----------------------------------------------------------- [jmaloy@fedora37 ~]$ ./passt/passt/pasta --config-net -f MSG_PEEK with offset supported. [root@fedora37 ~]# perf record iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 46362 [ 6] local 192.168.122.163 port 5201 connected to 192.168.122.1 port 46374 [ ID] Interval Transfer Bitrate [...] [ 6] 12.00-13.00 sec 3.18 GBytes 27.3 Gbits/sec [ 6] 13.00-14.00 sec 3.17 GBytes 27.3 Gbits/sec [ 6] 14.00-15.00 sec 3.13 GBytes 26.9 Gbits/sec [ 6] 15.00-16.00 sec 3.17 GBytes 27.3 Gbits/sec [ 6] 16.00-17.00 sec 3.17 GBytes 27.2 Gbits/sec [ 6] 17.00-18.00 sec 3.14 GBytes 27.0 Gbits/sec [ 6] 18.00-19.00 sec 3.17 GBytes 27.2 Gbits/sec [ 6] 19.00-20.00 sec 3.12 GBytes 26.8 Gbits/sec [ 6] 20.00-20.04 sec 119 MBytes 25.5 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 6] 0.00-20.04 sec 59.4 GBytes 25.4 Gbits/sec receiver ----------------------------------------------------------- Passt is used to support VMs in containers, such as KubeVirt, and is also generally supported in libvirt/QEMU since release 9.2 / 7.2. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> Signed-off-by: Jon Paul Maloy <jmaloy(a)redhat.com> --- net/ipv4/tcp.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 53bcc17c91e4..e9d3b5bf2f66 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -2310,6 +2310,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, int *cmsg_flags) { struct tcp_sock *tp = tcp_sk(sk); + size_t peek_offset; int copied = 0; u32 peek_seq; u32 *seq; @@ -2353,6 +2354,20 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, if (flags & MSG_PEEK) { peek_seq = tp->copied_seq; seq = &peek_seq; + if (!msg->msg_iter.__iov[0].iov_base) { + peek_offset = msg->msg_iter.__iov[0].iov_len; + msg->msg_iter.__iov = &msg->msg_iter.__iov[1]; + if (msg->msg_iter.nr_segs <= 1) + goto out; + msg->msg_iter.nr_segs -= 1; + if (msg->msg_iter.count <= peek_offset) + goto out; + msg->msg_iter.count -= peek_offset; + if (len <= peek_offset) + goto out; + len -= peek_offset; + *seq += peek_offset; + } } target = sock_rcvlowat(sk, flags & MSG_WAITALL, len); -- 2.39.2

1 0

passt: new version 2023_12_04.b86afe3 available
by Stefano Brivio 04 Dec '23

04 Dec '23

The new version with tag 2023_12_04.b86afe3 includes the following changes: b86afe3 tcp: Don't defer hash table removal e21b6d6 tcp: "TCP" hash secret doesn't need to be TCP specific cf83988 pif: Add helpers to get the name of a pif 460064d test: Avoid hitting guestfish command length limits 705549f flow,tcp: Use epoll_ref type including flow and side 788d2fe tcp_splice: Use unsigned to represent side ecea8d3 flow,tcp: Generalise TCP epoll_ref to generic flows 31bab5f tcp: Remove unneccessary bounds check in tcp_timer_handler() df96a4c flow: Introduce 'sidx' type to represent one side of one flow eb8b1a2 flow, tcp: Add logging helpers for connection related messages 96590b0 flow: Make unified version of flow table compaction 9d44aba util: MAX_FROM_BITS() should be unsigned e2e8219 flow, tcp: Consolidate flow pointer<->index helpers f08ce92 flow, tcp: Move TCP connection table to unified flow table 16ae032 flow, tcp: Generalise connection types ba84a3b treewide: Add messages to static_assert() calls d902bb6 tcp: remove useless assignment bae676a port_fwd, util: Include additional headers to fix build with musl 7e175f3 packet: Offset plus length is not always uint32_t, but it's always size_t 0655904 treewide: Use 'z' length modifier for size_t/ssize_t conversions 4117bd9 port_fwd, util: Don't bind UDP ports with opposite-side bound TCP ports https://passt.top/passt/log/?qt=range&q=2023_11_19.4f1709d..2023_12_04.b86a… Packages: - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia, openSUSE): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/6721932/ permanent mirror: https://passt.top/builds/copr/0^20231204.gb86afe3/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-gb86afe3-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_0b86afe3-1_all.deb -- Stefano

1 0