- dev - passt

[PATCH v3 0/4] Add vhost-user support to passt. (part 3)
by Laurent Vivier 05 Sep '24

05 Sep '24

This series of patches adds vhost-user support to passt and then allows passt to connect to QEMU network backend using virtqueue rather than a socket. With QEMU, rather than using to connect: -netdev stream,id=s,server=off,addr.type=unix,addr.path=/tmp/passt_1.socket we will use: -chardev socket,id=chr0,path=/tmp/passt_1.socket -netdev vhost-user,id=netdev0,chardev=chr0 -device virtio-net,netdev=netdev0 -object memory-backend-memfd,id=memfd0,share=on,size=$RAMSIZE -numa node,memdev=memfd0 The memory backend is needed to share data between passt and QEMU. Performance comparison between "-netdev stream" and "-netdev vhost-user": $ iperf3 -c localhost -p 10001 -t 60 -6 -u -b 50G socket: [ 5] 0.00-60.05 sec 95.6 GBytes 13.7 Gbits/sec 0.017 ms 6998988/10132413 (69%) receiver vhost-user: [ 5] 0.00-60.04 sec 237 GBytes 33.9 Gbits/sec 0.006 ms 53673/7813770 (0.69%) receiver $ iperf3 -c localhost -p 10001 -t 60 -4 -u -b 50G socket: [ 5] 0.00-60.05 sec 98.9 GBytes 14.1 Gbits/sec 0.018 ms 6260735/9501832 (66%) receiver vhost-user: [ 5] 0.00-60.05 sec 235 GBytes 33.7 Gbits/sec 0.008 ms 37581/7752699 (0.48%) receiver $ iperf3 -c localhost -p 10001 -t 60 -6 socket: [ 5] 0.00-60.00 sec 17.3 GBytes 2.48 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 17.3 GBytes 2.48 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 191 GBytes 27.4 Gbits/sec 0 sender [ 5] 0.00-60.05 sec 191 GBytes 27.3 Gbits/sec receiver $ iperf3 -c localhost -p 10001 -t 60 -4 socket: [ 5] 0.00-60.00 sec 15.6 GBytes 2.24 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 15.6 GBytes 2.24 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 189 GBytes 27.1 Gbits/sec 0 sender [ 5] 0.00-60.04 sec 189 GBytes 27.0 Gbits/sec receiver v3: - rebase on top of flow table - update tcp_vu.c to look like udp_vu.c (recv()/prepare()/send_frame()) - address comments from Stefano and David on version 2 v2: - remove PATCH 4 - rewrite PATCH 2 and 3 to follow passt coding style - move some code from PATCH 3 to PATCH 4 (previously PATCH 5) - partially addressed David's comment on PATCH 5 Laurent Vivier (4): packet: replace struct desc by struct iovec vhost-user: introduce virtio API vhost-user: introduce vhost-user API vhost-user: add vhost-user Makefile | 6 +- checksum.c | 1 - conf.c | 24 +- epoll_type.h | 4 + iov.c | 1 - isolation.c | 15 +- packet.c | 93 ++-- packet.h | 16 +- passt.c | 25 +- passt.h | 6 + pcap.c | 1 - tap.c | 106 +++- tap.h | 5 +- tcp.c | 33 +- tcp_buf.c | 6 +- tcp_internal.h | 3 +- tcp_vu.c | 593 ++++++++++++++++++++++ tcp_vu.h | 12 + udp.c | 71 +-- udp.h | 8 +- udp_internal.h | 34 ++ udp_vu.c | 338 +++++++++++++ udp_vu.h | 13 + util.h | 8 + vhost_user.c | 1277 ++++++++++++++++++++++++++++++++++++++++++++++++ vhost_user.h | 202 ++++++++ virtio.c | 656 +++++++++++++++++++++++++ virtio.h | 185 +++++++ vu_common.c | 27 + vu_common.h | 34 ++ 30 files changed, 3666 insertions(+), 137 deletions(-) create mode 100644 tcp_vu.c create mode 100644 tcp_vu.h create mode 100644 udp_internal.h create mode 100644 udp_vu.c create mode 100644 udp_vu.h create mode 100644 vhost_user.c create mode 100644 vhost_user.h create mode 100644 virtio.c create mode 100644 virtio.h create mode 100644 vu_common.c create mode 100644 vu_common.h -- 2.45.2

3 21

[PATCH] log: Don't prefix log file messages with time and severity if they're continuations
by Stefano Brivio 05 Sep '24

05 Sep '24

In fecb1b65b1ac ("log: Don't prefix message with timestamp on --debug if it's a continuation"), I fixed this for --debug on standard error, but not for log files: if messages are continuations, they shouldn't be prefixed by timestamp and severity. Otherwise, we'll print stuff like this: 0.0028: ERROR: Receive error on guest connection, reset0.0028: ERROR: : Bad file descriptor Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- log.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/log.c b/log.c index 433b552..a61468e 100644 --- a/log.c +++ b/log.c @@ -224,19 +224,23 @@ static int logfile_rotate(int fd, const struct timespec *now) /** * logfile_write() - Write entry to log file, trigger rotation if full * @newline: Append newline at the end of the message, if missing + * @cont: Continuation of a previous message, on the same line * @pri: Facility and level map, same as priority for vsyslog() * @now: Timestamp * @format: Same as vsyslog() format * @ap: Same as vsyslog() ap */ -static void logfile_write(bool newline, int pri, const struct timespec *now, +static void logfile_write(bool newline, bool cont, int pri, + const struct timespec *now, const char *format, va_list ap) { char buf[BUFSIZ]; - int n; + int n = 0; - n = logtime_fmt(buf, BUFSIZ, now); - n += snprintf(buf + n, BUFSIZ - n, ": %s", logfile_prefix[pri]); + if (!cont) { + n += logtime_fmt(buf, BUFSIZ, now); + n += snprintf(buf + n, BUFSIZ - n, ": %s", logfile_prefix[pri]); + } n += vsnprintf(buf + n, BUFSIZ - n, format, ap); @@ -278,7 +282,7 @@ void vlogmsg(bool newline, bool cont, int pri, const char *format, va_list ap) va_copy(ap2, ap); /* Don't clobber ap, we need it again */ if (log_file != -1) - logfile_write(newline, pri, now, format, ap2); + logfile_write(newline, cont, pri, now, format, ap2); else if (!(log_mask & LOG_MASK(LOG_DEBUG))) passt_vsyslog(newline, pri, format, ap2); -- 2.43.0

2 1

[PATCH 0/2] Adding SO_PEEK_OFF for TCPv6
by jmaloy＠redhat.com 01 Sep '24

01 Sep '24

From: Jon Maloy <jmaloy(a)redhat.com> Adding SO_PEEK_OFF for TCPv6 and selftest for both TCPv4 and TCPv6. Jon Maloy (2): tcp: add SO_PEEK_OFF socket option tor TCPv6 selftests: add selftest for tcp SO_PEEK_OFF support net/ipv6/af_inet6.c | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/tcp_so_peek_off.c | 181 ++++++++++++++++++ 3 files changed, 183 insertions(+) create mode 100644 tools/testing/selftests/net/tcp_so_peek_off.c -- 2.45.2

4 8

[PATCH] Makefile: Enable _FORTIFY_SOURCE iff needed
by Michal Privoznik 30 Aug '24

30 Aug '24

On some systems source fortification is enabled whenever code optimization is enabled (e.g. with -O2). Since code fortification is explicitly enabled too (with possibly different value than the system wants, there are three levels [1]), distros are required to patch our Makefile, e.g. [2]. Detect whether fortification is not already enabled and enable it explicitly only if really needed. 1: https://www.gnu.org/software/libc/manual/html_node/Source-Fortification.html 2: https://github.com/gentoo/gentoo/commit/edfeb8763ac56112c59248c62c9cda13e5d… Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- It may be worth exploring whether level 3 would be beneficial: https://developers.redhat.com/articles/2022/09/17/gccs-new-fortification-le… Makefile | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 01fada4..74a9513 100644 --- a/Makefile +++ b/Makefile @@ -33,9 +33,16 @@ AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/MIPS64EL/MIPSEL64/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/HPPA/PARISC/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/SH4/SH/') +# On some systems enabling optimization also enables source fortification, +# automagically. Do not override it. +FORTIFY_FLAG := +ifeq ($(shell $(CC) -O2 -dM -E - < /dev/null 2>&1 | grep ' _FORTIFY_SOURCE ' > /dev/null; echo $$?),1) +FORTIFY_FLAG := -D_FORTIFY_SOURCE=2 +endif + FLAGS := -Wall -Wextra -Wno-format-zero-length FLAGS += -pedantic -std=c11 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE -FLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE +FLAGS += $(FORTIFY_FLAG) -O2 -pie -fPIE FLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) FLAGS += -DNETNS_RUN_DIR=\"/run/netns\" FLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) -- 2.44.2

3 3

[PATCH v3 0/3] Probe host's ephemeral ports, rather than using RFC values
by David Gibson 30 Aug '24

30 Aug '24

As discussed on our recent call, this implements basing which ports we consider "ephemeral" on probing the host's settings, rather than just assuming the RFC 6335 recommended values, which are not what Linux uses by default. I think this is more correct, but additionally using the Linux values means we consider more ports ephemeral, reducing kernel memory consumption for -t all -u all. Changes in v3: * Used in_port_t instead of plan uint16_t * Considered using sscanf() rather than strchr() + strtol(), but decided against it. I can never remember exactly what is and isn't accepted by scanf(), plus clang-tidy complained about it. Changes in v2: * Add missing close() for the sysctl file David Gibson (3): conf, fwd: Make ephemeral port logic more flexible conf, fwd: Don't attempt to forward port 0 fwd, conf: Probe host's ephemeral ports conf.c | 19 +++++++++++---- fwd.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ fwd.h | 3 +++ util.h | 3 --- 4 files changed, 92 insertions(+), 7 deletions(-) -- 2.46.0

3 5

[net-next, v3 0/2] Adding SO_PEEK_OFF for TCPv6
by jmaloy＠redhat.com 30 Aug '24

30 Aug '24

From: Jon Maloy <jmaloy(a)redhat.com> Adding SO_PEEK_OFF for TCPv6 and selftest for both TCPv4 and TCPv6 Jon Maloy (2): tcp: add SO_PEEK_OFF socket option tor TCPv6 selftests: add selftest for tcp SO_PEEK_OFF support net/ipv6/af_inet6.c | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/tcp_so_peek_off.c | 183 ++++++++++++++++++ 3 files changed, 185 insertions(+) create mode 100644 tools/testing/selftests/net/tcp_so_peek_off.c -- 2.45.2

3 4

[PATCH v2 0/3] Probe host's ephemeral ports, rather than using RFC values
by David Gibson 29 Aug '24

29 Aug '24

As discussed on our recent call, this implements basing which ports we consider "ephemeral" on probing the host's settings, rather than just assuming the RFC 6335 recommended values, which are not what Linux uses by default. I think this is more correct, but additionally using the Linux values means we consider more ports ephemeral, reducing kernel memory consumption for -t all -u all. Changes in v2: * Add missing close() for the sysctl file David Gibson (3): conf, fwd: Make ephemeral port logic more flexible conf, fwd: Don't attempt to forward port 0 fwd, conf: Probe host's ephemeral ports conf.c | 19 +++++++++++---- fwd.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ fwd.h | 3 +++ util.h | 3 --- 4 files changed, 92 insertions(+), 7 deletions(-) -- 2.46.0

2 5

[PATCH 0/3] Probe host's ephemeral ports, rather than using RFC values
by David Gibson 29 Aug '24

29 Aug '24

As discussed on our recent call, this implements basing which ports we consider "ephemeral" on probing the host's settings, rather than just assuming the RFC 6335 recommended values, which are not what Linux uses by default. I think this is more correct, but additionally using the Linux values means we consider more ports ephemeral, reducing kernel memory consumption for -t all -u all. David Gibson (3): conf, fwd: Make ephemeral port logic more flexible conf, fwd: Don't attempt to forward port 0 fwd, conf: Probe host's ephemeral ports conf.c | 19 ++++++++++++---- fwd.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ fwd.h | 3 +++ util.h | 3 --- 4 files changed, 88 insertions(+), 7 deletions(-) -- 2.46.0

3 9

[net-next, v2 0/2] Adding SO_PEEK_OFF for TCPv6
by jmaloy＠redhat.com 28 Aug '24

28 Aug '24

From: Jon Maloy <jmaloy(a)redhat.com> Adding SO_PEEK_OFF for TCPv6 and selftest for it. Jon Maloy (2): tcp: add SO_PEEK_OFF socket option tor TCPv6 selftests: add selftest for tcp SO_PEEK_OFF support net/ipv6/af_inet6.c | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/tcp_so_peek_off.c | 184 ++++++++++++++++++ 3 files changed, 186 insertions(+) create mode 100644 tools/testing/selftests/net/tcp_so_peek_off.c -- 2.45.2

4 7

[PATCH v2 0/3] Dual stack sockets for UDP
by David Gibson 27 Aug '24

27 Aug '24

Here, at last, is a change to use dual stack sockets for UDP port forwards as we have for TCP for some time. With the merging of many parallel IPv4/IPv6 structures being done for the flow table and other recent work, this actually turned out much easier than I anticipated. Changes since v1: * Removed a redundant assignment to msg_namelen David Gibson (3): udp: Merge udp[46]_mh_recv arrays udp: Remove unnnecessary local from udp_sock_init() udp: Use dual stack sockets for port forwarding when possible udp.c | 101 +++++++++++++++++++++++++++++----------------------------- udp.h | 2 -- 2 files changed, 50 insertions(+), 53 deletions(-) -- 2.46.0

2 4