- dev - passt

[PATCH 0/3] pasta: Implement runtime auto-forwarding for UDP
by David Gibson 19 Nov '23

19 Nov '23

In "auto" mode port forwarding, we scan for listening TCP ports at start up, and then every second thereafter. For UDP, however, we scan only at start up. That's documented, but surprising, and there's no real reason for it to be that way. Implement periodic scanning for UDP auto forwarding too. Link: https://bugs.passt.top/show_bug.cgi?id=45 *** BLURB HERE *** David Gibson (3): tcp: Use common helper for rebinding inbound and outbound ports tcp: Simplify away tcp_port_rebind() udp,pasta: Periodically scan for ports to automatically forward passt.1 | 3 +- tcp.c | 109 +++++++++++++++++++++++--------------------------------- udp.c | 74 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 120 insertions(+), 66 deletions(-) -- 2.41.0

2 4

[PATCH] clang-tidy: Suppress silly misc-include-cleaner warnings
by David Gibson 19 Nov '23

19 Nov '23

clang-tidy from LLVM 17.0.3 (which is in Fedora 39) includes a new "misc-include-cleaner" warning that tries to make sure that headers *directly* provide the things that are used in the .c file. That sounds great in theory but is in practice unusable: Quite a few common things in the standard library are ultimately provided by OS-specific system headers, but for portability should be accessed via closer-to-standardised library headers. This will warn constantly about such cases: e.g. it will want you to include <linux/limits.h> instead of <limits.h> to get PATH_MAX. So, suppress this warning globally in the Makefile. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- Makefile | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 743c123..ff21459 100644 --- a/Makefile +++ b/Makefile @@ -250,6 +250,12 @@ docs: README.md # - bugprone-assignment-in-if-condition # Dubious value over the compiler's built-in warning. Would # increase verbosity. +# +# - misc-include-cleaner +# Wants to include headers which *directly* provide the things +# we use. That sounds nice, but means it will often want a OS +# specific header instead of a mostly standard one, such as +# <linux/limits.h> instead of <limits.h>. clang-tidy: $(SRCS) $(HEADERS) clang-tidy -checks=*,-modernize-*,\ @@ -275,7 +281,8 @@ clang-tidy: $(SRCS) $(HEADERS) -readability-function-cognitive-complexity,\ -altera-struct-pack-align,\ -concurrency-mt-unsafe,\ - -readability-identifier-length \ + -readability-identifier-length,\ + -misc-include-cleaner \ -config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \ --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS) $(CPPFLAGS)) -DCLANG_TIDY_58992 -- 2.41.0

2 1

[PATCH v2 0/2] Some fixes for valgrind suppressions
by David Gibson 16 Nov '23

16 Nov '23

With the new versions of compiler and valgrind in Fedora 39, I hit some spurious test failures. Here are some workarounds. Changes since v1: * Took entirely different approach, fixing the existing suppression * Extra patch allowing optimised valgrind builds David Gibson (2): valgrind: Adjust suppression for MSG_TRUNC with NULL buffer valgrind: Don't disable optimizations for valgrind builds Makefile | 2 +- tcp.c | 9 +++++++++ test/valgrind.supp | 3 +-- 3 files changed, 11 insertions(+), 3 deletions(-) -- 2.41.0

2 4

[PATCH] cppcheck,valgrind: Don't pass NULL to recv() with MSG_TRUNC
by David Gibson 16 Nov '23

16 Nov '23

Usually, of course, it's invalid to pass a NULL buffer to recv(). However, it's acceptable when using MSG_TRUNC, because that suppresses actually writing to the buffer. So, we pass NULL in tcp_sock_consume(). Unfortunately, checker tools aren't always aware of that special case: we already have a suppression for cppcheck to cover this. valgrind-3.22.0 (present in Fedora 39) has a similar problem, generating a spurious warning here. We could generate another suppression for valgrind, however, it so happens that we already have tcp_buf_discard ready to hand. If we pass this instead of NULL it makes both cppcheck and valgrind happy. We're still using the MSG_TRUNC flag, the kernel doesn't actually have to copy data, so we should still have the performance benefits of it. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tcp.c b/tcp.c index cfcd40a..f51d27a 100644 --- a/tcp.c +++ b/tcp.c @@ -2106,8 +2106,13 @@ static int tcp_sock_consume(const struct tcp_tap_conn *conn, uint32_t ack_seq) if (SEQ_LE(ack_seq, conn->seq_ack_from_tap)) return 0; - /* cppcheck-suppress [nullPointer, unmatchedSuppression] */ - if (recv(conn->sock, NULL, ack_seq - conn->seq_ack_from_tap, + /* Since we're using MSG_TRUNC, it's allowed to pass NULL instead of a + * real buffer. However some checker tools (including at least some + * versions of cppcheck and valgrind) aren't aware of that special case. + * We so happen to have a convenient discard buffer, so we might as well + * pass it to avoid spurious complaints from those tools. + */ + if (recv(conn->sock, tcp_buf_discard, ack_seq - conn->seq_ack_from_tap, MSG_DONTWAIT | MSG_TRUNC) < 0) return -errno; -- 2.41.0

2 3

passt: new version 2023_11_10.5ec3634 available
by Stefano Brivio 12 Nov '23

12 Nov '23

The new version with tag 2023_11_10.5ec3634 includes the following changes: 5ec3634 tap, pasta: Handle short writes to /dev/tap f0776ea tap, pasta: Handle incomplete tap sends for pasta too cf3eeba tcp: Don't use TCP_WINDOW_CLAMP 930bc3b tcp: Rename and small cleanup to tcp_clamp_window() 2c1554c test/lib/perf_report: Fix up table highlight for pasta's local flows https://passt.top/passt/log/?qt=range&q=2023_11_07.56d9f6d..2023_11_10.5ec3… Packages: - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia, openSUSE): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/6619714/ permanent mirror: https://passt.top/builds/copr/0^20231110.g5ec3634/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-g5ec3634-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_5ec3634-1_all.deb -- Stefano

1 0

[PATCH 0/2] Avoid bugs related to TCP_WINDOW_CLAMP
by David Gibson 10 Nov '23

10 Nov '23

The way we used TCP_WINDOW_CLAMP didn't really correspond to what it actually did. More recent kernels also have a nasty bug in the handler which we could trigger, causing TCP stalls. David Gibson (2): tcp: Rename and small cleanup to tcp_clamp_window() tcp: Don't use TCP_WINDOW_CLAMP tcp.c | 70 +++++++++--------------------------------------------- tcp_conn.h | 7 +++--- 2 files changed, 14 insertions(+), 63 deletions(-) -- 2.41.0

2 6

[PATCH 0/2] Handle incomplete tap writes for pasta
by David Gibson 10 Nov '23

10 Nov '23

passt can handle various cases where we're unable to write all the data we want to to the tap side. The same logic would make sense for pasta, however right now we don't properly report incomplete writes with the tuntap device of pasta, rather than the qemu socket of passt. I come across these as possible issues with the various stall bugs we've been investigating. It doesn't look like they're relevant there after all, but they should be correct changes nonetheless. David Gibson (2): tap, pasta: Handle incomplete tap sends for pasta too tap, pasta: Handle short writes to /dev/tap tap.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) -- 2.41.0

2 3

[PATCH 0/3] Minimal fixes for issues reported by static checkers
by Stefano Brivio 08 Nov '23

08 Nov '23

These are reported by clang-tidy and Coverity after David's series, I'm applying minimal fixes for them to go ahead and apply those series, as we should really make a new release at this point. They are all trivial, I'm posting them to ease reviews but I already applied these. Stefano Brivio (3): netlink: Sequence numbers are actually 32 bits wide port_fwd: Don't try to read bound ports from invalid file handles log: Match implicit va_start() with va_end() in vlogmsg() log.c | 2 ++ netlink.c | 20 ++++++++++---------- port_fwd.c | 3 +++ 3 files changed, 15 insertions(+), 10 deletions(-) -- 2.39.2

2 5

passt: new version 2023_11_07.56d9f6d available
by Stefano Brivio 07 Nov '23

07 Nov '23

The new version with tag 2023_11_07.56d9f6d includes the following changes: 56d9f6d Revert "selinux: Drop user_namespace class rules for Fedora 37" 74e6f48 selinux: Allow passt to talk over unconfined_t UNIX domain socket for --fd 50bc25a log: Match implicit va_start() with va_end() in vlogmsg() 9494a51 port_fwd: Don't try to read bound ports from invalid file handles b944622 netlink: Sequence numbers are actually 32 bits wide 53ff387 test/perf: Simplify calculation of "omit" time for TCP throughput aa0bb9f test/perf: Remove unnecessary --pacing-timer options 498108a test/perf: "MTU" changes in passt_tcp host to guest aren't useful f94adb1 test/perf: Explicitly control UDP packet length, instead of MTU 2926970 test/perf: Small MTUs for spliced TCP aren't interesting e516809 test/perf: Start iperf3 server less often f9ff667 test/perf: Get iperf3 stats from client side 8a41a8b test/perf: Remove stale iperf3c/iperf3s directives de974f0 udp: Remove socket from udp_{tap,splice}_map when timed out 480aa4a udp: Consistently use -1 to indicate un-opened sockets in maps c0426ff log: Add vlogmsg() 5972203 log: Enable format warnings 50d46ec log: Don't define logging function 4 times 0ad54e1 tcp: Remove remaining declaration of tcp_l2_mh 515db1e tcp_splice: Simplify selection of socket and pipe sides in socket handler 7486cd1 tcp_splice: Exploit side symmetry in tcp_splice_destroy() 69db3b3 tcp_splice: Exploit side symmetry in tcp_splice_connect_finish() 1b76257 tcp_splice: Exploit side symmetry in tcp_splice_timer() 8545058 tcp_splice: Rename sides of connection from a/b to 0/1 0e8e534 tcp_splice: Don't pool pipes in pairs 6357010 tcp_splice: Avoid awkward temporaries in tcp_splice_epoll_ctl() 409d3ca tcp_splice: Remove unnecessary forward declaration 5a79ba6 tcp_splice: Don't handle EPOLL_CTL_DEL as part of tcp_splice_epoll_ctl() 536acab tcp_splice: Correct error handling in tcp_splice_epoll_ctl() d33cbc6 tcp_splice: Remove redundant tcp_splice_epoll_ctl() f6d8dc2 pif: Pass originating pif to tap handler functions 732e249 pif: Record originating pif in listening socket refs 125c5e5 pif: Introduce notion of passt/pasta interface 0d73fa2 udp: Clean up ref initialisation in udp_sock_init() c09d0d0 port_fwd: Simplify get_bound_ports_*() to port_fwd_scan_*() dcf5c0e port_fwd: Move port scanning /proc fds into struct port_fwd 1a40d00 port_fwd: Split TCP and UDP cases for get_bound_ports() 180dbc9 port_fwd: Don't NS_CALL get_bound_ports() 5a04854 port_fwd: Pre-open /proc/net/* files rather than on-demand 4f0b9f9 util: Add open_in_ns() helper 17d40d1 port_fwd: Better parameterise procfs_scan_listen() e90f277 port_fwd: Move automatic port forwarding code to port_fwd.[ch] 26d86f1 conf: Cleaner initialisation of default forwarding modes 3fb3f0f selinux: Drop user_namespace class rules for Fedora 37 https://passt.top/passt/log/?qt=range&q=2023_10_04.f851084..2023_11_07.56d9… Packages: - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia, openSUSE): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/6607545/ permanent mirror: https://passt.top/builds/copr/0^20231107.g56d9f6d/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-g56d9f6d-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_56d9f6d-1_all.deb -- Stefano

1 0

[PATCH 0/8] Clean ups and speed ups to benchmarks
by David Gibson 07 Nov '23

07 Nov '23

Our standard "make check" includes a number of benchmarks, which take quite a long time to run. This series makes a number of improvements to how we run these, which reduces wasted time and reduces the full run time by some 10-12 minutes. David Gibson (8): test/perf: Remove stale iperf3c/iperf3s directives test/perf: Get iperf3 stats from client side test/perf: Start iperf3 server less often test/perf: Small MTUs for spliced TCP aren't interesting test/perf: Explicitly control UDP packet length, instead of MTU test/perf: "MTU" changes in passt_tcp host to guest aren't useful test/perf: Remove unnecessary --pacing-timer options test/perf: Simplify calculation of "omit" time for TCP throughput .gitignore | 2 +- test/lib/test | 83 ++++++++++++++++----------- test/perf/passt_tcp | 84 +++++++++++++-------------- test/perf/passt_udp | 86 +++++++++++++--------------- test/perf/pasta_tcp | 125 ++++++++++++++++------------------------ test/perf/pasta_udp | 136 ++++++++++++++++++++++++++------------------ 6 files changed, 260 insertions(+), 256 deletions(-) -- 2.41.0

2 9