November 2023 - dev

[PATCH 0/2] Handle incomplete tap writes for pasta
by David Gibson 10 Nov '23

10 Nov '23

passt can handle various cases where we're unable to write all the data we want to to the tap side. The same logic would make sense for pasta, however right now we don't properly report incomplete writes with the tuntap device of pasta, rather than the qemu socket of passt. I come across these as possible issues with the various stall bugs we've been investigating. It doesn't look like they're relevant there after all, but they should be correct changes nonetheless. David Gibson (2): tap, pasta: Handle incomplete tap sends for pasta too tap, pasta: Handle short writes to /dev/tap tap.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) -- 2.41.0

2 3

[PATCH 0/3] Minimal fixes for issues reported by static checkers
by Stefano Brivio 08 Nov '23

08 Nov '23

These are reported by clang-tidy and Coverity after David's series, I'm applying minimal fixes for them to go ahead and apply those series, as we should really make a new release at this point. They are all trivial, I'm posting them to ease reviews but I already applied these. Stefano Brivio (3): netlink: Sequence numbers are actually 32 bits wide port_fwd: Don't try to read bound ports from invalid file handles log: Match implicit va_start() with va_end() in vlogmsg() log.c | 2 ++ netlink.c | 20 ++++++++++---------- port_fwd.c | 3 +++ 3 files changed, 15 insertions(+), 10 deletions(-) -- 2.39.2

2 5

passt: new version 2023_11_07.56d9f6d available
by Stefano Brivio 07 Nov '23

07 Nov '23

The new version with tag 2023_11_07.56d9f6d includes the following changes: 56d9f6d Revert "selinux: Drop user_namespace class rules for Fedora 37" 74e6f48 selinux: Allow passt to talk over unconfined_t UNIX domain socket for --fd 50bc25a log: Match implicit va_start() with va_end() in vlogmsg() 9494a51 port_fwd: Don't try to read bound ports from invalid file handles b944622 netlink: Sequence numbers are actually 32 bits wide 53ff387 test/perf: Simplify calculation of "omit" time for TCP throughput aa0bb9f test/perf: Remove unnecessary --pacing-timer options 498108a test/perf: "MTU" changes in passt_tcp host to guest aren't useful f94adb1 test/perf: Explicitly control UDP packet length, instead of MTU 2926970 test/perf: Small MTUs for spliced TCP aren't interesting e516809 test/perf: Start iperf3 server less often f9ff667 test/perf: Get iperf3 stats from client side 8a41a8b test/perf: Remove stale iperf3c/iperf3s directives de974f0 udp: Remove socket from udp_{tap,splice}_map when timed out 480aa4a udp: Consistently use -1 to indicate un-opened sockets in maps c0426ff log: Add vlogmsg() 5972203 log: Enable format warnings 50d46ec log: Don't define logging function 4 times 0ad54e1 tcp: Remove remaining declaration of tcp_l2_mh 515db1e tcp_splice: Simplify selection of socket and pipe sides in socket handler 7486cd1 tcp_splice: Exploit side symmetry in tcp_splice_destroy() 69db3b3 tcp_splice: Exploit side symmetry in tcp_splice_connect_finish() 1b76257 tcp_splice: Exploit side symmetry in tcp_splice_timer() 8545058 tcp_splice: Rename sides of connection from a/b to 0/1 0e8e534 tcp_splice: Don't pool pipes in pairs 6357010 tcp_splice: Avoid awkward temporaries in tcp_splice_epoll_ctl() 409d3ca tcp_splice: Remove unnecessary forward declaration 5a79ba6 tcp_splice: Don't handle EPOLL_CTL_DEL as part of tcp_splice_epoll_ctl() 536acab tcp_splice: Correct error handling in tcp_splice_epoll_ctl() d33cbc6 tcp_splice: Remove redundant tcp_splice_epoll_ctl() f6d8dc2 pif: Pass originating pif to tap handler functions 732e249 pif: Record originating pif in listening socket refs 125c5e5 pif: Introduce notion of passt/pasta interface 0d73fa2 udp: Clean up ref initialisation in udp_sock_init() c09d0d0 port_fwd: Simplify get_bound_ports_*() to port_fwd_scan_*() dcf5c0e port_fwd: Move port scanning /proc fds into struct port_fwd 1a40d00 port_fwd: Split TCP and UDP cases for get_bound_ports() 180dbc9 port_fwd: Don't NS_CALL get_bound_ports() 5a04854 port_fwd: Pre-open /proc/net/* files rather than on-demand 4f0b9f9 util: Add open_in_ns() helper 17d40d1 port_fwd: Better parameterise procfs_scan_listen() e90f277 port_fwd: Move automatic port forwarding code to port_fwd.[ch] 26d86f1 conf: Cleaner initialisation of default forwarding modes 3fb3f0f selinux: Drop user_namespace class rules for Fedora 37 https://passt.top/passt/log/?qt=range&q=2023_10_04.f851084..2023_11_07.56d9… Packages: - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia, openSUSE): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/6607545/ permanent mirror: https://passt.top/builds/copr/0^20231107.g56d9f6d/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-g56d9f6d-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_56d9f6d-1_all.deb -- Stefano

1 0

[PATCH 0/8] Clean ups and speed ups to benchmarks
by David Gibson 07 Nov '23

07 Nov '23

Our standard "make check" includes a number of benchmarks, which take quite a long time to run. This series makes a number of improvements to how we run these, which reduces wasted time and reduces the full run time by some 10-12 minutes. David Gibson (8): test/perf: Remove stale iperf3c/iperf3s directives test/perf: Get iperf3 stats from client side test/perf: Start iperf3 server less often test/perf: Small MTUs for spliced TCP aren't interesting test/perf: Explicitly control UDP packet length, instead of MTU test/perf: "MTU" changes in passt_tcp host to guest aren't useful test/perf: Remove unnecessary --pacing-timer options test/perf: Simplify calculation of "omit" time for TCP throughput .gitignore | 2 +- test/lib/test | 83 ++++++++++++++++----------- test/perf/passt_tcp | 84 +++++++++++++-------------- test/perf/passt_udp | 86 +++++++++++++--------------- test/perf/pasta_tcp | 125 ++++++++++++++++------------------------ test/perf/pasta_udp | 136 ++++++++++++++++++++++++++------------------ 6 files changed, 260 insertions(+), 256 deletions(-) -- 2.41.0

2 9

[PATCH 0/2] udp: Fix bugs with saved socket fds
by David Gibson 07 Nov '23

07 Nov '23

We've had some issues with DNS requests starting to fail in long running guests or containers using passt/pasta. With assistance from a reporter, I've managed to track this down to some bugs in our management of saved UDP socket fds. Link: https://bugs.passt.top/show_bug.cgi?id=57 David Gibson (2): udp: Consistently use -1 to indicate un-opened sockets in maps udp: Remove socket from udp_{tap,splice}_map when timed out conf.c | 1 + udp.c | 36 +++++++++++++++++++++++++++--------- udp.h | 1 + 3 files changed, 29 insertions(+), 9 deletions(-) -- 2.41.0

2 5

[PATCH v2 0/3] Some improvements to log functions
by David Gibson 07 Nov '23

07 Nov '23

Several small improvements to the logging functions. Changes since v1: * Added printf warning attributes * Added vlogmsg() David Gibson (3): log: Don't define logging function 4 times log: Enable format warnings log: Add vlogmsg() Makefile | 3 ++- conf.c | 2 +- log.c | 72 +++++++++++++++++++++++++++++--------------------------- log.h | 14 +++++++---- packet.c | 7 +++--- pasta.c | 2 +- tcp.c | 13 ++++++---- 7 files changed, 63 insertions(+), 50 deletions(-) -- 2.41.0

2 5

[PATCH] tcp: Remove remaining declaration of tcp_l2_mh
by Laurent Vivier 07 Nov '23

07 Nov '23

Use of tcp_l2_mh has been removed in commit 38fbfdbcb95d, but its declaration and initialization are always in the code. Remove them as they are useless. Fixes: 38fbfdbcb95d ("tcp: Get rid of iov with cached MSS, drop sendmmsg(), add deferred flush") Cc: sbrivio(a)redhat.com Signed-off-by: Laurent Vivier <lvivier(a)redhat.com> --- tcp.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/tcp.c b/tcp.c index 945023c5b442..386cd015be1c 100644 --- a/tcp.c +++ b/tcp.c @@ -515,8 +515,6 @@ static struct iovec tcp6_l2_iov [TCP_FRAMES_MEM]; static struct iovec tcp4_l2_flags_iov [TCP_FRAMES_MEM]; static struct iovec tcp6_l2_flags_iov [TCP_FRAMES_MEM]; -static struct mmsghdr tcp_l2_mh [TCP_FRAMES_MEM]; - /* sendmsg() to socket */ static struct iovec tcp_iov [UIO_MAXIOV]; @@ -3143,7 +3141,6 @@ static void tcp_sock_refill_init(const struct ctx *c) */ int tcp_init(struct ctx *c) { - int i; #ifndef HAS_GETRANDOM int dev_random = open("/dev/random", O_RDONLY); unsigned int random_read = 0; @@ -3172,9 +3169,6 @@ int tcp_init(struct ctx *c) exit(EXIT_FAILURE); } - for (i = 0; i < ARRAY_SIZE(tcp_l2_mh); i++) - tcp_l2_mh[i] = (struct mmsghdr) { .msg_hdr.msg_iovlen = 1 }; - if (c->ifi4) tcp_sock4_iov_init(c); -- 2.41.0

3 2

[PATCH v2 00/11] tcp_splice: Better exploit symmetry between sides of connection
by David Gibson 07 Nov '23

07 Nov '23

For spliced connections, both "sides" are sockets, and for many purposes how we deal with each side is symmetric. Currently, however, we track the information for each side in independent fields in the structure, meaning we can't easily exploit that symmetry. This makes a number of reorganizations of the tcp splice code so that we can explot that symmetry to reduce code size. This will have some additional advantages when we come to integrate with the in-progress unified flow table. Based on top of the interface identifiers and automatic forwarding cleanup series. Changes since v1: * Small updates to comments and commit messages David Gibson (11): tcp_splice: Remove redundant tcp_splice_epoll_ctl() tcp_splice: Correct error handling in tcp_splice_epoll_ctl() tcp_splice: Don't handle EPOLL_CTL_DEL as part of tcp_splice_epoll_ctl() tcp_splice: Remove unnecessary forward declaration tcp_splice: Avoid awkward temporaries in tcp_splice_epoll_ctl() tcp_splice: Don't pool pipes in pairs tcp_splice: Rename sides of connection from a/b to 0/1 tcp_splice: Exploit side symmetry in tcp_splice_timer() tcp_splice: Exploit side symmetry in tcp_splice_connect_finish() tcp_splice: Exploit side symmetry in tcp_splice_destroy() tcp_splice: Simplify selection of socket and pipe sides in socket handler tcp_conn.h | 45 +++--- tcp_splice.c | 389 +++++++++++++++++++++------------------------------ 2 files changed, 179 insertions(+), 255 deletions(-) -- 2.41.0

2 12

[PATCH v2 0/4] Passt Interface Identifiers
by David Gibson 07 Nov '23

07 Nov '23

For the generalisations to forwarding we want to accomplish with the flow table, we will need a more formalised notion of passt "interfaces" - that is way that passt communicates with network, whether that be over L4 sockets (as on the host) or via an L2 tunnel. This series makes a small start on this, by introducing a type to identify passt interfaces. We don't use it for a whole lot yet, but it will become more useful in future. Based on the automatic port forwarding cleanup series Changes since v1: * Don't use a type alias for pif ids, just use bare uint8_t David Gibson (4): udp: Clean up ref initialisation in udp_sock_init() pif: Introduce notion of passt/pasta interface pif: Record originating pif in listening socket refs pif: Pass originating pif to tap handler functions Makefile | 2 +- icmp.c | 4 +++- icmp.h | 4 ++-- passt.h | 1 + pif.h | 27 +++++++++++++++++++++++++++ tap.c | 26 ++++++++++++++++---------- tcp.c | 11 ++++++++--- tcp.h | 7 ++++--- tcp_splice.c | 10 ++++++---- udp.c | 35 ++++++++++++++++++----------------- udp.h | 11 ++++++----- 11 files changed, 92 insertions(+), 46 deletions(-) create mode 100644 pif.h -- 2.41.0

2 5

[PATCH v2 0/9] Clean ups to automatic port forwarding
by David Gibson 07 Nov '23

07 Nov '23

This contains an assortment of cleanups to the code supporting the automatic port forwarding more for past - specifically get_bound_ports() and related functions. This shouldn't mae any functional changes. Based on the series with fixes for new cppcheck-2.12 warnings. Changes since v1: * Some comment tweaks based on Stefano's feedback. David Gibson (9): conf: Cleaner initialisation of default forwarding modes port_fwd: Move automatic port forwarding code to port_fwd.[ch] port_fwd: Better parameterise procfs_scan_listen() util: Add open_in_ns() helper port_fwd: Pre-open /proc/net/* files rather than on-demand port_fwd: Don't NS_CALL get_bound_ports() port_fwd: Split TCP and UDP cases for get_bound_ports() port_fwd: Move port scanning /proc fds into struct port_fwd port_fwd: Simplify get_bound_ports_*() to port_fwd_scan_*() Makefile | 2 +- conf.c | 113 +++++-------------------------------------- conf.h | 1 - passt.h | 5 -- port_fwd.c | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++ port_fwd.h | 9 ++++ tcp.c | 39 +-------------- util.c | 118 +++++++++++++++++++++------------------------ util.h | 3 +- 9 files changed, 218 insertions(+), 211 deletions(-) create mode 100644 port_fwd.c -- 2.41.0

2 10