November 2022 - dev

[PATCH 00/10] RFC: Preliminaries for using share IPv4 & IPv6 sockets
by David Gibson 08 Nov '22

08 Nov '22

We want to use the same sockets to listen on both IPv4 and IPv6 addresses (at least on Linux which allows this). This isn't too complicated conceptually, but to make it work we need to unify handling of v4 and v6 connections more than we currently have. This isn't really ready to go, but I'm posting for reference. It's divided into rather a lot of steps, because I kept hitting problems and not being sure exactly what change had broken things. Based on my earlier series making handling of IPv4 addresses endian safer. David Gibson (10): tcp: no v6 flag in ref tcp: Helper to encode IPv4-mapped IPv6 addresses tcp: Partially unify IPv4 and IPv6 paths in tcp_hash_match() tcp: Hash IPv4 and IPv4-mapped-IPv6 addresses the same tcp: Take tcp_hash_insert() address from struct tcp_conn tcp: Unify IPv4 and IPv6 paths for hashing and matching tcp: Remove ugly address union from struct tcp_conn tcp: Unify initial sequence numbers for IPv4 and IPv6 tcp: Have tcp_seq_init() take its parameters from struct tcp_conn tcp: Fix small error in tcp_seq_init() time handling siphash.c | 2 + tcp.c | 202 ++++++++++++++++++--------------------------------- tcp.h | 1 - tcp_splice.c | 13 ++-- util.c | 33 +++++++++ util.h | 2 + 6 files changed, 116 insertions(+), 137 deletions(-) -- 2.38.1

2 21

[PATCH] test: Wait for network before starting passt in two_guests setup
by Stefano Brivio 07 Nov '22

07 Nov '22

As pasta now configures that target network namespace with --config-net, we need to wait for addresses and routes to be actually present. Just sending netlink messages doesn't mean this is done synchronously. A more elegant alternative, which probably makes sense regardless of this test setup, would be to query, from pasta, addresses and routes we added, and wait until they're there, before proceeding. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/lib/setup | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/lib/setup b/test/lib/setup index ab6d8d5..07d5056 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -208,6 +208,8 @@ setup_two_guests() { __ifname="$(context_run qemu_1 "ip -j link show | jq -rM '.[] | select(.link_type == \"ether\").ifname'")" + sleep 1 + __opts= [ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/passt_1.pcap" [ ${DEBUG} -eq 1 ] && __opts="${__opts} -d" -- 2.35.1

2 4

[PATCH] pasta: Workaround: wait for execvp() to be done in child before entering netns
by Stefano Brivio 07 Nov '22

07 Nov '22

This happens about every third time on the two_guests/basic test, and on that test only: we clone() twice, first to spawn a child, then to spawn a thread to check that we can enter the target network namespace. In this thread, we open a file descriptor associated to the target namespace. It might happen that it doesn't exist yet: the kernel can legitimately take its time to create one, after clone(). In this case, at least on a 5.15 Linux kernel, trying to open that file again always yields EACCES, and we get stuck there. This only occurs if we spawn two instances of pasta very close together, as it's done in the two_guests/basic case. I couldn't figure out what the race condition is, yet, and especially if it's a kernel issue or something we're doing wrong. However, if we wait until the execvp() in the child is done, the issue disappears. I'm not sure yet if it's just because of timing and this is hiding an unrelated race condition. The workaround consists of checking /proc/PID/exe against our own. If it's different, that means execvp() already completed and we can proceed. It's rather ugly, but much better than the alternative. Leave a FIXME there for the moment being. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- pasta.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/pasta.c b/pasta.c index db86317..36072b2 100644 --- a/pasta.c +++ b/pasta.c @@ -81,9 +81,26 @@ void pasta_child_handler(int signal) */ static int pasta_wait_for_ns(void *arg) { + char ns_exe_link[PATH_MAX], ns[PATH_MAX]; struct ctx *c = (struct ctx *)arg; int flags = O_RDONLY | O_CLOEXEC; - char ns[PATH_MAX]; + char exe[PATH_MAX] = { 0 }; + + /* FIXME: Why do we have to wait until execvp() is done in the child? + * If we don't, and the first call to open() below returns ENOENT, any + * subsequent call to it returns EACCES, at least on Linux 5.15, even + * though the observed PID is correct, and another process can open that + * path, and call setns() on that. + */ + snprintf(ns_exe_link, PATH_MAX, "/proc/%i/exe", pasta_child_pid); + if (readlink("/proc/self/exe", exe, PATH_MAX - 1) != -1) { + char ns_exe[PATH_MAX] = { 0 }; + + do { + if (readlink(ns_exe_link, ns_exe, PATH_MAX - 1) == -1) + break; + } while (!strncmp(exe, ns_exe, PATH_MAX - 1)); + } snprintf(ns, PATH_MAX, "/proc/%i/ns/net", pasta_child_pid); do -- 2.35.1

2 3

passt: new version 2022_11_04.e308018 available
by Stefano Brivio 04 Nov '22

04 Nov '22

The new version with tag 2022_11_04.e308018 includes the following changes: e308018 test/memory/passt: Change passt.avx2 path to /bin in test itself 11efaef passt, qrap, README: Update notes and documentation for AF_UNIX support in qemu 6c16977 test/perf: Finally drop workaround for virtio_net TX stall 667397d test: Switch to qemu -netdev stream option instead of using qrap 1401962 test: Wait for network before starting passt in two_guests setup db74679 udp: Check for answers to forwarded DNS queries before handling local redirects 73f50a7 conf: Split the notions of read DNS addresses and offered ones 7656a6f conf: Adjust netmask on mismatch between IPv4 address/netmask and gateway de93acb tcp: Correct function comments for address types f7653a1 Use endian-safer typing in struct tap4_l4_t 7c7b68d Use typing to reduce chances of IPv4 endianness errors dd3470d Use IPV4_IS_LOOPBACK more widely dd09cce Minor improvements to IPv4 netmask handling 2b793d9 Correct some missing endian conversions of IPv4 addresses 40fc9e6 test: Add memory/passt test cases ce2a0a5 test/lib: Add "td" directive, handled by table_value() bfd311a test/lib/perf_report: Use own flag to track initialisation 2d4468e tap: Support for detection of existing sockets on ramfs e76e65a test/lib: Move screen-scraping setup and layout functions to _ugly files ea5e046 README: Add Podman, vhost-user links, and links to Bugzilla queries 10cabe3 passt.1: Fix typo: "addressses", reported by Lintian https://passt.top/passt/log/?qt=range&q=2022_10_26.f212044..2022_11_04.e308… Packages: - Copr (CentOS Stream, EPEL, Fedora, Mageia, openSUSE): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/5010835/ permanent mirror: https://passt.top/builds/copr/0^20221104.ge308018/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_0e308018-1_all.deb - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Package for other RPM-based distributions, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt-ge308018-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ -- Stefano

1 0

[PATCH v2 0/6] Improve IPv4 address endian handling and related bug fixes
by David Gibson 04 Nov '22

04 Nov '22

While working on allowing us to use IPv4/IPv6 combined listening sockets, I made several endian mistakes. So, I took a diversion to make it harder to make such mistakes. Along the way I found some small existing endian and other bugs in IPv4 address handling. Changes since v1: * Added extra patch making more safety conversion * Added extra patch making some tangentially related comment fixes David Gibson (6): Correct some missing endian conversions of IPv4 addresses Minor improvements to IPv4 netmask handling Use IPV4_IS_LOOPBACK more widely Use typing to reduce chances of IPv4 endianness errors Use endian-safer typing in struct tap4_l4_t tcp: Correct function comments for address types checksum.c | 7 ++-- checksum.h | 3 +- conf.c | 118 ++++++++++++++++++++++++++++++----------------------- dhcp.c | 15 ++++--- icmp.c | 5 +-- passt.c | 2 +- passt.h | 16 ++++---- pasta.c | 7 +--- tap.c | 49 +++++++++++----------- tap.h | 8 ++-- tcp.c | 60 +++++++++++++-------------- tcp.h | 2 +- udp.c | 30 +++++++------- udp.h | 2 +- util.h | 12 +++++- 15 files changed, 182 insertions(+), 154 deletions(-) -- 2.38.1

1 6

[PATCH 0/3] Prepare for native AF_UNIX socket support in qemu 7.2
by Stefano Brivio 04 Nov '22

04 Nov '22

Patches 1/3 and 2/3 deal with the tests, patch 3/3 updates usage notes and documentation bits. Indeed, we'll need to keep qrap around for a little longer, but it's probably time to start telling users about this. Stefano Brivio (3): test: Switch to qemu -netdev stream option instead of using qrap test/perf: Finally drop workaround for virtio_net TX stall passt, qrap, README: Update notes and documentation for AF_UNIX support in qemu README.md | 10 ++++----- passt.1 | 8 ++----- qrap.1 | 13 +++++------- qrap.c | 5 +++-- tap.c | 10 ++++----- test/README.md | 5 +++++ test/demo/passt | 9 +++----- test/distro/debian | 24 ++++++++++----------- test/distro/fedora | 38 ++++++++++++++++----------------- test/distro/opensuse | 14 ++++++------- test/distro/ubuntu | 12 +++++------ test/lib/setup | 50 ++++++++++++++++++++------------------------ test/perf/passt_tcp | 15 ------------- test/perf/passt_udp | 15 ------------- 14 files changed, 93 insertions(+), 135 deletions(-) -- 2.35.1

1 3

[PATCH v4 0/5] Test cases to display memory usage in kernel and userspace
by Stefano Brivio 04 Nov '22

04 Nov '22

Note that 2/5 is not actually needed for the purposes of these "test cases" (which always succeed, at the moment), it just fixes a condition I hit while writing them. v4: Fold changes by David: use a dummy interface for the purpose of starting passt, and drop setup and teardown steps related to the "outer" instance of passt, now gone. Install passt in /bin instead of /usr/bin, but also add a link v3: In 5/5, drop useless conditionals and teardown steps as pointed out by David v2: Actually include passt.mem.mbuto in 5/5, and move the introduction of INITRAMFS_MEM for the setup function there Stefano Brivio (5): test/lib: Move screen-scraping setup and layout functions to _ugly files tap: Support for detection of existing sockets on ramfs test/lib/perf_report: Use own flag to track initialisation test/lib: Add "td" directive, handled by table_value() test: Add memory/passt test cases tap.c | 3 +- test/Makefile | 5 +- test/lib/layout | 69 ---------------- test/lib/layout_ugly | 113 ++++++++++++++++++++++++++ test/lib/perf_report | 9 ++- test/lib/setup | 23 ------ test/lib/setup_ugly | 58 ++++++++++++++ test/lib/term | 28 +++++++ test/lib/test | 3 + test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ test/passt.mem.mbuto | 44 ++++++++++ test/run | 6 ++ 12 files changed, 451 insertions(+), 97 deletions(-) create mode 100644 test/lib/layout_ugly create mode 100755 test/lib/setup_ugly create mode 100644 test/memory/passt create mode 100755 test/passt.mem.mbuto -- 2.35.1

2 6

[PATCH v3 0/3] Fixes and workarounds for pasta with Podman in Google Cloud
by Stefano Brivio 03 Nov '22

03 Nov '22

These patches work around or fix some issues found while testing the Podman integration for pasta in Google Compute Engine environments. v3: Fix up botched rebase, especially in 2/3, further reword 2/3, fix up 1/3 to match the recent patch from David that stores IPv4 netmasks as prefix lengths instead of masks v2: Rephrase commit messages of 1/3 and 2/3 adding examples with addresses. In 2/3, set host addresses (dns4 and dns6) and increase the pointer regardless of what we're doing with dns4_send and dns6_send. Stefano Brivio (3): conf: Adjust netmask on mismatch between IPv4 address/netmask and gateway conf: Split the notions of read DNS addresses and offered ones udp: Check for answers to forwarded DNS queries before handling local redirects conf.c | 74 ++++++++++++++++++++++++++++++++++++++++++++------------ dhcp.c | 4 +-- dhcpv6.c | 5 ++-- ndp.c | 6 ++--- passt.h | 8 ++++-- udp.c | 22 ++++++++--------- 6 files changed, 84 insertions(+), 35 deletions(-) -- 2.35.1

1 3

[PATCH v2 0/3] Fixes and workarounds for pasta with Podman in Google Cloud
by Stefano Brivio 03 Nov '22

03 Nov '22

These patches work around or fix some issues found while testing the Podman integration for pasta in Google Compute Engine environments. v2: Rephrase commit messages of 1/3 and 2/3 adding examples with addresses. In 2/3, set host addresses (dns4 and dns6) and increase the pointer regardless of what we're doing with dns4_send and dns6_send. Stefano Brivio (3): conf: Consistency check between configured IPv4 netmask and gateway conf: Split the notions of read DNS addresses and offered ones udp: Check for answers to forwarded DNS queries before handling local redirects conf.c | 64 +++++++++++++++++++++++++++++++++++++++++++------------- dhcp.c | 4 ++-- dhcpv6.c | 5 +++-- ndp.c | 6 +++--- passt.h | 8 +++++-- udp.c | 22 +++++++++---------- 6 files changed, 74 insertions(+), 35 deletions(-) -- 2.35.1

1 4

[PATCH v3 0/5] Test cases to display memory usage in kernel and userspace
by Stefano Brivio 03 Nov '22

03 Nov '22

Note that 2/5 is not actually needed for the purposes of these "test cases" (which always succeed, at the moment), it just fixes a condition I hit while writing them. v3: In 5/5, drop useless conditionals and teardown steps as pointed out by David v2: Actually include passt.mem.mbuto in 5/5, and move the introduction of INITRAMFS_MEM for the setup function there Stefano Brivio (5): test/lib: Move screen-scraping setup and layout functions to _ugly files tap: Support for detection of existing sockets on ramfs test/lib/perf_report: Use own flag to track initialisation test/lib: Add "td" directive, handled by table_value() test: Add memory/passt test cases tap.c | 3 +- test/Makefile | 5 +- test/lib/layout | 69 ---------------- test/lib/layout_ugly | 117 +++++++++++++++++++++++++++ test/lib/perf_report | 9 ++- test/lib/setup | 23 ------ test/lib/setup_ugly | 72 +++++++++++++++++ test/lib/term | 28 +++++++ test/lib/test | 3 + test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ test/passt.mem.mbuto | 42 ++++++++++ test/run | 6 ++ 12 files changed, 467 insertions(+), 97 deletions(-) create mode 100644 test/lib/layout_ugly create mode 100755 test/lib/setup_ugly create mode 100644 test/memory/passt create mode 100755 test/passt.mem.mbuto -- 2.35.1

2 7