- dev - passt

[PATCH v2 0/4] New line reading implementation
by David Gibson 06 Jul '22

06 Jul '22

As discussed on our call, the current line_read() implementation is pretty ugly and has some definite bugs. This series replaces it with a new implementation which should be more solid and more readable. The new implementation is larger than I'd really like, but it turns out its fiddlier to handle the various edge cases than you might think. David Gibson (4): Add cleaner line-by-line reading primitives Parse resolv.conf with new lineread implementation Use new lineread implementation for procfs_scan_listen() Remove unused line_read() Makefile | 8 ++-- conf.c | 22 ++++++---- lineread.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lineread.h | 31 +++++++++++++++ util.c | 64 +++-------------------------- 5 files changed, 170 insertions(+), 70 deletions(-) create mode 100644 lineread.c create mode 100644 lineread.h -- 2.36.1

2 7

[PATCH] qrap: Don't rely on errno after perror(), and reset it before usage
by Stefano Brivio 06 Jul '22

06 Jul '22

In commit fca5e11773d0 ("qrap: Add probe retry on connection reset from passt for KubeVirt integration") I just used errno to check if the connection was reset on recv(), but perror() might set it to EINVAL if e.g. an underlying logging mechanism fails, so we won't actually catch the connection reset. And in case recv() returns 0, errno won't be set, but we're still using it without resetting it first, which leads to unpredictable results in that case. Reset errno before probing with connect(), send() and recv(), and save it for later checks before calling perror(). Fixes: fca5e11773d0 ("qrap: Add probe retry on connection reset from passt for KubeVirt integration") Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- qrap.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/qrap.c b/qrap.c index 4173281..d73cf6c 100644 --- a/qrap.c +++ b/qrap.c @@ -112,8 +112,8 @@ void usage(const char *name) */ int main(int argc, char **argv) { + int i, s, qemu_argc = 0, addr_map = 0, has_dev = 0, retry_on_reset, err; struct timeval tv = { .tv_sec = 0, .tv_usec = (long)(500 * 1000) }; - int i, s, qemu_argc = 0, addr_map = 0, has_dev = 0, retry_on_reset; char *qemu_argv[ARG_MAX], dev_str[ARG_MAX]; struct sockaddr_un addr = { .sun_family = AF_UNIX, @@ -249,14 +249,21 @@ retry: perror("setsockopt SO_SNDTIMEO"); snprintf(addr.sun_path, UNIX_PATH_MAX, UNIX_SOCK_PATH, i); - if (connect(s, (const struct sockaddr *)&addr, sizeof(addr))) + + errno = 0; + + if (connect(s, (const struct sockaddr *)&addr, sizeof(addr))) { + err = errno; perror("connect"); - else if (send(s, &probe, sizeof(probe), 0) != sizeof(probe)) + } else if (send(s, &probe, sizeof(probe), 0) != sizeof(probe)) { + err = errno; perror("send"); - else if (recv(s, &probe_r, 1, MSG_PEEK) <= 0) + } else if (recv(s, &probe_r, 1, MSG_PEEK) <= 0) { + err = errno; perror("recv"); - else + } else { break; + } /* FIXME: in a KubeVirt environment, libvirtd invokes qrap three * times in a strict sequence when a virtual machine needs to @@ -280,7 +287,7 @@ retry: * this FIXME will probably remain until the tool itself is * obsoleted. */ - if (retry_on_reset && errno == ECONNRESET) { + if (retry_on_reset && err == ECONNRESET) { retry_on_reset--; usleep(50 * 1000); goto retry; -- 2.35.1

1 0

[PATCH v2] test: Add external mbuto profile, drop udhcpc, and switch to it
by Stefano Brivio 06 Jul '22

06 Jul '22

This depends on a future change in mbuto to accept external profile files. Add a file defining what we need for tests and demos, dropping udhcpc and script as they're not needed anymore, and switch to it. Suggested-by: David Gibson <david(a)gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- v2: The dhclient script is still needed, keep it test/demo/passt | 2 +- test/lib/setup | 2 +- test/passt.mbuto | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+), 2 deletions(-) create mode 100755 test/passt.mbuto diff --git a/test/demo/passt b/test/demo/passt index ee4e7c2..7c69e94 100644 --- a/test/demo/passt +++ b/test/demo/passt @@ -56,7 +56,7 @@ say Let's create a small initramfs image for the guest. guest cd __TEMPDIR__ guest git clone git://mbuto.sh/mbuto guest cd mbuto -guest ./mbuto -f passt.img -p passt -c lz4 +guest ./mbuto -f passt.img -p __TEMPDIR__/passt/test/passt.mbuto -c lz4 sleep 2 nl diff --git a/test/lib/setup b/test/lib/setup index e57d97c..afa5101 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -34,7 +34,7 @@ setup_passt() { pane_run GUEST "git -C ${__mbuto_dir} clone git://mbuto.sh/mbuto" pane_status GUEST - pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p passt -c lz4 -f mbuto.img" + pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p test/passt.mbuto -c lz4 -f mbuto.img" pane_status GUEST rm -rf "${__mbuto_dir}" diff --git a/test/passt.mbuto b/test/passt.mbuto new file mode 100755 index 0000000..150c062 --- /dev/null +++ b/test/passt.mbuto @@ -0,0 +1,37 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# test/passt.mbuto - mbuto (https://mbuto.sh) profile for test images +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +PROGS="${PROGS:-ash,dash,bash ip mount ls insmod mkdir ln cat chmod lsmod + modprobe find grep mknod mv rm umount jq iperf3 dhclient busybox logger + sed tr chown sipcalc cut md5sum nc dd strace ping tail killall sleep + sysctl nproc tcp_rr tcp_crr udp_rr which tee seq bc}" + +KMODS="${KMODS:- virtio_net virtio_pci}" + +LINKS="${LINKS:- + ash,dash,bash /init + ash,dash,bash /bin/sh + ash,dash,bash /usr/bin/bash}" + +COPIES="${COPIES} + /sbin/dhclient-script" + +DIRS="${DIRS} /tmp" + +FIXUP="${FIXUP} + :> /etc/fstab + sh +m +" + +OUTPUT="KERNEL=__KERNEL__ +INITRD=__INITRD__ +" -- 2.35.1

1 0

[PATCH] mbuto: Fix for kernel modules with additional extensions in filename
by Stefano Brivio 06 Jul '22

06 Jul '22

David reports that on Fedora 36 mbuto fails to source the virtio_net module for the passt profile, because modules are compressed and they have a .ko.xz file extension. I tried this on Fedora Cloud Base 36-1.5 on x86_64, and mbuto actually locates the module, but it fails to parse the insmod commands from the dependencies as it expects a .ko at the end of any module file. I'm not sure that's the same issue David found, but start fixing this part at least. While at it: we're calling kmod_node always on the module name passed to kmod_add, instead of calling it for that module plus dependencies: fix this as well. Reported-by: David Gibson <david(a)gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- mbuto | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mbuto b/mbuto index 693aaaa..4ffe549 100755 --- a/mbuto +++ b/mbuto @@ -868,11 +868,12 @@ kmod_add() { ;; *) for __t in ${__f}; do - __t="$(${BASENAME} -- "${__t%*.ko}")" + __t="$(${BASENAME} -- "${__t}")" + __t="${__t%%.*}" if ${MODPROBE} -S "${KERNEL}" -d "${__d_opt}" -q \ --show-depends -- "$(${BASENAME} -- "${__t}")" >/dev/null; then - __src="$(${FIND} ${__find_path} -name "${__t}.ko")" + __src="$(${FIND} ${__find_path} -name "${__t}.*")" [ -z "${__src}" ] && continue __dst="${wd}${__src##${MODDIR}}" @@ -881,7 +882,7 @@ kmod_add() { "${CP}" -a "${__src}" "${__dst}" fi - kmod_node "${1}" + kmod_node "${__t}" fi done esac -- 2.35.1

2 1

More mbuto pain
by David Gibson 05 Jul '22

05 Jul '22

Now that the mbuto profile change is around, I tried getting the passt tests working. I fixed a couple of minor problems, before hitting another mbuto issue. On my host, mbuto isn't managing to find and include the virtio_net module. I think this is because on current Fedora the modules in /lib/modules are compressed. So it needs to include: /lib/modules/5.18.6-200.fc36.x86_64/kernel/drivers/net/virtio_net.ko.xz but I think it's only looking for a plain '.ko' file. I had a look at the mbuto code to see if I could fix it, but I'm afraid mbuto is a long way past the complexity level at which I find shell remotely comprehensible. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson

3 3

[PATCH] test: Add external mbuto profile, drop udhcpc, and switch to it
by Stefano Brivio 02 Jul '22

02 Jul '22

This depends on a future change in mbuto to accept external profile files. Add a file defining what we need for tests and demos, dropping udhcpc, plus udhcpc and dhclient scripts as they're not needed anymore, and switch to it. Suggested-by: David Gibson <david(a)gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/demo/passt | 2 +- test/lib/setup | 2 +- test/passt.mbuto | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+), 2 deletions(-) create mode 100755 test/passt.mbuto diff --git a/test/demo/passt b/test/demo/passt index 0ece04f..b67ed26 100644 --- a/test/demo/passt +++ b/test/demo/passt @@ -56,7 +56,7 @@ say Let's create a small initramfs image for the guest. guest cd __TEMPDIR__ guest git clone git://mbuto.sh/mbuto guest cd mbuto -guest ./mbuto -f passt.img -p passt -c lz4 +guest ./mbuto -f passt.img -p __TEMPDIR__/passt/test/passt.mbuto -c lz4 sleep 2 nl diff --git a/test/lib/setup b/test/lib/setup index 0b6d915..cfbac3c 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -34,7 +34,7 @@ setup_passt() { pane_run GUEST "git -C ${__mbuto_dir} clone git://mbuto.sh/mbuto" pane_status GUEST - pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p passt -c lz4 -f mbuto.img" + pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p test/passt.mbuto -c lz4 -f mbuto.img" pane_status GUEST rm -rf "${__mbuto_dir}" diff --git a/test/passt.mbuto b/test/passt.mbuto new file mode 100755 index 0000000..5cafb30 --- /dev/null +++ b/test/passt.mbuto @@ -0,0 +1,34 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# test/passt.mbuto - mbuto (https://mbuto.sh) profile for test images +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +PROGS="${PROGS:-ash,dash,bash ip mount ls insmod mkdir ln cat chmod lsmod + modprobe find grep mknod mv rm umount jq iperf3 dhclient busybox logger + sed tr chown sipcalc cut md5sum nc dd strace ping tail killall sleep + sysctl nproc tcp_rr tcp_crr udp_rr which tee seq bc}" + +KMODS="${KMODS:- virtio_net virtio_pci}" + +LINKS="${LINKS:- + ash,dash,bash /init + ash,dash,bash /bin/sh + ash,dash,bash /usr/bin/bash}" + +DIRS="${DIRS} /tmp" + +FIXUP="${FIXUP} + :> /etc/fstab + sh +m +" + +OUTPUT="KERNEL=__KERNEL__ +INITRD=__INITRD__ +" -- 2.35.1

2 3

[PATCH 0/5] Test fixes, batch 3
by David Gibson 28 Jun '22

28 Jun '22

Here's another batch of fixes to get more passt/pasta tests running on my system, this time focused on the distro test cases. A number of those tests now run on my Fedora 36 host. Unfortunately they still fail on the Fedora 30, aarch64 test for reasons I haven't figured out (one of the yum downloads times out with passt but not -net user). This series is based on my earlier series adding the new "lineread" subsystem. David Gibson (5): tests: qemu-system-ppc64le isn't a thing Invoke specific qemu-system-* binaries tests: Cache downloaded guest images Handle the case of a DNS server on localhost tests: Try multiple paths for aarch64 EDK2 bios image conf.c | 16 +++++++ passt.1 | 6 +-- test/.gitignore | 1 + test/demo/passt | 2 +- test/distro/debian | 60 +++++++++++--------------- test/distro/fedora | 100 +++++++++++++++++++------------------------ test/distro/opensuse | 23 +++++----- test/distro/ubuntu | 25 +++++------ test/lib/setup | 15 +++++-- test/lib/test | 12 ++++++ test/run | 4 ++ 11 files changed, 140 insertions(+), 124 deletions(-) -- 2.36.1

1 6

[PATCH 0/4] New line reading implementation
by David Gibson 24 Jun '22

24 Jun '22

As discussed on our call, the current line_read() implementation is pretty ugly and has some definite bugs. This series replaces it with a new implementation which should be more solid and more readable. The new implementation is larger than I'd really like, but it turns out its fiddlier to handle the various edge cases than you might think. David Gibson (4): Add cleaner line-by-line reading primitives Parse resolv.conf with new lineread implementation Use new lineread implementation for procfs_scan_listen() Remove unused line_read() Makefile | 8 ++-- conf.c | 22 +++++++---- lineread.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lineread.h | 23 ++++++++++++ util.c | 64 +++---------------------------- 5 files changed, 155 insertions(+), 70 deletions(-) create mode 100644 lineread.c create mode 100644 lineread.h -- 2.36.1

2 6

Weird timeout issue with passt
by David Gibson 23 Jun '22

23 Jun '22

I alluded to this in my last patchset, but here's the information I've gathered on the current problem I'm hitting running the passt distro tests. It's pretty weird. On at least two occasions the tests have stalled during the Fedora 30, aarch64 test, with the guest getting a timeout downloading the package lists before installing new packages. For me this is selecting the mirror mirror.2degrees.nz. I'm not sure which factors are are relevant to reproducing the problem though. * The problem seems to me that the download suddenly stops progressing * partway through, causing dnf to eventually time out * If I manually try a "dnf clean all && dnf makecache -v" using the same guest image, it doesn't fail every time, but it fails significantly more often than not * It doesn't fail on the same file every time * I haven't been able to reproduce manually downloading the failing file with curl (tried repeatedly) * If I restrict dnf to a single repository rather than the whole set, I haven't managed to reproduce the problem * If I use qemu's -net user slirp instead of passt with the same disk image , I haven't been able to reproduce the problem (tried a bunch of times) * I've reproduced with the guest using both IPv4 and IPv6 * I have reproduced what looks like the same problem with an x86 guest image under KVM (also Fedora 30), but it seems to happen much less often (seen once in 10 or more attempts) * Seems to reproduce fairly readily with an x86 guest under TCG though, so I'm guessing the difference is timing related. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson

2 1

[PATCH 0/6] Assorted makefile cleanups
by David Gibson 20 Jun '22

20 Jun '22

These are not directly apropose anything, but I encountered a number of minor uglies in the Makefiles while I was working on other stuff, so heres a bunch of cleanups. David Gibson (6): Makefile: Avoid using wildcard sources Makefile: Use $(BIN) and $(MANPAGES) variable to simplify several targets Makefile: Simplify pasta* targets with a pattern rule Makefile: Tweak $(RM) usage Makefile: Don't create extraneous -.s file Makefile: Spell prefix as PREFIX Makefile | 77 +++++++++++++++++++++++++++--------------------------- seccomp.sh | 5 ++-- 2 files changed, 41 insertions(+), 41 deletions(-) -- 2.36.1

2 11