September 2022 - dev

[PATCH] test/passt.mbuto: Don't fail on missing guest public key
by Stefano Brivio 23 Sep '22

23 Sep '22

We won't necessarily run mbuto as part of regular tests: it can also be used for demos or out-of-tree tests. To keep the profile simple, leave the whole sshd setup there, which is otherwise harmless, but don't fail if guest-key.pub is missing in the current directory. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/passt.mbuto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/passt.mbuto b/test/passt.mbuto index d29f456..51c9ada 100755 --- a/test/passt.mbuto +++ b/test/passt.mbuto @@ -68,7 +68,7 @@ EOF ln -s /run/sshd /usr/share/empty.sshd cat > /root/.ssh/authorized_keys <<EOF -'"$(cat guest-key.pub)"' +'"$(cat guest-key.pub 2>/dev/null || :)"' EOF chmod 600 /root/.ssh/authorized_keys chmod 700 /root -- 2.35.1

1 0

[PATCH 0/7] Fixes and workarounds for tests, Coverity warnings
by Stefano Brivio 23 Sep '22

23 Sep '22

With this series, I'm almost able to run the full test suite together with the new command dispatch mechanism. I still hit frequent failures in the passt_tcp performance test, so I'm not pushing this out at the moment, but as it's taking me a while, I'd rather share this earlier. Included are also two fixes for harmless (but ugly) issues discovered by Coverity. Stefano Brivio (7): test/perf: Always use /sbin/sysctl in tcp test test/perf: Switch performance test duration to 10 seconds instead of 30 tap: Check return value of accept4() before calling getsockopt() conf, tcp, udp: Arrays for ports need 2^16 values, not 2^16-8 test/lib: Restore IFS while executing directives in def blocks test/lib: Wait on iperf3 clients to be done, then send SIGINT to servers test/perf: Disable periodic throughput reports to avoid vhost hang conf.c | 2 +- tap.c | 6 ++++-- tcp.h | 4 ++-- test/lib/test | 20 ++++++++++---------- test/perf/passt_tcp | 10 +++++----- test/perf/passt_udp | 4 ++-- test/perf/pasta_tcp | 10 +++++----- test/perf/pasta_udp | 4 ++-- udp.h | 4 ++-- 9 files changed, 33 insertions(+), 31 deletions(-) -- 2.35.1

2 18

[PATCH 0/5] Some more hacks and workarounds to make tests pass again
by Stefano Brivio 23 Sep '22

23 Sep '22

Some of these are actual fixes, some are just hacks to keep tests running while we figure out issues. A part of this series should actually be reverted once we figure out why iperf3 clients and servers get stuck now and then. Stefano Brivio (5): test/lib: Run also iperf3 clients in background, revert to time-based wait test/lib: Wait for kernel to free up ports used by iperf3 before reusing them test/perf: Wait for neper servers in guest to be ready before starting client test/lib: Wait for DHCPv4 before starting DHCPv6 client in two_guests test test/distro: Update workarounds for Ubuntu 22.04 on s390x test/distro/ubuntu | 4 +++- test/lib/setup | 1 + test/lib/test | 11 ++++++++--- test/perf/passt_tcp | 4 ++++ test/perf/passt_udp | 2 ++ 5 files changed, 18 insertions(+), 4 deletions(-) -- 2.35.1

2 6

[PATCH] Makefile: Allow define overrides by prepending, not appending, CFLAGS
by Stefano Brivio 23 Sep '22

23 Sep '22

If we append CFLAGS to the ones passed via command line (if any), -D options we append will override -D options passed on command line (if any). For example, OpenSUSE build flags include -D_FORTIFY_SOURCE=3, and we want to have -D_FORTIFY_SOURCE=2, if and only if not overridden. The current behaviour implies we redefine _FORTIFY_SOURCE as 2, though. Instead of appending CFLAGS, prepend them by adding all the default build flags to another variable, a simply expanded one (defined with :=), named FLAGS, and pass that *before* CFLAGS in targets, so that defines from command line can override default flags. Reported-by: Dario Faggioli <dfaggioli(a)suse.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Makefile | 49 +++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/Makefile b/Makefile index af3d1ff..6805c23 100644 --- a/Makefile +++ b/Makefile @@ -23,13 +23,13 @@ AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/I[456]86/I386/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPC64/PPC/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPCLE/PPC64LE/') -CFLAGS += -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE -CFLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE -CFLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) -CFLAGS += -DNETNS_RUN_DIR=\"/run/netns\" -CFLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) -CFLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) -CFLAGS += -DARCH=\"$(TARGET_ARCH)\" +FLAGS := -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE +FLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE +FLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) +FLAGS += -DNETNS_RUN_DIR=\"/run/netns\" +FLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) +FLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) +FLAGS += -DARCH=\"$(TARGET_ARCH)\" PASST_SRCS = arch.c arp.c checksum.c conf.c dhcp.c dhcpv6.c icmp.c igmp.c \ isolation.c lineread.c mld.c ndp.c netlink.c packet.c passt.c pasta.c \ @@ -50,36 +50,36 @@ HEADERS = $(PASST_HEADERS) # from the pointer arithmetic used from the tcp_tap_handler() path to get the # remote connection address. ifeq ($(shell $(CC) -dumpversion),11) -ifneq (,$(filter -flto%,$(CFLAGS))) -ifneq (,$(filter -O2,$(CFLAGS))) - CFLAGS += -DTCP_HASH_NOINLINE - CFLAGS += -DSIPHASH_20B_NOINLINE +ifneq (,$(filter -flto%,$(FLAGS) $(CFLAGS))) +ifneq (,$(filter -O2,$(FLAGS) $(CFLAGS))) + FLAGS += -DTCP_HASH_NOINLINE + FLAGS += -DSIPHASH_20B_NOINLINE endif endif endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_snd_wnd = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_SND_WND + FLAGS += -DHAS_SND_WND endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_bytes_acked = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_BYTES_ACKED + FLAGS += -DHAS_BYTES_ACKED endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_min_rtt = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_MIN_RTT + FLAGS += -DHAS_MIN_RTT endif C := \#include <sys/random.h>\nint main(){int a=getrandom(0, 0, 0);} ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_GETRANDOM + FLAGS += -DHAS_GETRANDOM endif ifeq ($(shell :|$(CC) -fstack-protector-strong -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -fstack-protector-strong + FLAGS += -fstack-protector-strong endif prefix ?= /usr/local @@ -98,18 +98,19 @@ endif all: $(BIN) $(MANPAGES) docs -static: CFLAGS += -static -DGLIBC_NO_STATIC_NSS +static: FLAGS += -static -DGLIBC_NO_STATIC_NSS static: clean all seccomp.h: $(PASST_SRCS) $(PASST_HEADERS) @ EXTRA_SYSCALLS=$(EXTRA_SYSCALLS) ./seccomp.sh $^ passt: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) -passt.avx2: CFLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops +passt.avx2: FLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops passt.avx2: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(filter-out -O2,$(CFLAGS)) $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) + $(CC) $(filter-out -O2,$(FLAGS) $(CFLAGS)) \ + $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) passt.avx2: passt @@ -117,12 +118,12 @@ pasta.avx2 pasta.1 pasta: pasta%: passt% ln -s $< $@ qrap: $(QRAP_SRCS) passt.h - $(CC) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) valgrind: EXTRA_SYSCALLS="rt_sigprocmask rt_sigtimedwait rt_sigaction \ getpid gettid kill clock_gettime mmap munmap open \ unlink gettimeofday futex" -valgrind: CFLAGS:=-g -O0 $(filter-out -O%,$(CFLAGS)) +valgrind: FLAGS:=-g -O0 $(filter-out -O%,$(FLAGS)) valgrind: all .PHONY: clean @@ -261,7 +262,7 @@ clang-tidy: $(SRCS) $(HEADERS) -altera-struct-pack-align,\ -concurrency-mt-unsafe \ -config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \ - --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(CFLAGS)) + --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS)) ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1) TARGET := $(shell ${CC} -v 2>&1 | sed -n 's/Target: $.*$/\1/p') @@ -304,5 +305,5 @@ cppcheck: $(SRCS) $(HEADERS) --suppress=unmatchedSuppression:udp.c \ --suppress=unmatchedSuppression:util.c \ --suppress=unmatchedSuppression:util.h \ - $(filter -D%,$(CFLAGS)) \ + $(filter -D%,$(FLAGS) $(CFLAGS)) \ . -- 2.35.1

2 3

all pending patches pushed
by Stefano Brivio 23 Sep '22

23 Sep '22

Tests finally run reliably again, so I pushed all the pending patches. Sorry for the delay. Updated CI videos and test logs will appear in a while. -- Stefano

1 0

[PATCH 0/2] test: Fix two issues made apparent by new command dispatch
by Stefano Brivio 19 Sep '22

19 Sep '22

This is based on pending series by David, but they're logically separated fixes, so I'm sending them as a different series. Patch 2/2 adds some 'sleep 1' commands that were previously missing, which actually goes against the intention of speeding up those tests. However, without those, a number of tests fail quite frequently due to races between starting server and client, and I could never get a clean run with the new command dispatch mechanism otherwise. We can probably find a more elegant (and time-saving) mechanism to make sure the server is ready (based e.g. on parsing its output), but at the moment I'm just trying to make the test suite work again for me in its entirety. Stefano Brivio (2): test/perf: Check for /sbin/sysctl with which(1), not simply sysctl test/passt_in_ns: Consistent sleep commands before starting socat client test/passt_in_ns/tcp | 29 +++++++++++++++++++++++++---- test/passt_in_ns/udp | 18 ++++++++++++++++-- test/perf/passt_tcp | 4 ++-- test/perf/passt_udp | 2 +- test/perf/pasta_tcp | 2 +- 5 files changed, 45 insertions(+), 10 deletions(-) -- 2.35.1

2 13

[PATCH 0/2] doc/demo: Fix two issues seen on Fedora 36
by Stefano Brivio 17 Sep '22

17 Sep '22

Stefano Brivio (2): doc/demo: Drop /sbin from dhclient command, pass script file explicitly doc/demo: Clone and use mbuto in init namespace doc/demo.sh | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) -- 2.35.1

2 5

[PATCH] README: Update Availability and Try It sections with new packages
by Stefano Brivio 17 Sep '22

17 Sep '22

We now have official packages for Fedora, unofficial (Fedora Copr) for other common RPM-based distributions, and the existing packages with static builds for Debian, and for other RPM-based distributions. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- README.md | 57 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 2d7ae3c..fb09c6a 100644 --- a/README.md +++ b/README.md @@ -339,15 +339,19 @@ speeding up local connections, and usually requiring NAT. _pasta_: * ⌚ drop-in replacement for VPNKit (rootless Docker) ### Availability -* ✅ convenience unofficial packages for Debian, RPM-based distributions on - x86_64 (static builds) +* ✅ official [packages](https://src.fedoraproject.org/rpms/passt) for Fedora +* ✅ unofficial + [packages](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for CentOS + Stream, EPEL, Mageia, OpenSUSE +* ✅ unofficial packages from x86_64 static builds for other RPM-based + distributions and Debian * ✅ testing on non-x86_64 architectures (aarch64, armv7l, i386, ppc64, ppc64le, s390x) -* ✅ example Debian [package files](/passt/tree/contrib/debian), - example [`spec` file](/passt/tree/contrib/fedora) for Fedora +* ✅ example Debian [package files](/passt/tree/contrib/debian) * 🛠 official [OpenSUSE packages](https://build.opensuse.org/package/show/home:mnhauke/passt) -* ⌚ official packages for Debian, Fedora, etc. +* ⌚ official packages for Debian + ([RFP](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010498)), Arch Linux ### Services * ✅ built-in [ARP proxy](/passt/tree/arp.c) @@ -530,16 +534,19 @@ See also the [test logs](/builds/latest/test/). cd passt make - * alternatively, [static builds](/builds/latest/x86_64/) for x86_64 as of - the latest commit are also available for convenience. Non-official - packages for Debian (and derivatives) and RPM-based distributions are also - available there. These binaries and packages are simply built with: + * alternatively, install one of the available packages: - make pkgs + * [Debian and Debian-based](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) + * [Fedora](https://src.fedoraproject.org/rpms/passt) (official) + * [CentOS Stream](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [EPEL](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Mageia](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [OpenSUSE](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Other RPM-based distributions](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) - * or use one of the unofficial - [repositories](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for - CentOS Stream, EPEL, Fedora, Mageia and OpenSUSE Tumbleweed + Static binaries and packages are simply built with: + + make pkgs * have a look at the _man_ page for synopsis and options: @@ -565,10 +572,7 @@ See also the [test logs](/builds/latest/test/). echo working from/to the guest for IPv4 and IPv6 * to connect to a service on the VM, just connect to the same port directly - with the address of the network namespace. For example, to ssh to the guest, - from the main namespace on the host: - - ssh 192.0.2.2 + with the address of the current network namespace ### pasta @@ -578,16 +582,19 @@ See also the [test logs](/builds/latest/test/). cd passt make - * alternatively, [static builds](/builds/latest/x86_64/) for x86_64 as of - the latest commit are also available for convenience. Non-official - packages for Debian (and derivatives) and RPM-based distributions are also - available there. These binaries and packages are simply built with: + * alternatively, install one of the available packages: + + * [Debian and Debian-based](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) + * [Fedora](https://src.fedoraproject.org/rpms/passt) (official) + * [CentOS Stream](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [EPEL](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Mageia](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [OpenSUSE](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Other RPM-based distributions](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) - make pkgs + Static binaries and packages are simply built with: - * or use one of the unofficial - [repositories](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for - CentOS Stream, EPEL, Fedora, Mageia and OpenSUSE Tumbleweed + make pkgs * have a look at the _man_ page for synopsis and options: -- 2.35.1

1 0

[PATCH] Makefile: Include seccomp.h in HEADERS and require it for static checkers
by Stefano Brivio 14 Sep '22

14 Sep '22

Targets running static checkers (cppcheck and clang-tidy) need seccomp.h, but the latter is not included in HEADERS. Add it. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 6805c23..c47a5f6 100644 --- a/Makefile +++ b/Makefile @@ -42,7 +42,7 @@ MANPAGES = passt.1 pasta.1 qrap.1 PASST_HEADERS = arch.h arp.h checksum.h conf.h dhcp.h dhcpv6.h icmp.h \ isolation.h lineread.h ndp.h netlink.h packet.h passt.h pasta.h \ pcap.h siphash.h tap.h tcp.h tcp_splice.h udp.h util.h -HEADERS = $(PASST_HEADERS) +HEADERS = $(PASST_HEADERS) seccomp.h # On gcc 11.2, with -O2 and -flto, tcp_hash() and siphash_20b(), if inlined, # seem to be hitting something similar to: @@ -104,11 +104,11 @@ static: clean all seccomp.h: $(PASST_SRCS) $(PASST_HEADERS) @ EXTRA_SYSCALLS=$(EXTRA_SYSCALLS) ./seccomp.sh $^ -passt: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h +passt: $(PASST_SRCS) $(HEADERS) $(CC) $(FLAGS) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) passt.avx2: FLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops -passt.avx2: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h +passt.avx2: $(PASST_SRCS) $(HEADERS) $(CC) $(filter-out -O2,$(FLAGS) $(CFLAGS)) \ $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) -- 2.35.1

2 1

[PATCH] test: term: When checking if status line is a number, hide errors
by Stefano Brivio 14 Sep '22

14 Sep '22

We use the [ "$x" -eq "$x" ] syntax to check if $x is a number. The behaviour is clearly implied by POSIX, but some shells might actually report the (intended) error, and dash floods script.log with "Illegal number" error messages. Hide them. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/lib/term | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/lib/term b/test/lib/term index b31deac..eade2cd 100755 --- a/test/lib/term +++ b/test/lib/term @@ -232,7 +232,7 @@ pane_status() { [ ${DEMO} -eq 1 ] && return 0 __status="$(pane_parse "${1}")" - while ! [ "${__status}" -eq "${__status}" ]; do + while ! [ "${__status}" -eq "${__status}" ] 2>/dev/null; do sleep 1 pane_run "${1}" 'echo $?' pane_wait "${1}" -- 2.35.1

2 1