[PATCH 00/15] Test fixes, batch 4
This set supersedes the previous "batch 3" which had some problems. This fixes a number of problems I've encountered trying to run the passt tests on a Fedora host. It also makes a number of small cleanups and improvements to the test running framework. David Gibson (14): Handle the case of a DNS server on localhost tests: qemu-system-ppc64le isn't a thing Invoke specific qemu-system-* binaries tests: Introduce makefile for building test assets tests: Move mbuto download and execution to asset build tests: Search multiple places for aarch64 EDK2 bios image Clean up passt.pid file tests: Remove unused set_mode() function tests: Remove not-very-useful "req" directive tests: Don't automatically traverse directories of test files tests: Explicitly list test files in test/run, remove "onlyfor" support tests: Move distro image download to asset build makefile tests: Prepare distro images during asset build phase tests: Remove unused DNS6 calculation from fedora tests Stefano Brivio (1): test: Add external mbuto profile, drop udhcpc, and switch to it .gitignore | 1 + Makefile | 3 +- conf.c | 16 +++ passt.1 | 6 +- test/.gitignore | 6 ++ test/Makefile | 177 +++++++++++++++++++++++++++++++ test/build/all | 22 +++- test/build/install | 34 ------ test/demo/passt | 5 +- test/demo/pasta | 1 - test/demo/podman | 2 - test/dhcp/passt | 1 - test/dhcp/pasta | 1 - test/distro/debian | 124 +++------------------- test/distro/fedora | 206 ++++-------------------------------- test/distro/opensuse | 24 ++--- test/distro/ubuntu | 39 ++----- test/find-arm64-firmware.sh | 13 +++ test/icmp/passt_in_ns | 1 - test/lib/setup | 50 ++++----- test/lib/test | 35 ++---- test/lib/util | 13 --- test/ndp/passt | 1 - test/ndp/pasta | 1 - test/passt.mbuto | 34 ++++++ test/perf/passt_tcp | 1 - test/perf/passt_udp | 1 - test/perf/pasta_tcp | 1 - test/perf/pasta_udp | 1 - test/prepare-distro-img.sh | 18 ++++ test/run | 58 +++++----- test/tcp/passt | 1 - test/tcp/passt_in_ns | 1 - test/tcp/pasta | 1 - test/udp/passt | 1 - test/udp/passt_in_ns | 1 - test/udp/pasta | 1 - test/valgrind/passt | 1 - test/valgrind/passt_in_ns | 1 - 39 files changed, 405 insertions(+), 499 deletions(-) create mode 100644 test/Makefile delete mode 100644 test/build/install create mode 100755 test/find-arm64-firmware.sh create mode 100755 test/passt.mbuto create mode 100755 test/prepare-distro-img.sh -- 2.36.1
From: Stefano Brivio
By default, passt detects the nameserver used by the host system by reading
/etc/resolv.conf, and advertises that to the guest via DHCP. However this
breaks down if the host's nameserver is local (on 127.0.0.1 or ::1);
connecting to localhost on the guest won't reach the host's nameserver.
Using a local nameserver is a reasonably common case when using dnsmasq
or similar to merge name resolution on a home network with name resolution
from an organization-private VPN.
We already have the gateway mapping support to allow reaching host-local
services from the guest via the address of the default gateway. Add code
to detect the case of a local DNS server and use the gateway mapping to
advertise it usefully to the guest.
Signed-off-by: David Gibson
Several tests run pp64le guests using "qemu-system-ppc64le". But, at the
system level there's no difference between ppc64 and ppc64le - it's the
same hardware, just placed into different endian modes by OS early boot
code. Reflecting that, qemu only supplies a single "qemu-system-ppc64".
Some distros alias qemu-system-ppc64le to qemu-system-ppc64 (Debian does),
but it's best not to count on this (Fedora doesn't, for example).
Signed-off-by: David Gibson
A lot of tests and examples invoke qemu with the command "kvm". However,
as far as I can tell, "kvm" being aliased to the appropriate qemu system
binary is Debian specific. The binary names from qemu upstream -
qemu-system-$ARCH - also aren't universal, but they are more common (they
should be good for both Debian and Fedora at least).
In order to still get KVM acceleration when available, we use the option
"-M accel=kvm:tcg" to tell qemu to try using either KVM or TCG in that
order
A number of the places we invoked "kvm" are expecting specifically an x86
guest, and so it's also safer to explicitly invoke qemu-system-x86_64.
Some others appear to be independent of the target arch (just wanting the
same arch as the host to allow KVM acceleration). Although I suspect there
may be more subtle x86 specific options in the qemu command lines, attempt
to preserve arch independence by using $(uname -m).
Signed-off-by: David Gibson
A number of passt/pasta testcases have initial steps which are just about
building images or other assets we need for the test proper. Repeating
these for each test run can be quite costly.
This patch makes a start on moving this sort of test asset building to
a separate phase before running the tests proper. For now just add a
Makefile to handle the asset building (although it doesn't build
anything yet), and make the path where we'll be building the assets
available to the tests.
Signed-off-by: David Gibson
Move the download of mbuto and using it to create a sample initramfs to
the asset build makefile, rather than embedding it in the test scripts
themselves.
The two_guests tests used to use two separate copies of the mbuto
image. As an initramfs the mbuto image is strictly readonly though,
so that's not necessary. So, also use the same image for both guests.
Signed-off-by: David Gibson
Apparently qemu's ARM virt machine needs to be explicitly given a firmware
image, rather than just supplying a sane default. Unfortunately the EDK2
firmware image we need isn't in the same place on all host distros.
Currently the test scripts hardcode the Debian location, meaning it will
break on hosts that have it somewhere else. This patch searches multiple
locations for the firmware, and creates a local link during the asset build
phase, which the tests can then use.
For now it only searches the locations used by Debian and Fedora, but
that's a small improvement in robustness already, and can be later improved
further if we need to.
Signed-off-by: David Gibson
If the tests are interrupted at the right point a passt.pid file can be
left over. Clean it up with "make clean" and add it to .gitignore so it
doesn't get accidentally committed.
Signed-off-by: David Gibson
This utility function is never called.
Signed-off-by: David Gibson
The test scripts support a "req" directive which requires one test script
to be run before another. It's implemented by doing a topological sort
based on these directives in the runner scripts, which is about as awkward
as you'd expect in Bourne shell.
It turns out we only use this functionality in one place - to make the
"make install" test run after the plain "make" test. We also already have
a simpler way of making sure tests run in a specific order: just put them
into the same test script file.
So, remove support for the "req" directive and just fold the build/all and
build/install test scripts together.
Signed-off-by: David Gibson
The top level listing control of which tests to run is in test/run, however
it uses the test() function which runs an entire directory of test files,
filtered by some criteria. This makes it awkward to narrow down to a
subset of tests when debugging a specific failure.
To make this easier, have test() take an explicit list of test files to
run, and have the caller in test/run handle the directory traversal. The
construct we use for this is pretty awkward to handle the fact that we're
in the source tree root directory rather than test/ at this point in
test/run. Later cleanups will improve that.
Signed-off-by: David Gibson
Currently test/run uses wildcards to run all of the tests in a directory.
However, that wildcard list is filtered down by the "onlyfor" directives
in the test files... usually to a single file.
Therefore, just explicitly list the files we *really* want to run for this
test mode. This makes it easier to see at the top level what tests will
be executed, and to change that list temporarily while debugging specific
failures.
This means the "onlyfor" directive no longer has any purpose, and we can
remove it. "onlyfor" was also the only used of the $MODE variable, so we
can remove that too.
Signed-off-by: David Gibson
Rather than directly download distro images from the test scripts, handle
all the downloads during the test asset build, then just clone them for
the tests themselves. This avoids repeated downloads which can be very
slow when debugging failing tests.
Signed-off-by: David Gibson
Before booting the guest images, the distro test cases need to modify the
guest images, using virt-edit and guestfish, to boot in the way we need.
At present this gets repeated on every test run, even though it's not
really doing anything we want to test for.
In addition many of the images have the same preparation steps leading to
a lot of duplicated stages in the tests. A number of additional images can
be prepared using common steps, even if the ones used now have small
differences.
Therefore move the preparation of most of the guest images to the asset
build phase, where they can be done a single time for multiple test runs,
using a common preparation script. We can even avoid making a copy of the
disk image for booting, by using qemu's -snapshot option.
A few of the distros (openSUSE and older Ubuntu) do need different steps.
For now we don't chage how they are run, they could possibly be handled
more like this in future.
Signed-off-by: David Gibson
The Fedora test file extracts some information from the host resolv.conf
into a DNS6 variable which is then never used. Remove this unnecessary
step, which is presumably a leftover from an earlier iteration.
This was the only user of 'head' and 'sed' in the test file, so those can
also be removed from the required tools. The debian and ubuntu test files
also listed 'head' and 'sed' as tools, although they don't use them,
I'm guessing because of an earlier version which had the same DNS6 code.
Remove those as well.
The opensuse test file still actually uses DNS6, so leave it there for now.
The DNS handling and network config handling for SuSE looks to be kind of
broken, but fixing that is a job for another day.
Signed-off-by: David Gibson
On Wed, 6 Jul 2022 17:28:54 +1000
David Gibson
This set supersedes the previous "batch 3" which had some problems. This fixes a number of problems I've encountered trying to run the passt tests on a Fedora host. It also makes a number of small cleanups and improvements to the test running framework.
David Gibson (14): Handle the case of a DNS server on localhost tests: qemu-system-ppc64le isn't a thing Invoke specific qemu-system-* binaries tests: Introduce makefile for building test assets tests: Move mbuto download and execution to asset build tests: Search multiple places for aarch64 EDK2 bios image Clean up passt.pid file tests: Remove unused set_mode() function tests: Remove not-very-useful "req" directive tests: Don't automatically traverse directories of test files tests: Explicitly list test files in test/run, remove "onlyfor" support tests: Move distro image download to asset build makefile tests: Prepare distro images during asset build phase tests: Remove unused DNS6 calculation from fedora tests
Stefano Brivio (1): test: Add external mbuto profile, drop udhcpc, and switch to it
Thanks a lot, it all looks good to me, I would just drop 1/15 (mbuto profile) as I already replaced it with a v2 patch. I'm running the tests now, let's see. :) -- Stefano
On Mon, Jul 11, 2022 at 11:42:09AM +0200, Stefano Brivio wrote:
On Thu, 7 Jul 2022 16:53:11 +0200 Stefano Brivio
wrote: [...]
I'm running the tests now, let's see. :)
For some reason qemu refuses to boot the OpenSUSE Leap 15.1 image made this way, I haven't really looked into that yet.
Huh. I ran into a bunch of problems with the OpenSUSE Leap images when I tried to do the image preparation stuff for them. But when I rolled that back and just did the image download separate, it seemed to work for me. Well... that is, it boots up and executes commands okay. The test fails, I think because I have no IPv6 nameservers on the host, which is messing with the scripting here, but the same thing happens before the change to image downloads. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
On Tue, 12 Jul 2022 18:26:37 +1000
David Gibson
On Mon, Jul 11, 2022 at 11:42:09AM +0200, Stefano Brivio wrote:
On Thu, 7 Jul 2022 16:53:11 +0200 Stefano Brivio
wrote: [...]
I'm running the tests now, let's see. :)
For some reason qemu refuses to boot the OpenSUSE Leap 15.1 image made this way, I haven't really looked into that yet.
Huh. I ran into a bunch of problems with the OpenSUSE Leap images when I tried to do the image preparation stuff for them. But when I rolled that back and just did the image download separate, it seemed to work for me. Well... that is, it boots up and executes commands okay.
Ah, wait a moment, did you perhaps forget to commit the distro/opensuse part for patch 14/15? The changes for distro/debian and distro/fedora in that patch look complete, distro/ubuntu has changes just for two versions, and distro/opensuse is not changed by that patch at all. It took me a bit to figure out because the prepared images actually work, but the test doesn't use them -- it's trying to use a temporary file that doesn't exist anymore.
The test fails, I think because I have no IPv6 nameservers on the host, which is messing with the scripting here, but the same thing happens before the change to image downloads.
If you're wondering why I went that way with OpenSUSE: I was too lazy to find out how to configure the interface in a "proper" way, so I just bring eth0 up, which sets up IPv6 addresses and routes via NDP. I guess it simply needs: wicked ifup eth0 and then we can use both IPv4 and IPv6, but I haven't tried. -- Stefano
On Tue, Jul 12, 2022 at 02:13:07PM +0200, Stefano Brivio wrote:
On Tue, 12 Jul 2022 18:26:37 +1000 David Gibson
wrote: On Mon, Jul 11, 2022 at 11:42:09AM +0200, Stefano Brivio wrote:
On Thu, 7 Jul 2022 16:53:11 +0200 Stefano Brivio
wrote: [...]
I'm running the tests now, let's see. :)
For some reason qemu refuses to boot the OpenSUSE Leap 15.1 image made this way, I haven't really looked into that yet.
Huh. I ran into a bunch of problems with the OpenSUSE Leap images when I tried to do the image preparation stuff for them. But when I rolled that back and just did the image download separate, it seemed to work for me. Well... that is, it boots up and executes commands okay.
Ah, wait a moment, did you perhaps forget to commit the distro/opensuse part for patch 14/15? The changes for distro/debian and distro/fedora in that patch look complete, distro/ubuntu has changes just for two versions, and distro/opensuse is not changed by that patch at all.
No. I fiddled around with handling the SuSE and older Ubuntu images, got bogged down in details and decided to postpone that until a later time. So I'm deliberately only handling the images which can be handled by the common script I added there for now.
It took me a bit to figure out because the prepared images actually work, but the test doesn't use them -- it's trying to use a temporary file that doesn't exist anymore.
Um, what prepared images? AFAICT my Makefile only generates prepared images for Debian, Fedora and the newer Ubuntu cases. For SuSE and the older Ubuntu the test scripts now have qemu-img commands creating temporary images from the pre-downlaoded images (added in patch 13/15).
The test fails, I think because I have no IPv6 nameservers on the host, which is messing with the scripting here, but the same thing happens before the change to image downloads.
If you're wondering why I went that way with OpenSUSE: I was too lazy to find out how to configure the interface in a "proper" way, so I just bring eth0 up, which sets up IPv6 addresses and routes via NDP. I guess it simply needs:
wicked ifup eth0
and then we can use both IPv4 and IPv6, but I haven't tried.
Right. Maybe I'll look at that if I get time. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
On Wed, 13 Jul 2022 12:11:24 +1000
David Gibson
On Tue, Jul 12, 2022 at 02:13:07PM +0200, Stefano Brivio wrote:
On Tue, 12 Jul 2022 18:26:37 +1000 David Gibson
wrote: On Mon, Jul 11, 2022 at 11:42:09AM +0200, Stefano Brivio wrote:
On Thu, 7 Jul 2022 16:53:11 +0200 Stefano Brivio
wrote: [...]
I'm running the tests now, let's see. :)
For some reason qemu refuses to boot the OpenSUSE Leap 15.1 image made this way, I haven't really looked into that yet.
Huh. I ran into a bunch of problems with the OpenSUSE Leap images when I tried to do the image preparation stuff for them. But when I rolled that back and just did the image download separate, it seemed to work for me. Well... that is, it boots up and executes commands okay.
Ah, wait a moment, did you perhaps forget to commit the distro/opensuse part for patch 14/15? The changes for distro/debian and distro/fedora in that patch look complete, distro/ubuntu has changes just for two versions, and distro/opensuse is not changed by that patch at all.
No. I fiddled around with handling the SuSE and older Ubuntu images, got bogged down in details and decided to postpone that until a later time. So I'm deliberately only handling the images which can be handled by the common script I added there for now.
It took me a bit to figure out because the prepared images actually work, but the test doesn't use them -- it's trying to use a temporary file that doesn't exist anymore.
Um, what prepared images? AFAICT my Makefile only generates prepared images for Debian, Fedora and the newer Ubuntu cases. For SuSE and the older Ubuntu the test scripts now have qemu-img commands creating temporary images from the pre-downlaoded images (added in patch 13/15).
Oops, right, I missed that, I was confused by the fact that the image didn't exist. I guess DOWNLOAD_ASSETS (from patch 13/15) in test/Makefile should also include OPENSUSE_IMGS, that's all. Added, re-running the whole thing now. -- Stefano
On Wed, Jul 13, 2022 at 08:04:14AM +0200, Stefano Brivio wrote:
On Wed, 13 Jul 2022 12:11:24 +1000 David Gibson
wrote: On Tue, Jul 12, 2022 at 02:13:07PM +0200, Stefano Brivio wrote:
On Tue, 12 Jul 2022 18:26:37 +1000 David Gibson
wrote: On Mon, Jul 11, 2022 at 11:42:09AM +0200, Stefano Brivio wrote:
On Thu, 7 Jul 2022 16:53:11 +0200 Stefano Brivio
wrote: [...]
I'm running the tests now, let's see. :)
For some reason qemu refuses to boot the OpenSUSE Leap 15.1 image made this way, I haven't really looked into that yet.
Huh. I ran into a bunch of problems with the OpenSUSE Leap images when I tried to do the image preparation stuff for them. But when I rolled that back and just did the image download separate, it seemed to work for me. Well... that is, it boots up and executes commands okay.
Ah, wait a moment, did you perhaps forget to commit the distro/opensuse part for patch 14/15? The changes for distro/debian and distro/fedora in that patch look complete, distro/ubuntu has changes just for two versions, and distro/opensuse is not changed by that patch at all.
No. I fiddled around with handling the SuSE and older Ubuntu images, got bogged down in details and decided to postpone that until a later time. So I'm deliberately only handling the images which can be handled by the common script I added there for now.
It took me a bit to figure out because the prepared images actually work, but the test doesn't use them -- it's trying to use a temporary file that doesn't exist anymore.
Um, what prepared images? AFAICT my Makefile only generates prepared images for Debian, Fedora and the newer Ubuntu cases. For SuSE and the older Ubuntu the test scripts now have qemu-img commands creating temporary images from the pre-downlaoded images (added in patch 13/15).
Oops, right, I missed that, I was confused by the fact that the image didn't exist.
I guess DOWNLOAD_ASSETS (from patch 13/15) in test/Makefile should also include OPENSUSE_IMGS, that's all. Added, re-running the whole thing now.
Ah, dammit, you're right. In fact OPENSUSE_IMGS is in the list in 13/15, then it's incorrectly removed in 14/15. IIRC, I took it off the download list, because I got some weird results which I thought meant that just the download patch was breaking the opensuse tests. Eventually realized I was misinterpreting the results, but forgot to put it back in the list. Didn't catch it because the downloads were so slow I never removed the images to try the build from scratch. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
On Wed, 6 Jul 2022 17:28:54 +1000
David Gibson
This set supersedes the previous "batch 3" which had some problems. This fixes a number of problems I've encountered trying to run the passt tests on a Fedora host. It also makes a number of small cleanups and improvements to the test running framework.
David Gibson (14): Handle the case of a DNS server on localhost tests: qemu-system-ppc64le isn't a thing Invoke specific qemu-system-* binaries tests: Introduce makefile for building test assets tests: Move mbuto download and execution to asset build tests: Search multiple places for aarch64 EDK2 bios image Clean up passt.pid file tests: Remove unused set_mode() function tests: Remove not-very-useful "req" directive tests: Don't automatically traverse directories of test files tests: Explicitly list test files in test/run, remove "onlyfor" support tests: Move distro image download to asset build makefile tests: Prepare distro images during asset build phase tests: Remove unused DNS6 calculation from fedora tests
Stefano Brivio (1): test: Add external mbuto profile, drop udhcpc, and switch to it
Applied without 1/15 (superseded) and with minimal modifications to 13/15 (OPENSUSE_IMGS added to download assets in Makefile, dropped spurious -O options from xzcat command in distro/opensuse). Thanks! -- Stefano
participants (2)
-
David Gibson
-
Stefano Brivio