[PATCH v2 0/5] Sandbox test suite and enable podman tests on more hosts
I noticed the podman tests weren't actually running on my Fedora host. This turns out to be because cataonit is not in the path on Fedora (it's in /usr/libexec). While attempting to get this working with my "test in a box" script I ran into some additional problems: the podman tests downloaded and built podman, which requires external network access. That doesn't work in an isolated network environment. Changes since v1: * Test that podman is using the correct pasta binary * Added patch to prevent make cppcheck from checking the downloaded podman source as well. David Gibson (5): cppcheck: Explicitly give files to check test: Make sure to update mbuto repository test: Build and download podman as a test asset test: catatonit may not be in $PATH test: Verify that podman tests are using the pasta binary we expect Makefile | 2 +- seccomp.sh | 4 ++-- test/.gitignore | 1 + test/Makefile | 20 +++++++++++++++++--- test/pasta_podman/bats | 15 ++++++++++----- 5 files changed, 31 insertions(+), 11 deletions(-) -- 2.44.0
Currently "make cppcheck" invokes cppcheck on ".", so it will check all the
.c and .h files it can find in the source tree. This isn't ideal, because
it can find files that aren't actually part of the real build, or even
stale files which aren't in git.
More practically, some upcoming changes are looking at downloading other
source trees for some tests. Static errors in there is Not Our Problem,
so checking them is both slow and pointless.
So, change the Makefile to invoke cppcheck only on the specific source
files that are part of the build. For some reason in this format the
badBitmaskCheck warnings in seccomp.h which were suppressed by 5beb3472e
("cppcheck: Avoid errors due to zeroes in bitwise ORs") no longer trigger.
That means we get unmatchedSuppression warnings instead. We add an
unmatchedSuppression suppression instead of simply removing the original
suppressions, just in case this odd behaviour isn't the same for all
cppcheck versions.
Signed-off-by: David Gibson
We download and use mbuto to build trivial boot images for our VM tests.
However, if mbuto is already cloned, we won't update it to the current
version. Add some make logic to ensure that we do this.
Signed-off-by: David Gibson
The pasta_podman/bats test scrpt downloads and builds podman, then runs its
pasta specific tests. Downloading from within a test case has some
drawbacks:
* It can be very tedious if you have poor connectivity to the server
* It makes a test that's ostensibly for pasta itself dependent on the
state of the github server
* It precludes runnning the tests in an isolated network environment
The same concerns largely apply to building podman too, because it's pretty
common for Go builds to download dependencies themselves. Therefore move
the download and build of podman from the test itself, to the Makefile
where we prepare other test assets.
To avoid cryptic failures if something went wrong with the build, make
running the test dependent on having the built podman binary.
Signed-off-by: David Gibson
The pasta_podman/bats test script looks for 'catatonit' amongst other tools
to be avaiiliable on the host. However, while the podman tests do require
catatonit, it doesn't necessarily need to be in the regular path. For
example Fedora and RHEL place catatonit in /usr/libexec and podman finds it
there fine.
Therefore, remove it as an htools dependency.
Signed-off-by: David Gibson
Paul Holzinger pointed out that when we invoke the podman tests inside the
passt testsuite, the way we point podman at the newly built pasta binary
is kind of indirect. It's therefore prudent to check that podman is
actually using the binary we expect it to - in particular that it is using
the binary built in this tree, not some system installed pasta binary.
Suggested-by: Paul Holzinger
On Thu, 21 Mar 2024 15:57:37 +1100
David Gibson
I noticed the podman tests weren't actually running on my Fedora host. This turns out to be because cataonit is not in the path on Fedora (it's in /usr/libexec).
While attempting to get this working with my "test in a box" script I ran into some additional problems: the podman tests downloaded and built podman, which requires external network access. That doesn't work in an isolated network environment.
Changes since v1: * Test that podman is using the correct pasta binary * Added patch to prevent make cppcheck from checking the downloaded podman source as well.
David Gibson (5): cppcheck: Explicitly give files to check test: Make sure to update mbuto repository test: Build and download podman as a test asset test: catatonit may not be in $PATH test: Verify that podman tests are using the pasta binary we expect
Applied. -- Stefano
participants (2)
-
David Gibson
-
Stefano Brivio