Re: [PATCH 3/3] test: Re-implement pasta NDP tests using tunbridge & exeter

17 Oct 2025

      On Thu, Oct 16, 2025 at 11:31:19PM +0200, Stefano Brivio wrote:
...
On Fri, 10 Oct 2025 13:17:13 +1100
David Gibson  wrote:
...
On Fri, Oct 10, 2025 at 01:20:23AM +0200, Stefano Brivio wrote:
...
On Thu, 9 Oct 2025 15:47:01 +1100
David Gibson  wrote:
...
On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote:
...
On Wed, 8 Oct 2025 13:32:27 +1100
David Gibson  wrote:
...
On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote:    
> On Thu,  2 Oct 2025 17:57:08 +1000
> David Gibson  wrote:
>       
> > Convert the pasta NDP tests from shell and our own DSL to Python using
> > the exeter test protocol and tunbridge network simulation library.
> > 
> > Signed-off-by: David Gibson 
> > ---
> >  test/Makefile          |  2 +-
> >  test/pasta/dhcp        |  5 ++++
> >  test/pasta/ndp.py      | 59 ++++++++++++++++++++++++++++++++++++++++++
> >  test/run               |  6 +++--
> >  test/tasst/__init__.py |  4 +++
> >  test/tasst/pasta.py    | 40 ++++++++++++++++++++++++++++
> >  6 files changed, 113 insertions(+), 3 deletions(-)
> >  create mode 100755 test/pasta/ndp.py
> >  create mode 100644 test/tasst/pasta.py
> > 
> > diff --git a/test/Makefile b/test/Makefile
> > index f66c7e7e..95e3d75e 100644
> > --- a/test/Makefile
> > +++ b/test/Makefile
> > @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)
> >  
> >  EXETER_SH = smoke/smoke.sh build/static_checkers.sh
> >  EXETER_PYPATH = exeter/py3:tunbridge/:.
> > -EXETER_PYTHON = smoke/smoke.py build/build.py
> > +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py
> >  EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats)
> >  BATS_FILES = $(EXETER_BATS) \
> >  	podman/test/system/505-networking-pasta.bats
> > diff --git a/test/pasta/dhcp b/test/pasta/dhcp
> > index e1c66be6..61279fbf 100644
> > --- a/test/pasta/dhcp
> > +++ b/test/pasta/dhcp
> > @@ -18,6 +18,11 @@ test	Interface name
> >  nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
> >  check	[ -n "__IFNAME__" ]
> >  
> > +# Bring up the interface
> > +ns	ip link set dev __IFNAME__ up
> > +# Wait for SLAAC & DAD to complete
> > +ns	while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done
> > +
> >  test	DHCP: address
> >  ns	/sbin/dhclient -4 --no-pid __IFNAME__
> >  nsout	ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
> > diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py
> > new file mode 100755
> > index 00000000..8c7ce31e
> > --- /dev/null
> > +++ b/test/pasta/ndp.py
> > @@ -0,0 +1,59 @@
> > +#! /usr/bin/env python3
> > +#
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +#
> > +# test/pasta/ndp.py - pasta NDP functionality
> > +#
> > +# Copyright Red Hat
> > +# Author: David Gibson 
> > +
> > +import contextlib
> > +import dataclasses
> > +from typing import Iterator
> > +
> > +import exeter
> > +import tunbridge
> > +import tasst
> > +
> > +
> > +@dataclasses.dataclass
> > +class UnconfiguredScenario(exeter.Scenario):
> > +    """Tests for a pasta instance without --config-net"""
> > +
> > +    host: tunbridge.Site
> > +    guest: tunbridge.Site
> > +    ifname: str
> > +    addr6: tunbridge.ip.AddrMask6
> > +    gw6: tunbridge.ip.Addr6      
> 
> Until this point, it looks like stuff I can happily copy and paste,
> and grasp, even. But then:
>       
> > +    @exeter.scenariotest
> > +    def test_ifname(self) -> None:
> > +        ifs = tunbridge.ip.ifs(self.guest)
> > +        exeter.assert_eq(set(ifs), {'lo', self.ifname})      
> 
> ...why does a "Scenario" have a .ifname?
Yeah, the readability of the Scenario mechanism was something I was
particularly concerned about.  I think the concept is valuable, but
I'm very open to different ways of naming or organising it, if we can
up with something better.
From the description you give below, the name seems to fit.
...
A "Scenario" (specifically a subclass of exeter.Scenario) is a group
of tests with a common set of parameters.  In this case
UnconfiguredScenario is a bunch of tests about the behaviour of pasta
without --config-net.  Each of those tests has access to the host and
guest sites, the expected interface name, address and gateway in the
guest - that is, the contents of an UncofiguredScenario instance.
I'm not sure if I understand this correctly, but if each guest has a
single interface, that sounds a bit limiting.
Sorry, to be clear: a Scenario in the general sense can contain
whatever parameters you like.  This *particular* Scenario -
UnconfiguredScenario - has just those things, because those are all
that its tests require.
Ah, okay. Still, if I now want to take UnconfiguredScenario and add a
couple of dummy interfaces to it for a quick test, I guess I have the
choice to either do that with some "external" hack, or... copy and
rename it, so that it doesn't affect all the usages?
No.  A Scenario instance isn't responsible for managing the simulated
environment - that's the setup function - it's just conveying the
information about it that the tests need.  So, you can make a setup
function that adds the dummy interfaces, and still yield an
UnconfiguredScenario.  It doesn't need to have information about the
dummy interfaces because the tests carried by UnconfiguredScenario
don't care about them.
Oh, sorry, it's a class, of course, I see now.
...
The scenario mechanism does several things:
  1) Groups together some related (parameterized) tests
  2) Allows all of those tests to be registered at once
  3) Provides a mechanism for providing a bunch of information to
     those tests (without requiring them each to have a large set of
     direct parameters)
I'm aware that doing those things with the same construct may be
confusing - it's just ways of doing them separately also seem
confusing and/or awkward in their own ways.  Maybe there's a better
way, but I haven't spotted it yet.
It really is confusing to me, but the description above is rather clear
so I'll try to propose something once I get to write some kind of setup
function and test cases myself.
Makes sense.
...
...
...
...
...
Actually, I think any abstraction that doesn't offer arbitrary sets of
(and relationships between) the objects shown via netlink (or, at
least, namespaces, links, routes, addresses, neighbours) might be
limiting and not generic enough.
Absolutely, and the abstraction does allow that.
...
...
That instance describes a real (simulated) environment in which we can
run those tests.
You use this by supplying a function which sets things up, then yields
an UnconfiguredScenario instance describing what it set up.  exeter
will run all of the UnconfiguredScenario tests on the environment the
setup function created, each one as a separate test case.
This part is now clear to me, and I think it's not complicated to grasp
the concept vaguely but enough to copy, paste, and modify code doing
this.
Ok.
...
It would be even better to hide this entirely, because "yielding a
scenario" is a Python thing. In general, there's an imperative part in
all this (bordering functional programming, but still, not descriptive)
which I struggle to see as beneficial.
Here the tasks at hand are, roughly:
1. represent two network namespaces, with two interfaces each (loopback
   and non-loopback), with pasta connecting one of the interfaces of the
   inner one
There's a bit more to it than that - we need to specify the host's
routing setup, because that will affect what pasta does.  That's what
simple_host() is about, creating a host with the single gateway
routing that's our easiest / most common case.
Okay, sure, by "interfaces" I meant configured interfaces with
addresses and a default route, too. But that doesn't really modify my
point, that is:
...
...
2. bring up one of the interfaces
3. compare addresses
...and doing 1. like that is simply not... intuitive, I think.
I'm not really clear on what you're getting at here.  There is an
unavoidable tradeoff here between obviousness for a single case,
versus reuseability for multiple related cases.  Is it just that some
of the relevant setup is hidden inside simple_host() that's the
problem? Or is it something else?
...yes, one part is that it's hidden. Another part are, specifically,
these lines:
host: tunbridge.Site
  guest: tunbridge.Site
  ifname: str
[...]
@exeter.scenariotest
  def test_ifname(self) -> None:
[...]
None of these clearly links to "two network namespaces: A, with
interface a1 and address x1, ...".
Fair.  This needs a docstring explaining the parameters / fields.
That might help a tiny bit but I think the syntax and notations are
kind of self-explanatory.
My concern is at a more conceptual level, and it's better summarised
below, but here, specifically, we're writing:
host: tunbridge.Site
to say:
give me the "host" network namespace
What's "me" in this sentence?
...
and to say that, in my ideal world, I would probably go for something
on the line(s) of:
A
Not really following, I hope it will make more sense to me below.
...
...
...
I understand this is probably very close to the bare minimum you can
get by modelling this all with actual code, and that's why I think
actual (imperative/functional) code is usually not used to
model/describe things.
Imperative/functional code as opposed to..?
...declarative.
Ok, I've generally seen functional programming put under the
declarative heading.  Or as my exam invigilator famously said once
"Declaraytive Programming Paradijjums".

So, I guess that makes me unclear what does and doesn't count as
declarative for you.

[snip]
...
...
...
...
Syntax certainly isn't irrelevant, but so far I haven't grasped what
you dislike about the function syntax versus a specialized grammer.
Is it:
  - The irritating silly parentheses?
  - Longish (qualified) function names?
  - The indentation from the with syntax?
  - Something else?
It's *also* the first two
Ok.  First I can't easily change.  Second can be mitigated by handling
the imports differently.
...
(the indentation looks actually convenient),
Ok, good.  I also think this is useful because it conveys the lifetime
of each object, which will be important once we get to tests where you
need to change things part way through.
...
but that's not my main point. My main point is that this isn't
fundamentally declarative. You're turning it into something that
resembles that, but the syntax is still from an imperative programming
language.
And in my mind the main feature of a network (topology) simulator is
that you describe the topology (and it will build it for you), not that
you... have to build a description?
Using an example that's obviously familiar to you: think of taking a
device tree for some system with a couple of USB and I²C busses and a
flash controller, and writing all that in Python based on some form of
"bus" module/component.
I mean... old school Open Firmware kind of is this, but with Forth
instead of Python.
Okay, you can model data structures in Python, obviously, but that
wasn't my point. Anyway, it's all in the example below.
...
...
Once one sees how practical device trees are for that, the Python
version would look wrong, wouldn't it?
That really depends on the context.  If I was making an API for
building a device tree, I'd probably come up with something pretty
like this.
...an API for building one, yes. But not if you were writing a device
tree itself.
...
...
Now, while I think that some single bits of DTS syntax are
unnecessarily complicated, conceptually, a "networking" device tree
would look more usable to me than the approach you're taking.
Of course, we need the whole "testing" / exeter part as well, and test
cases are fundamentally sequential/imperative.
But (sorry, it's been a few years I don't touch these):
namespace@1 {
  interfaces {
    lo {
      address = 127.0.0.1;
    };
    eth0 {
      address = ...;
    };
  };
  routes {
    /* something simpler than ip -j ro sh ? */
  };
}
...
link@... {
  vxlan {
    endpoints {
      a {
         ns = <&namespace@1>;
      };
      b ...
...
this looks much more natural to me, as an input for a simulator (I
would personally make the syntax much more "elastic" by just throwing a
link into a namespace but I'm trying to keep it clean just for this
example).
Aha, I think I finally get what you're saying.  More below.
...
Maybe tunbridge implements this somewhere and I missed it? Or would
this be part of a "Scenario" description eventually?
This is entirely unrelated to what Scenario is trying to accomplish.
That may cause you to reconsider whether "Scenario" is a good name,
which is ok.
So.  A declarative way of defining networks would be nice to have.
From my perspective that's fundamental, rather. I gave it for granted.
...
I think doing it with the flexibility we want is much harder than you
estimate.
I'll pretend I'm not commenting on this line by... oops. :)
Heh.  Well, I'd love to be proved wrong on this one.
...
...
It looks easy for simple static situations like the
examples above, but:
* If you want to describe a topology that changes partway through,
   that's a huge step up in complexity, and kind of necessarily
   reintroduces imperative elements.
But you can use JSON or a ton of other formats that allow for ordering
of elements.
Just ordering network elements w.r.t. each other isn't enough.  If you
have a test where you want /this/ network topology - do some stuff -
then change the toplogy to /that/ - do some more stuff.  I'm not sure
how you encode that in a way that isn't imperative, or at least
pseudo-imperative.
...
Alternatively, one could add attributes for lifecycle and
timing (think of nftables sets) but it looks much less intuitive than
the alternative.
I wasn't previously familiar with nftables sets.  I had a quick look
at the docs, but I don't see how it relates to the current proposal.
...
...
Device tree absolutely suffers
   from this - that's what motivated the godawful runtime overlay
   mechanism, and right now, I'm struggling to find time to
   participate in the latest of many discussions about how to better
   handle devices which can be runtime added and removed.
I'm not suggesting that we use ANS Forth by the way.
Well, we agree on that at least :).
...
...
* If you want to build complex scenarios out of simpler ones, you
   need what amounts to a macro system.
There are a ton of ways. You can also use a filesystem and includes. Or
simply definitions of blocks, not necessarily macros,
You want to parameterise the blocks - at which point it's basically
macros.
...
and JSON
implicitly gives you all that.
No, it doesn't.  If you have a JSON sub-object that's repeated several
times through a tree, you have to write it out, in full, each time.
YAML does allow repeating, as do some other JSON extensions, but bare
JSON does not.  It still doesn't allow parameterisation of those
blocks.
...
As it's nothing security relevant, I would actually go with something
that's in theory more complicated but in practice more digestible such
as YAML.
I don't terribly like YAML, because I think there are a bunch of edge
cases where it's not obvious reading it whether something is a list or
object or somethine else.  The typing and back-referencing would be
useful for this problem, I'll grant you.
...
...
Again, a big leap up in
   complexity.  Device tree struggles with this too - it originated
   primarily as a machine->machine format, where having heaps of
   repeated information is fine.  As it transitioned to being a
   human->machine format, not so much.  Hence /include/, expression
   support and semi-standard running dts files through cpp before
   compilation.  It's still pretty clunky in this regard.
It absolutely is, but that's because it was designed for a different
purpose.
...
Plus.. I think the interpreter for this hypothetical declarative
language would need an internal structure pretty similar to what
tunbridge, so this is kind of already a first step towards it.
Okay, that's good to know.
I'm estimating I'm currently writing about 5-10 scripts per month,
including pasta/iproute2 one-liners, setting up strange stuff, to
reproduce / debug issues.
Given that this looks so fundamental for my usage I'm thinking that I
could make at least part of this a priority of mine.
I realised I can implement netlink stuff and handling of networking
configuration concepts quite fast with Rust and neli, so I'm pondering
to write a proof of concept that can parse the example above (minus
Forth notations, but with some kind of pointer) and make it create at
least namespaces, links, addresses, and routes.
If it helps visualising how that could possibly look like with / in
tunbridge itself, I'll take care of it soon rather than later.
The only little problem is that I'm much faster with Rust (because of
neli) than I can possibly picture myself with Python, and that doesn't
play along with tunbridge. But perhaps as a proof of concept it helps
anyway?
It would.  At earlier points I did consider writing tunbridge (or
whatever I was calling the idea at the time) in Rust.  In principle at
least, the lifetime tracking would make a very natural way for
ensuring you bring up / pull down things in a sensible order.

In practice, however, I almost instantly ran into intractable borrow
checker problems.  Specifically, Rust's notorious difficulty with
references between parts of the same structure.  That arises almost
immediately once you start building composite objects out of smaller
components:

	struct Node { ... }
	struct Veth<'a> { node0: &'a Node, node1: &'a Node, ... }

Then you want something that represents two nodes with a veth between
them and you get:

	struct TwoNodes { node0: Node, node1: Node, veth: Veth<'??> }

There are ways around it, of course, but everything I found so far was
either hideously unergonomic, depended on obscure crates or both.
There are a bunch of language extensions that would help, but while
much discussed, none have landed yet.

If a way to solve this nicely appears, I'm not against moving
tunbridge to Rust.  I mean, slightly more against it than I was months
ago before I made a start in Python, but still.
...
In general, do you think there's something in particular I could
contribute at this stage, if I want to see my declarative dream come
true?
Yes.  A bunch of pseudo-code examples - both the network declarations
and example tests that might go with them.

[snip]
...
...
...
...
...
...
Path I can easily add.  Version would require an extra invocation of
pasta, which I don't really want to do.
Ah, right, never mind. The path will be good enough for that.
...
> This part also looks
> quite readable and intuitive to me without having looked into tunbridge
> recently.
Ok, that's promising.
I mean, I think it's all usable for the moment, and perhaps a starting
point for some other kind of... front-end? I'm not sure. As I mentioned
I'm a bit worried about the potential for universal intuitiveness and
usability.
So am I, but I have to weigh it against being able to re-use both
tests and setups without having to re-express both in each case.
I think setups written like that are reusable (or can be made
reusable). My usability point is about other project/usages. For passt
and pasta themselves, this level or reusability looks enough to me for
the foreseeable future.
Even though, one day, I guess we might want to generate pseudo-random
(fractal-tree-like?) topologies (and I was recently trying out a
pasta-in-pasta-in-pasta-in-pasta-in-pasta setup to reproduce that
HTTP/FIN issue). For that, a declarative approach would make things
easier, I suppose.
Declarative, or imperative?  I actually have something like that in
tunbridge's selftests: a function that builds a stack of N nested
namespaces.
    https://gitlab.com/dgibson/tunbridge/-/blob/main/tunbridge/unshare.py#L302
...but they are all the same. Think, for example, of connecting every
odd-numbered pair with veth tunnels, and every even-numbered pair with
pasta. Say: n1 <-- veth --> n2 <-- pasta --> n3 <-- veth --> n4.
That would certainly be possible.  More complex, of course, but not
dramtically so.
...
What's really well suited for this situation, in my experience, is a
declarative description format that can be easily generated and
manipulated by imperative code.
Ah, so there's both an imperative and declarative component.  The idea
in tunbridge is that you can do this, but rather than emit the
"declarative part" as concrete text for another parser, it's emitted
as a data structure (generally a bunch of wrappers around context
managers).

It is true that as currently designed, tunbridge builds the data
structure representation at the same time as building the actual
simulated network.  With a declarative language approach, building the
description (language fragment) is separate from instantiating the
simulation.  Is that something you see as valuable?  Or only a side
effect of the other things about the declarative approach you like?
...
The name of this kind of "indirection" in computer science research
currently escapes me, but I'm fairly sure there must be some theory
about it. In any case, I can include something like this in my (now
planned) proof of concept.
-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson