On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote:

On Thu, 2 Oct 2025 17:57:08 +1000 David Gibson wrote:

...

Convert the pasta NDP tests from shell and our own DSL to Python using the exeter test protocol and tunbridge network simulation library.

Signed-off-by: David Gibson --- test/Makefile | 2 +- test/pasta/dhcp | 5 ++++ test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++++ test/run | 6 +++-- test/tasst/__init__.py | 4 +++ test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ 6 files changed, 113 insertions(+), 3 deletions(-) create mode 100755 test/pasta/ndp.py create mode 100644 test/tasst/pasta.py

diff --git a/test/Makefile b/test/Makefile index f66c7e7e..95e3d75e 100644 --- a/test/Makefile +++ b/test/Makefile @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)

EXETER_SH = smoke/smoke.sh build/static_checkers.sh EXETER_PYPATH = exeter/py3:tunbridge/:. -EXETER_PYTHON = smoke/smoke.py build/build.py +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats) BATS_FILES = $(EXETER_BATS) \ podman/test/system/505-networking-pasta.bats diff --git a/test/pasta/dhcp b/test/pasta/dhcp index e1c66be6..61279fbf 100644 --- a/test/pasta/dhcp +++ b/test/pasta/dhcp @@ -18,6 +18,11 @@ test Interface name nsout IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname' check [ -n "__IFNAME__" ]

+# Bring up the interface +ns ip link set dev __IFNAME__ up +# Wait for SLAAC & DAD to complete +ns while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done + test DHCP: address ns /sbin/dhclient -4 --no-pid __IFNAME__ nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py new file mode 100755 index 00000000..8c7ce31e --- /dev/null +++ b/test/pasta/ndp.py @@ -0,0 +1,59 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# test/pasta/ndp.py - pasta NDP functionality +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import dataclasses +from typing import Iterator + +import exeter +import tunbridge +import tasst + + +@dataclasses.dataclass +class UnconfiguredScenario(exeter.Scenario): + """Tests for a pasta instance without --config-net""" + + host: tunbridge.Site + guest: tunbridge.Site + ifname: str + addr6: tunbridge.ip.AddrMask6 + gw6: tunbridge.ip.Addr6

Until this point, it looks like stuff I can happily copy and paste, and grasp, even. But then:

...

+ @exeter.scenariotest + def test_ifname(self) -> None: + ifs = tunbridge.ip.ifs(self.guest) + exeter.assert_eq(set(ifs), {'lo', self.ifname})

...why does a "Scenario" have a .ifname?

Yeah, the readability of the Scenario mechanism was something I was particularly concerned about. I think the concept is valuable, but I'm very open to different ways of naming or organising it, if we can up with something better. A "Scenario" (specifically a subclass of exeter.Scenario) is a group of tests with a common set of parameters. In this case UnconfiguredScenario is a bunch of tests about the behaviour of pasta without --config-net. Each of those tests has access to the host and guest sites, the expected interface name, address and gateway in the guest - that is, the contents of an UncofiguredScenario instance. That instance describes a real (simulated) environment in which we can run those tests. You use this by supplying a function which sets things up, then yields an UnconfiguredScenario instance describing what it set up. exeter will run all of the UnconfiguredScenario tests on the environment the setup function created, each one as a separate test case. Usually, there are multiple ways to set up a suitable enviroment: running pasta with an existing guest ns vs. pasta creating the ns is a simple example. You can create different setup functions for each of those, and re-use all the tests in the Scenario against each of those setups.

...

+ + @tunbridge.ndp.NdpScenario.subscenario + def test_ndp(self) -> tunbridge.ndp.NdpScenario: + tunbridge.ip.ifup(self.guest, self.ifname)

This raises the question of how much of tunbridge one needs to know to be able to write a basic test. Why is ifup() in 'ip'? I thought it would be more of a "link" thing.

Finding misleading names is a big reason for seeking early feedback. There's kind of a reason for ifup to be in ip: it optionally takes IP addresses to configure on the interface. But... there's no inherent reason it couldn't take other sorts of network address too, so I'll look into moving that into a "link" module or something like it.

I admit I haven't had time to browse tunbridge recently, I'm just looking at this series right now.

That's fine. At some point it would be good to have you look at tunbridge too, but reading this series _without_ reading tunbridge is a very useful perspective at this stage.

...

+ return tunbridge.ndp.NdpScenario(client=self.guest, + ifname=self.ifname, + network=self.addr6.network, + gateway=self.gw6)

This makes sense to me.

Ok, good. The Scenario stuff might not be as impenetrable as I feared.

...

+ + +@UnconfiguredScenario.test +@contextlib.contextmanager +def simh_pasta_setup() -> Iterator[UnconfiguredScenario]: + with (tunbridge.sample.simple_host('host') as simh, + tunbridge.sample.isolated('guest', simh.site) as guest): + assert simh.ip6 is not None + assert simh.gw6_ll is not None + with tasst.pasta.pasta(simh.site, guest): + yield UnconfiguredScenario(host=simh.site, + guest=guest, + ifname=simh.ifname, + addr6=simh.ip6, + gw6=simh.gw6_ll)

...and this too.

But there's one thing I'm missing: if it's a network simulator, why do you need to call a simple_host() method to *describe* the fact that you have a host / site? That looks rather unexpected.

I mean, I would have expected a syntax, in pseudocode, expressing:

1. x := node (properties such as a list of interfaces a, b, c)

2. pasta implements/connects a

...I think this is mostly embedded in the sample.simple_host() thing, but I'm not sure how. Maybe it will become clearer once I actually look into tunbridge, though.

Right. "simple_host" isn't just an arbitrary node, but a (small) predefined network topology: a node configured with a single default gateway (also simulated, albeit minimally) - that is, the "classic" pasta host. The idea is that the tunbridge.sample module will have a bunch of such example networks - so far there's: - isolated() (node with loopback only) - back_to_back() (two nodes connected by a veth) - simple_host() Suggestions for better names welcome, as always.

Of course, I'm trying to push away my bias coming from the fact I was, several years ago, for kselftests, aiming at something like this instead:

A veth B x=$(addr A veth) B ping -c1 $x A $x vxlan B $(addr B veth) ...

(where 'veth', 'vxlan' were both reserved keywords). Maybe once non-trivial links are implemented in tunbridge it will all become more obvious.

I think tunbridge is not dissimilar to this, though with functions rather than reserved words. It's a bit hidden here, because we're using these pre-built chunks - I expect that would be the case for your system as well, once you get to complex enough setups that you want to re-use non-trivial pieces. For example the guts of back_to_back() is: with isolated(f'{name}0', sb) as s0, \ isolated(f'{name}1', sb) as s1: if0, if1 = f'veth{name}0', f'veth{name}1' with veth.veth(s0, if0, s1, if1): ... There's more, but that's mostly about IP allocation (it optionally does that).

...

+ + +if __name__ == '__main__': + exeter.main() diff --git a/test/run b/test/run index 3872a56e..4f09d767 100755 --- a/test/run +++ b/test/run @@ -43,8 +43,10 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"}

COMMIT="$(git log --oneline --no-decorate -1)"

-# Let exeter tests written in Python find their modules +# Let exeter tests written in Python find their modules and binaries to run export PYTHONPATH=${BASEPATH}/exeter/py3:${BASEPATH}/tunbridge:${BASEPATH} +export PASTA=${PASTA:-${BASEPATH}/../pasta} +

. lib/util . lib/context @@ -75,8 +77,8 @@ run() { exeter build/build.py exeter build/static_checkers.sh

+ exeter pasta/ndp.py setup pasta - test pasta/ndp test pasta/dhcp test pasta/tcp test pasta/udp diff --git a/test/tasst/__init__.py b/test/tasst/__init__.py index fd4fe9a8..f5386b3a 100644 --- a/test/tasst/__init__.py +++ b/test/tasst/__init__.py @@ -8,3 +8,7 @@ # # Copyright Red Hat # Author: David Gibson + +from . import pasta + +__all__ = ['pasta'] diff --git a/test/tasst/pasta.py b/test/tasst/pasta.py new file mode 100644 index 00000000..91f59036 --- /dev/null +++ b/test/tasst/pasta.py @@ -0,0 +1,40 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# TASST - Test A Simple Socket Transport +# +# test/tasst/pasta.py - Helpers for seeting up pasta instances +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import os +from typing import Iterator + +import tunbridge + + +@contextlib.contextmanager +def pasta(host: tunbridge.Site, guest: tunbridge.Site, *opts: str) \ + -> Iterator[tunbridge.site.SiteProcess]: + if tunbridge.unshare.parent(guest) is not host: + raise ValueError("pasta guest must be a namespace under host site") + + # This implies guest is a namespace site + assert isinstance(guest, tunbridge.unshare.NsenterSite) + + exe = os.environ['PASTA'] + + with host.tempdir() as piddir: + pidfile = os.path.join(piddir, 'pasta.pid') + cmd = [exe, '-f', '-P', pidfile] + list(opts) + [f'{guest.pid}'] + with host.bg(*cmd, stop=True) as pasta: + # Wait for the PID file to be written + pidstr = None + while not pidstr: + pidstr = host.readfile(pidfile, check=False) + pid = int(pidstr) + print(f'pasta started, host: {host}, guest: {guest}, pid: {pid}') + yield pasta

...perhaps we could also print version and path.

Path I can easily add. Version would require an extra invocation of pasta, which I don't really want to do.

This part also looks quite readable and intuitive to me without having looked into tunbridge recently.

Ok, that's promising. -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote:

On Wed, 8 Oct 2025 13:32:27 +1100 David Gibson wrote:

...

On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote:

...

On Thu, 2 Oct 2025 17:57:08 +1000 David Gibson wrote:

...

Convert the pasta NDP tests from shell and our own DSL to Python using the exeter test protocol and tunbridge network simulation library.

Signed-off-by: David Gibson --- test/Makefile | 2 +- test/pasta/dhcp | 5 ++++ test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++++ test/run | 6 +++-- test/tasst/__init__.py | 4 +++ test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ 6 files changed, 113 insertions(+), 3 deletions(-) create mode 100755 test/pasta/ndp.py create mode 100644 test/tasst/pasta.py

diff --git a/test/Makefile b/test/Makefile index f66c7e7e..95e3d75e 100644 --- a/test/Makefile +++ b/test/Makefile @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)

EXETER_SH = smoke/smoke.sh build/static_checkers.sh EXETER_PYPATH = exeter/py3:tunbridge/:. -EXETER_PYTHON = smoke/smoke.py build/build.py +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats) BATS_FILES = $(EXETER_BATS) \ podman/test/system/505-networking-pasta.bats diff --git a/test/pasta/dhcp b/test/pasta/dhcp index e1c66be6..61279fbf 100644 --- a/test/pasta/dhcp +++ b/test/pasta/dhcp @@ -18,6 +18,11 @@ test Interface name nsout IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname' check [ -n "__IFNAME__" ]

+# Bring up the interface +ns ip link set dev __IFNAME__ up +# Wait for SLAAC & DAD to complete +ns while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done + test DHCP: address ns /sbin/dhclient -4 --no-pid __IFNAME__ nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py new file mode 100755 index 00000000..8c7ce31e --- /dev/null +++ b/test/pasta/ndp.py @@ -0,0 +1,59 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# test/pasta/ndp.py - pasta NDP functionality +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import dataclasses +from typing import Iterator + +import exeter +import tunbridge +import tasst + + +@dataclasses.dataclass +class UnconfiguredScenario(exeter.Scenario): + """Tests for a pasta instance without --config-net""" + + host: tunbridge.Site + guest: tunbridge.Site + ifname: str + addr6: tunbridge.ip.AddrMask6 + gw6: tunbridge.ip.Addr6

Until this point, it looks like stuff I can happily copy and paste, and grasp, even. But then:

...

+ @exeter.scenariotest + def test_ifname(self) -> None: + ifs = tunbridge.ip.ifs(self.guest) + exeter.assert_eq(set(ifs), {'lo', self.ifname})

...why does a "Scenario" have a .ifname?

Yeah, the readability of the Scenario mechanism was something I was particularly concerned about. I think the concept is valuable, but I'm very open to different ways of naming or organising it, if we can up with something better.

From the description you give below, the name seems to fit.

...

A "Scenario" (specifically a subclass of exeter.Scenario) is a group of tests with a common set of parameters. In this case UnconfiguredScenario is a bunch of tests about the behaviour of pasta without --config-net. Each of those tests has access to the host and guest sites, the expected interface name, address and gateway in the guest - that is, the contents of an UncofiguredScenario instance.

I'm not sure if I understand this correctly, but if each guest has a single interface, that sounds a bit limiting.

Sorry, to be clear: a Scenario in the general sense can contain whatever parameters you like. This *particular* Scenario - UnconfiguredScenario - has just those things, because those are all that its tests require.

Actually, I think any abstraction that doesn't offer arbitrary sets of (and relationships between) the objects shown via netlink (or, at least, namespaces, links, routes, addresses, neighbours) might be limiting and not generic enough.

Absolutely, and the abstraction does allow that.

...

That instance describes a real (simulated) environment in which we can run those tests.

You use this by supplying a function which sets things up, then yields an UnconfiguredScenario instance describing what it set up. exeter will run all of the UnconfiguredScenario tests on the environment the setup function created, each one as a separate test case.

This part is now clear to me, and I think it's not complicated to grasp the concept vaguely but enough to copy, paste, and modify code doing this.

Ok.

It would be even better to hide this entirely, because "yielding a scenario" is a Python thing. In general, there's an imperative part in all this (bordering functional programming, but still, not descriptive) which I struggle to see as beneficial.

Here the tasks at hand are, roughly:

1. represent two network namespaces, with two interfaces each (loopback and non-loopback), with pasta connecting one of the interfaces of the inner one

There's a bit more to it than that - we need to specify the host's routing setup, because that will affect what pasta does. That's what simple_host() is about, creating a host with the single gateway routing that's our easiest / most common case.

2. bring up one of the interfaces

3. compare addresses

...and doing 1. like that is simply not... intuitive, I think.

I'm not really clear on what you're getting at here. There is an unavoidable tradeoff here between obviousness for a single case, versus reuseability for multiple related cases. Is it just that some of the relevant setup is hidden inside simple_host() that's the problem? Or is it something else?

...

Usually, there are multiple ways to set up a suitable enviroment: running pasta with an existing guest ns vs. pasta creating the ns is a simple example. You can create different setup functions for each of those, and re-use all the tests in the Scenario against each of those setups.

...

...

+ + @tunbridge.ndp.NdpScenario.subscenario + def test_ndp(self) -> tunbridge.ndp.NdpScenario: + tunbridge.ip.ifup(self.guest, self.ifname)

This raises the question of how much of tunbridge one needs to know to be able to write a basic test. Why is ifup() in 'ip'? I thought it would be more of a "link" thing.

Finding misleading names is a big reason for seeking early feedback. There's kind of a reason for ifup to be in ip: it optionally takes IP addresses to configure on the interface. But... there's no inherent reason it couldn't take other sorts of network address too, so I'll look into moving that into a "link" module or something like it.

I think sticking to netlink objects would make this a bit more familiar, if possible.

Noted.

...

...

I admit I haven't had time to browse tunbridge recently, I'm just looking at this series right now.

That's fine. At some point it would be good to have you look at tunbridge too, but reading this series _without_ reading tunbridge is a very useful perspective at this stage.

...

...

+ return tunbridge.ndp.NdpScenario(client=self.guest, + ifname=self.ifname, + network=self.addr6.network, + gateway=self.gw6)

This makes sense to me.

Ok, good. The Scenario stuff might not be as impenetrable as I feared.

Here I was simply commenting on the fact that I intuitively understand those arguments and how they belong to the scenario, not on the Scenario abstraction itself, but in any case, yes, given a bit of time and sufficient motivation, I don't think it's impenetrable either.

I should clarify - this is not as impenetrable as I feared for a first draft. Therefore, I am encouraged to think I can lift it up to actually nice to use in the relatively near future.

So, while at it, let me share my most substantial worry about all this at the moment. While not impenetrable implies it's usable, I'm not sure how much further that goes.

That's mostly fine if the only goal is to develop and run tests for passt (and I say "mostly" because to run these tests as part of automatic distribution testing you need to package them, and have packages for many distributions, which is a bit difficult to justify if you have a single usage, but let's set this aside for a moment).

Still, that single-goal perspective doesn't look sustainable to me. That's the case for the current test suite, but it was never meant to be a real "framework" or simulator or anything anybody would like to use for anything else.

If I'm looking for a tool that lets me quickly set up a VXLAN tunnel between two nodes and try to flip offloads on and off I think it's unreasonable to expect I'll go for some Scenario abstraction on the basis of being, after all... not impenetrable.

Scenarios aren't about writing *a* test. If you have a one off test with a one-off setup, you can just write that out. Positing the existence of a vxlan() function in tunbridge, this would be something like: with back_to_back(...): with vxlan(...): site.fg(some commands) assert <whatever> Scenarios (which are an exeter thing, not a tunbridge thing) are strictly about reusing tests in multiple related but non-identical situations. Using them in just this initial patch probably looks a bit like overkill. But the point is that we don't have to redefine the same tests when we want to run them for pasta and for passt, and in a bunch of different configurations of each.

And this kind of stuff is a very recurrent need in Linux networking development, in my experience, as well as an unsatisfied need in testing of many related projects.

Agreed.

Of course, one pressing goal right now is to have a more structured way to define tests for passt, and anything that lets us achieve that goal with a reasonable amount of time and effort is welcome.

But not having an interface that lets people build a test tunnel between two nodes in a couple of minutes of reading examples carries a serious risk that this gets stuck "forever" to passt and its tests.

Right. I see the concern. Again it comes back to this tradeoff between immediate readability of a single test, versus reusability of logic across a whole bunch of tests. I _think_ most of your concerns are coming down to the fact that the steps for building the simulated networks aren't obvious to you, because they're hidden within helpers. So... I guess I hope that this will become better with a larger library of example tests?

...

...

...

+ + +@UnconfiguredScenario.test +@contextlib.contextmanager +def simh_pasta_setup() -> Iterator[UnconfiguredScenario]: + with (tunbridge.sample.simple_host('host') as simh, + tunbridge.sample.isolated('guest', simh.site) as guest): + assert simh.ip6 is not None + assert simh.gw6_ll is not None + with tasst.pasta.pasta(simh.site, guest): + yield UnconfiguredScenario(host=simh.site, + guest=guest, + ifname=simh.ifname, + addr6=simh.ip6, + gw6=simh.gw6_ll)

...and this too.

But there's one thing I'm missing: if it's a network simulator, why do you need to call a simple_host() method to *describe* the fact that you have a host / site? That looks rather unexpected.

I mean, I would have expected a syntax, in pseudocode, expressing:

1. x := node (properties such as a list of interfaces a, b, c)

2. pasta implements/connects a

...I think this is mostly embedded in the sample.simple_host() thing, but I'm not sure how. Maybe it will become clearer once I actually look into tunbridge, though.

Right. "simple_host" isn't just an arbitrary node, but a (small) predefined network topology: a node configured with a single default gateway (also simulated, albeit minimally) - that is, the "classic" pasta host. The idea is that the tunbridge.sample module will have a bunch of such example networks - so far there's: - isolated() (node with loopback only) - back_to_back() (two nodes connected by a veth) - simple_host()

Suggestions for better names welcome, as always.

I'm a bit worried by the mere fact that those example networks (and they're all methods instead of some kind of grammar!) are needed.

Depends what you mean by "needed". You could open code the contents of simple_host() in each test - it's not that much - but doing that every time seems tedious. The idea here is you can build complex networks by composing simple components into small chunks, then small chunks into bigger chunks and so forth.

Anyway, I don't find back_to_back() particularly descriptive (what

Understood. To me it suggests two machines directly connected, rather than via a switch or a router... but that might be because I was connecting physical machines like that in the 90s.

makes it not front-to-front?). Perhaps a more mundane "two_nodes()" makes it more obvious (they won't be isolated, of course).

I'll consider that option for the next spin. Another possible option: what about isolated_node() for the lo-only node, and isolated_pair() for the veth pair (the implication being they're connected to each other, but isolated from the rest of the world). Not sure if that's more confusing or less... ...actually, I think I just talked myself out of that idea. On the same grounds isolated() is probably not great - the node *starts* isolated, but it probably won't stay that way (e.g. back_to_back() takes two isolated()s then connects them with a veth()). I'll rethink the names on that basis.

...

...

Of course, I'm trying to push away my bias coming from the fact I was, several years ago, for kselftests, aiming at something like this instead:

A veth B x=$(addr A veth) B ping -c1 $x A $x vxlan B $(addr B veth) ...

(where 'veth', 'vxlan' were both reserved keywords). Maybe once non-trivial links are implemented in tunbridge it will all become more obvious.

I think tunbridge is not dissimilar to this, though with functions rather than reserved words.

That's pretty much the whole difference I was trying to convey, though. Syntax is not entirely irrelevant. Of course, it doesn't need to be reserved words in arbitrary positions, but probably there are other ways to consider.

Syntax certainly isn't irrelevant, but so far I haven't grasped what you dislike about the function syntax versus a specialized grammer. Is it: - The irritating silly parentheses? - Longish (qualified) function names? - The indentation from the with syntax? - Something else?

...

It's a bit hidden here, because we're using these pre-built chunks - I expect that would be the case for your system as well, once you get to complex enough setups that you want to re-use non-trivial pieces.

For example the guts of back_to_back() is:

with isolated(f'{name}0', sb) as s0, \ isolated(f'{name}1', sb) as s1: if0, if1 = f'veth{name}0', f'veth{name}1' with veth.veth(s0, if0, s1, if1): ...

There's more, but that's mostly about IP allocation (it optionally does that).

...

...

+ + +if __name__ == '__main__': + exeter.main() diff --git a/test/run b/test/run index 3872a56e..4f09d767 100755 --- a/test/run +++ b/test/run @@ -43,8 +43,10 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"}

COMMIT="$(git log --oneline --no-decorate -1)"

-# Let exeter tests written in Python find their modules +# Let exeter tests written in Python find their modules and binaries to run export PYTHONPATH=${BASEPATH}/exeter/py3:${BASEPATH}/tunbridge:${BASEPATH} +export PASTA=${PASTA:-${BASEPATH}/../pasta} +

. lib/util . lib/context @@ -75,8 +77,8 @@ run() { exeter build/build.py exeter build/static_checkers.sh

+ exeter pasta/ndp.py setup pasta - test pasta/ndp test pasta/dhcp test pasta/tcp test pasta/udp diff --git a/test/tasst/__init__.py b/test/tasst/__init__.py index fd4fe9a8..f5386b3a 100644 --- a/test/tasst/__init__.py +++ b/test/tasst/__init__.py @@ -8,3 +8,7 @@ # # Copyright Red Hat # Author: David Gibson + +from . import pasta + +__all__ = ['pasta'] diff --git a/test/tasst/pasta.py b/test/tasst/pasta.py new file mode 100644 index 00000000..91f59036 --- /dev/null +++ b/test/tasst/pasta.py @@ -0,0 +1,40 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# TASST - Test A Simple Socket Transport +# +# test/tasst/pasta.py - Helpers for seeting up pasta instances +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import os +from typing import Iterator + +import tunbridge + + +@contextlib.contextmanager +def pasta(host: tunbridge.Site, guest: tunbridge.Site, *opts: str) \ + -> Iterator[tunbridge.site.SiteProcess]: + if tunbridge.unshare.parent(guest) is not host: + raise ValueError("pasta guest must be a namespace under host site") + + # This implies guest is a namespace site + assert isinstance(guest, tunbridge.unshare.NsenterSite) + + exe = os.environ['PASTA'] + + with host.tempdir() as piddir: + pidfile = os.path.join(piddir, 'pasta.pid') + cmd = [exe, '-f', '-P', pidfile] + list(opts) + [f'{guest.pid}'] + with host.bg(*cmd, stop=True) as pasta: + # Wait for the PID file to be written + pidstr = None + while not pidstr: + pidstr = host.readfile(pidfile, check=False) + pid = int(pidstr) + print(f'pasta started, host: {host}, guest: {guest}, pid: {pid}') + yield pasta

...perhaps we could also print version and path.

Path I can easily add. Version would require an extra invocation of pasta, which I don't really want to do.

Ah, right, never mind. The path will be good enough for that.

...

...

This part also looks quite readable and intuitive to me without having looked into tunbridge recently.

Ok, that's promising.

I mean, I think it's all usable for the moment, and perhaps a starting point for some other kind of... front-end? I'm not sure. As I mentioned I'm a bit worried about the potential for universal intuitiveness and usability.

So am I, but I have to weigh it against being able to re-use both tests and setups without having to re-express both in each case. -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

On Fri, Oct 10, 2025 at 01:20:23AM +0200, Stefano Brivio wrote:

On Thu, 9 Oct 2025 15:47:01 +1100 David Gibson wrote:

...

On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote:

...

On Wed, 8 Oct 2025 13:32:27 +1100 David Gibson wrote:

...

On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote:

...

On Thu, 2 Oct 2025 17:57:08 +1000 David Gibson wrote:

...

Convert the pasta NDP tests from shell and our own DSL to Python using the exeter test protocol and tunbridge network simulation library.

Signed-off-by: David Gibson --- test/Makefile | 2 +- test/pasta/dhcp | 5 ++++ test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++++ test/run | 6 +++-- test/tasst/__init__.py | 4 +++ test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ 6 files changed, 113 insertions(+), 3 deletions(-) create mode 100755 test/pasta/ndp.py create mode 100644 test/tasst/pasta.py

diff --git a/test/Makefile b/test/Makefile index f66c7e7e..95e3d75e 100644 --- a/test/Makefile +++ b/test/Makefile @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)

EXETER_SH = smoke/smoke.sh build/static_checkers.sh EXETER_PYPATH = exeter/py3:tunbridge/:. -EXETER_PYTHON = smoke/smoke.py build/build.py +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats) BATS_FILES = $(EXETER_BATS) \ podman/test/system/505-networking-pasta.bats diff --git a/test/pasta/dhcp b/test/pasta/dhcp index e1c66be6..61279fbf 100644 --- a/test/pasta/dhcp +++ b/test/pasta/dhcp @@ -18,6 +18,11 @@ test Interface name nsout IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname' check [ -n "__IFNAME__" ]

+# Bring up the interface +ns ip link set dev __IFNAME__ up +# Wait for SLAAC & DAD to complete +ns while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done + test DHCP: address ns /sbin/dhclient -4 --no-pid __IFNAME__ nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py new file mode 100755 index 00000000..8c7ce31e --- /dev/null +++ b/test/pasta/ndp.py @@ -0,0 +1,59 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# test/pasta/ndp.py - pasta NDP functionality +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import dataclasses +from typing import Iterator + +import exeter +import tunbridge +import tasst + + +@dataclasses.dataclass +class UnconfiguredScenario(exeter.Scenario): + """Tests for a pasta instance without --config-net""" + + host: tunbridge.Site + guest: tunbridge.Site + ifname: str + addr6: tunbridge.ip.AddrMask6 + gw6: tunbridge.ip.Addr6

Until this point, it looks like stuff I can happily copy and paste, and grasp, even. But then:

...

+ @exeter.scenariotest + def test_ifname(self) -> None: + ifs = tunbridge.ip.ifs(self.guest) + exeter.assert_eq(set(ifs), {'lo', self.ifname})

...why does a "Scenario" have a .ifname?

Yeah, the readability of the Scenario mechanism was something I was particularly concerned about. I think the concept is valuable, but I'm very open to different ways of naming or organising it, if we can up with something better.

From the description you give below, the name seems to fit.

...

A "Scenario" (specifically a subclass of exeter.Scenario) is a group of tests with a common set of parameters. In this case UnconfiguredScenario is a bunch of tests about the behaviour of pasta without --config-net. Each of those tests has access to the host and guest sites, the expected interface name, address and gateway in the guest - that is, the contents of an UncofiguredScenario instance.

I'm not sure if I understand this correctly, but if each guest has a single interface, that sounds a bit limiting.

Sorry, to be clear: a Scenario in the general sense can contain whatever parameters you like. This *particular* Scenario - UnconfiguredScenario - has just those things, because those are all that its tests require.

Ah, okay. Still, if I now want to take UnconfiguredScenario and add a couple of dummy interfaces to it for a quick test, I guess I have the choice to either do that with some "external" hack, or... copy and rename it, so that it doesn't affect all the usages?

No. A Scenario instance isn't responsible for managing the simulated environment - that's the setup function - it's just conveying the information about it that the tests need. So, you can make a setup function that adds the dummy interfaces, and still yield an UnconfiguredScenario. It doesn't need to have information about the dummy interfaces because the tests carried by UnconfiguredScenario don't care about them. The scenario mechanism does several things: 1) Groups together some related (parameterized) tests 2) Allows all of those tests to be registered at once 3) Provides a mechanism for providing a bunch of information to those tests (without requiring them each to have a large set of direct parameters) I'm aware that doing those things with the same construct may be confusing - it's just ways of doing them separately also seem confusing and/or awkward in their own ways. Maybe there's a better way, but I haven't spotted it yet.

...

...

Actually, I think any abstraction that doesn't offer arbitrary sets of (and relationships between) the objects shown via netlink (or, at least, namespaces, links, routes, addresses, neighbours) might be limiting and not generic enough.

Absolutely, and the abstraction does allow that.

...

...

That instance describes a real (simulated) environment in which we can run those tests.

You use this by supplying a function which sets things up, then yields an UnconfiguredScenario instance describing what it set up. exeter will run all of the UnconfiguredScenario tests on the environment the setup function created, each one as a separate test case.

This part is now clear to me, and I think it's not complicated to grasp the concept vaguely but enough to copy, paste, and modify code doing this.

Ok.

...

It would be even better to hide this entirely, because "yielding a scenario" is a Python thing. In general, there's an imperative part in all this (bordering functional programming, but still, not descriptive) which I struggle to see as beneficial.

Here the tasks at hand are, roughly:

1. represent two network namespaces, with two interfaces each (loopback and non-loopback), with pasta connecting one of the interfaces of the inner one

There's a bit more to it than that - we need to specify the host's routing setup, because that will affect what pasta does. That's what simple_host() is about, creating a host with the single gateway routing that's our easiest / most common case.

Okay, sure, by "interfaces" I meant configured interfaces with addresses and a default route, too. But that doesn't really modify my point, that is:

...

...

2. bring up one of the interfaces

3. compare addresses

...and doing 1. like that is simply not... intuitive, I think.

I'm not really clear on what you're getting at here. There is an unavoidable tradeoff here between obviousness for a single case, versus reuseability for multiple related cases. Is it just that some of the relevant setup is hidden inside simple_host() that's the problem? Or is it something else?

...yes, one part is that it's hidden. Another part are, specifically, these lines:

host: tunbridge.Site guest: tunbridge.Site ifname: str

[...]

@exeter.scenariotest def test_ifname(self) -> None:

[...]

None of these clearly links to "two network namespaces: A, with interface a1 and address x1, ...".

Fair. This needs a docstring explaining the parameters / fields.

I understand this is probably very close to the bare minimum you can get by modelling this all with actual code, and that's why I think actual (imperative/functional) code is usually not used to model/describe things.

Imperative/functional code as opposed to..?

...

...

...

Usually, there are multiple ways to set up a suitable enviroment: running pasta with an existing guest ns vs. pasta creating the ns is a simple example. You can create different setup functions for each of those, and re-use all the tests in the Scenario against each of those setups.

...

...

+ + @tunbridge.ndp.NdpScenario.subscenario + def test_ndp(self) -> tunbridge.ndp.NdpScenario: + tunbridge.ip.ifup(self.guest, self.ifname)

This raises the question of how much of tunbridge one needs to know to be able to write a basic test. Why is ifup() in 'ip'? I thought it would be more of a "link" thing.

Finding misleading names is a big reason for seeking early feedback. There's kind of a reason for ifup to be in ip: it optionally takes IP addresses to configure on the interface. But... there's no inherent reason it couldn't take other sorts of network address too, so I'll look into moving that into a "link" module or something like it.

I think sticking to netlink objects would make this a bit more familiar, if possible.

Noted.

...

...

...

I admit I haven't had time to browse tunbridge recently, I'm just looking at this series right now.

That's fine. At some point it would be good to have you look at tunbridge too, but reading this series _without_ reading tunbridge is a very useful perspective at this stage.

...

...

+ return tunbridge.ndp.NdpScenario(client=self.guest, + ifname=self.ifname, + network=self.addr6.network, + gateway=self.gw6)

This makes sense to me.

Ok, good. The Scenario stuff might not be as impenetrable as I feared.

Here I was simply commenting on the fact that I intuitively understand those arguments and how they belong to the scenario, not on the Scenario abstraction itself, but in any case, yes, given a bit of time and sufficient motivation, I don't think it's impenetrable either.

I should clarify - this is not as impenetrable as I feared for a first draft. Therefore, I am encouraged to think I can lift it up to actually nice to use in the relatively near future.

...

So, while at it, let me share my most substantial worry about all this at the moment. While not impenetrable implies it's usable, I'm not sure how much further that goes.

That's mostly fine if the only goal is to develop and run tests for passt (and I say "mostly" because to run these tests as part of automatic distribution testing you need to package them, and have packages for many distributions, which is a bit difficult to justify if you have a single usage, but let's set this aside for a moment).

Still, that single-goal perspective doesn't look sustainable to me. That's the case for the current test suite, but it was never meant to be a real "framework" or simulator or anything anybody would like to use for anything else.

If I'm looking for a tool that lets me quickly set up a VXLAN tunnel between two nodes and try to flip offloads on and off I think it's unreasonable to expect I'll go for some Scenario abstraction on the basis of being, after all... not impenetrable.

Scenarios aren't about writing *a* test. If you have a one off test with a one-off setup, you can just write that out. Positing the existence of a vxlan() function in tunbridge, this would be something like:

with back_to_back(...): with vxlan(...): site.fg(some commands) assert <whatever>

Scenarios (which are an exeter thing, not a tunbridge thing) are strictly about reusing tests in multiple related but non-identical situations.

Using them in just this initial patch probably looks a bit like overkill. But the point is that we don't have to redefine the same tests when we want to run them for pasta and for passt, and in a bunch of different configurations of each.

I see, I'm quite convinced by the concept itself, actually. What I really can't wrap my head around is that particular syntax and imperative code to describe a topology with a VXLAN tunnel.

...

...

And this kind of stuff is a very recurrent need in Linux networking development, in my experience, as well as an unsatisfied need in testing of many related projects.

Agreed.

...

Of course, one pressing goal right now is to have a more structured way to define tests for passt, and anything that lets us achieve that goal with a reasonable amount of time and effort is welcome.

But not having an interface that lets people build a test tunnel between two nodes in a couple of minutes of reading examples carries a serious risk that this gets stuck "forever" to passt and its tests.

Right. I see the concern. Again it comes back to this tradeoff between immediate readability of a single test, versus reusability of logic across a whole bunch of tests. I _think_ most of your concerns are coming down to the fact that the steps for building the simulated networks aren't obvious to you, because they're hidden within helpers.

Hmm, no, not so much, that part is clear and I'm convinced we need something like that.

...

So... I guess I hope that this will become better with a larger library of example tests?

It should make things easier to grasp, but not really address my main concern, see below.

...

...

...

...

...

+ + +@UnconfiguredScenario.test +@contextlib.contextmanager +def simh_pasta_setup() -> Iterator[UnconfiguredScenario]: + with (tunbridge.sample.simple_host('host') as simh, + tunbridge.sample.isolated('guest', simh.site) as guest): + assert simh.ip6 is not None + assert simh.gw6_ll is not None + with tasst.pasta.pasta(simh.site, guest): + yield UnconfiguredScenario(host=simh.site, + guest=guest, + ifname=simh.ifname, + addr6=simh.ip6, + gw6=simh.gw6_ll)

...and this too.

But there's one thing I'm missing: if it's a network simulator, why do you need to call a simple_host() method to *describe* the fact that you have a host / site? That looks rather unexpected.

I mean, I would have expected a syntax, in pseudocode, expressing:

1. x := node (properties such as a list of interfaces a, b, c)

2. pasta implements/connects a

...I think this is mostly embedded in the sample.simple_host() thing, but I'm not sure how. Maybe it will become clearer once I actually look into tunbridge, though.

Right. "simple_host" isn't just an arbitrary node, but a (small) predefined network topology: a node configured with a single default gateway (also simulated, albeit minimally) - that is, the "classic" pasta host. The idea is that the tunbridge.sample module will have a bunch of such example networks - so far there's: - isolated() (node with loopback only) - back_to_back() (two nodes connected by a veth) - simple_host()

Suggestions for better names welcome, as always.

I'm a bit worried by the mere fact that those example networks (and they're all methods instead of some kind of grammar!) are needed.

Depends what you mean by "needed". You could open code the contents of simple_host() in each test - it's not that much - but doing that every time seems tedious. The idea here is you can build complex networks by composing simple components into small chunks, then small chunks into bigger chunks and so forth.

...

Anyway, I don't find back_to_back() particularly descriptive (what

Understood. To me it suggests two machines directly connected, rather than via a switch or a router... but that might be because I was connecting physical machines like that in the 90s.

Oh, because you'd turn their back to each other... I see now. I happened to do that with Ethernet but usually as a "first install" or whatever emergency hack, so I would just grab/make a long cable.

...

...

makes it not front-to-front?). Perhaps a more mundane "two_nodes()" makes it more obvious (they won't be isolated, of course).

I'll consider that option for the next spin.

Another possible option: what about isolated_node() for the lo-only node, and isolated_pair() for the veth pair (the implication being they're connected to each other, but isolated from the rest of the world). Not sure if that's more confusing or less...

...actually, I think I just talked myself out of that idea. On the same grounds isolated() is probably not great - the node *starts* isolated, but it probably won't stay that way (e.g. back_to_back() takes two isolated()s then connects them with a veth()). I'll rethink the names on that basis.

Right, two isolated nodes are not really supposed to talk to each other.

...

...

...

...

Of course, I'm trying to push away my bias coming from the fact I was, several years ago, for kselftests, aiming at something like this instead:

A veth B x=$(addr A veth) B ping -c1 $x A $x vxlan B $(addr B veth) ...

(where 'veth', 'vxlan' were both reserved keywords). Maybe once non-trivial links are implemented in tunbridge it will all become more obvious.

I think tunbridge is not dissimilar to this, though with functions rather than reserved words.

That's pretty much the whole difference I was trying to convey, though. Syntax is not entirely irrelevant. Of course, it doesn't need to be reserved words in arbitrary positions, but probably there are other ways to consider.

Syntax certainly isn't irrelevant, but so far I haven't grasped what you dislike about the function syntax versus a specialized grammer. Is it: - The irritating silly parentheses? - Longish (qualified) function names? - The indentation from the with syntax? - Something else?

It's *also* the first two

Ok. First I can't easily change. Second can be mitigated by handling the imports differently.

(the indentation looks actually convenient),

Ok, good. I also think this is useful because it conveys the lifetime of each object, which will be important once we get to tests where you need to change things part way through.

but that's not my main point. My main point is that this isn't fundamentally declarative. You're turning it into something that resembles that, but the syntax is still from an imperative programming language.

And in my mind the main feature of a network (topology) simulator is that you describe the topology (and it will build it for you), not that you... have to build a description?

Using an example that's obviously familiar to you: think of taking a device tree for some system with a couple of USB and I²C busses and a flash controller, and writing all that in Python based on some form of "bus" module/component.

I mean... old school Open Firmware kind of is this, but with Forth instead of Python.

Once one sees how practical device trees are for that, the Python version would look wrong, wouldn't it?

That really depends on the context. If I was making an API for building a device tree, I'd probably come up with something pretty like this.

Now, while I think that some single bits of DTS syntax are unnecessarily complicated, conceptually, a "networking" device tree would look more usable to me than the approach you're taking.

Of course, we need the whole "testing" / exeter part as well, and test cases are fundamentally sequential/imperative.

But (sorry, it's been a few years I don't touch these):

namespace@1 { interfaces { lo { address = 127.0.0.1; }; eth0 { address = ...; }; }; routes { /* something simpler than ip -j ro sh ? */ }; }

...

link@... { vxlan { endpoints { a { ns = <&namespace@1>; }; b ...

...

this looks much more natural to me, as an input for a simulator (I would personally make the syntax much more "elastic" by just throwing a link into a namespace but I'm trying to keep it clean just for this example).

Aha, I think I finally get what you're saying. More below.

Maybe tunbridge implements this somewhere and I missed it? Or would this be part of a "Scenario" description eventually?

This is entirely unrelated to what Scenario is trying to accomplish. That may cause you to reconsider whether "Scenario" is a good name, which is ok. So. A declarative way of defining networks would be nice to have. I think doing it with the flexibility we want is much harder than you estimate. It looks easy for simple static situations like the examples above, but: * If you want to describe a topology that changes partway through, that's a huge step up in complexity, and kind of necessarily reintroduces imperative elements. Device tree absolutely suffers from this - that's what motivated the godawful runtime overlay mechanism, and right now, I'm struggling to find time to participate in the latest of many discussions about how to better handle devices which can be runtime added and removed. * If you want to build complex scenarios out of simpler ones, you need what amounts to a macro system. Again, a big leap up in complexity. Device tree struggles with this too - it originated primarily as a machine->machine format, where having heaps of repeated information is fine. As it transitioned to being a human->machine format, not so much. Hence /include/, expression support and semi-standard running dts files through cpp before compilation. It's still pretty clunky in this regard. Plus.. I think the interpreter for this hypothetical declarative language would need an internal structure pretty similar to what tunbridge, so this is kind of already a first step towards it.

...

...

...

It's a bit hidden here, because we're using these pre-built chunks - I expect that would be the case for your system as well, once you get to complex enough setups that you want to re-use non-trivial pieces.

For example the guts of back_to_back() is:

with isolated(f'{name}0', sb) as s0, \ isolated(f'{name}1', sb) as s1: if0, if1 = f'veth{name}0', f'veth{name}1' with veth.veth(s0, if0, s1, if1): ...

There's more, but that's mostly about IP allocation (it optionally does that).

...

...

+ + +if __name__ == '__main__': + exeter.main() diff --git a/test/run b/test/run index 3872a56e..4f09d767 100755 --- a/test/run +++ b/test/run @@ -43,8 +43,10 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"}

COMMIT="$(git log --oneline --no-decorate -1)"

-# Let exeter tests written in Python find their modules +# Let exeter tests written in Python find their modules and binaries to run export PYTHONPATH=${BASEPATH}/exeter/py3:${BASEPATH}/tunbridge:${BASEPATH} +export PASTA=${PASTA:-${BASEPATH}/../pasta} +

. lib/util . lib/context @@ -75,8 +77,8 @@ run() { exeter build/build.py exeter build/static_checkers.sh

+ exeter pasta/ndp.py setup pasta - test pasta/ndp test pasta/dhcp test pasta/tcp test pasta/udp diff --git a/test/tasst/__init__.py b/test/tasst/__init__.py index fd4fe9a8..f5386b3a 100644 --- a/test/tasst/__init__.py +++ b/test/tasst/__init__.py @@ -8,3 +8,7 @@ # # Copyright Red Hat # Author: David Gibson + +from . import pasta + +__all__ = ['pasta'] diff --git a/test/tasst/pasta.py b/test/tasst/pasta.py new file mode 100644 index 00000000..91f59036 --- /dev/null +++ b/test/tasst/pasta.py @@ -0,0 +1,40 @@ +#! /usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# TASST - Test A Simple Socket Transport +# +# test/tasst/pasta.py - Helpers for seeting up pasta instances +# +# Copyright Red Hat +# Author: David Gibson + +import contextlib +import os +from typing import Iterator + +import tunbridge + + +@contextlib.contextmanager +def pasta(host: tunbridge.Site, guest: tunbridge.Site, *opts: str) \ + -> Iterator[tunbridge.site.SiteProcess]: + if tunbridge.unshare.parent(guest) is not host: + raise ValueError("pasta guest must be a namespace under host site") + + # This implies guest is a namespace site + assert isinstance(guest, tunbridge.unshare.NsenterSite) + + exe = os.environ['PASTA'] + + with host.tempdir() as piddir: + pidfile = os.path.join(piddir, 'pasta.pid') + cmd = [exe, '-f', '-P', pidfile] + list(opts) + [f'{guest.pid}'] + with host.bg(*cmd, stop=True) as pasta: + # Wait for the PID file to be written + pidstr = None + while not pidstr: + pidstr = host.readfile(pidfile, check=False) + pid = int(pidstr) + print(f'pasta started, host: {host}, guest: {guest}, pid: {pid}') + yield pasta

...perhaps we could also print version and path.

Path I can easily add. Version would require an extra invocation of pasta, which I don't really want to do.

Ah, right, never mind. The path will be good enough for that.

...

...

This part also looks quite readable and intuitive to me without having looked into tunbridge recently.

Ok, that's promising.

I mean, I think it's all usable for the moment, and perhaps a starting point for some other kind of... front-end? I'm not sure. As I mentioned I'm a bit worried about the potential for universal intuitiveness and usability.

So am I, but I have to weigh it against being able to re-use both tests and setups without having to re-express both in each case.

I think setups written like that are reusable (or can be made reusable). My usability point is about other project/usages. For passt and pasta themselves, this level or reusability looks enough to me for the foreseeable future.

Even though, one day, I guess we might want to generate pseudo-random (fractal-tree-like?) topologies (and I was recently trying out a pasta-in-pasta-in-pasta-in-pasta-in-pasta setup to reproduce that HTTP/FIN issue). For that, a declarative approach would make things easier, I suppose.

Declarative, or imperative? I actually have something like that in tunbridge's selftests: a function that builds a stack of N nested namespaces. https://gitlab.com/dgibson/tunbridge/-/blob/main/tunbridge/unshare.py#L302 -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

On Thu, Oct 16, 2025 at 11:31:19PM +0200, Stefano Brivio wrote:

On Fri, 10 Oct 2025 13:17:13 +1100 David Gibson wrote:

...

On Fri, Oct 10, 2025 at 01:20:23AM +0200, Stefano Brivio wrote:

...

On Thu, 9 Oct 2025 15:47:01 +1100 David Gibson wrote:

...

On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote:

...

On Wed, 8 Oct 2025 13:32:27 +1100 David Gibson wrote:

...

On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote: > On Thu, 2 Oct 2025 17:57:08 +1000 > David Gibson wrote: > > > Convert the pasta NDP tests from shell and our own DSL to Python using > > the exeter test protocol and tunbridge network simulation library. > > > > Signed-off-by: David Gibson > > --- > > test/Makefile | 2 +- > > test/pasta/dhcp | 5 ++++ > > test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++++ > > test/run | 6 +++-- > > test/tasst/__init__.py | 4 +++ > > test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ > > 6 files changed, 113 insertions(+), 3 deletions(-) > > create mode 100755 test/pasta/ndp.py > > create mode 100644 test/tasst/pasta.py > > > > diff --git a/test/Makefile b/test/Makefile > > index f66c7e7e..95e3d75e 100644 > > --- a/test/Makefile > > +++ b/test/Makefile > > @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS) > > > > EXETER_SH = smoke/smoke.sh build/static_checkers.sh > > EXETER_PYPATH = exeter/py3:tunbridge/:. > > -EXETER_PYTHON = smoke/smoke.py build/build.py > > +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py > > EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats) > > BATS_FILES = $(EXETER_BATS) \ > > podman/test/system/505-networking-pasta.bats > > diff --git a/test/pasta/dhcp b/test/pasta/dhcp > > index e1c66be6..61279fbf 100644 > > --- a/test/pasta/dhcp > > +++ b/test/pasta/dhcp > > @@ -18,6 +18,11 @@ test Interface name > > nsout IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname' > > check [ -n "__IFNAME__" ] > > > > +# Bring up the interface > > +ns ip link set dev __IFNAME__ up > > +# Wait for SLAAC & DAD to complete > > +ns while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done > > + > > test DHCP: address > > ns /sbin/dhclient -4 --no-pid __IFNAME__ > > nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' > > diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py > > new file mode 100755 > > index 00000000..8c7ce31e > > --- /dev/null > > +++ b/test/pasta/ndp.py > > @@ -0,0 +1,59 @@ > > +#! /usr/bin/env python3 > > +# > > +# SPDX-License-Identifier: GPL-2.0-or-later > > +# > > +# test/pasta/ndp.py - pasta NDP functionality > > +# > > +# Copyright Red Hat > > +# Author: David Gibson > > + > > +import contextlib > > +import dataclasses > > +from typing import Iterator > > + > > +import exeter > > +import tunbridge > > +import tasst > > + > > + > > +@dataclasses.dataclass > > +class UnconfiguredScenario(exeter.Scenario): > > + """Tests for a pasta instance without --config-net""" > > + > > + host: tunbridge.Site > > + guest: tunbridge.Site > > + ifname: str > > + addr6: tunbridge.ip.AddrMask6 > > + gw6: tunbridge.ip.Addr6 > > Until this point, it looks like stuff I can happily copy and paste, > and grasp, even. But then: > > > + @exeter.scenariotest > > + def test_ifname(self) -> None: > > + ifs = tunbridge.ip.ifs(self.guest) > > + exeter.assert_eq(set(ifs), {'lo', self.ifname}) > > ...why does a "Scenario" have a .ifname?

Yeah, the readability of the Scenario mechanism was something I was particularly concerned about. I think the concept is valuable, but I'm very open to different ways of naming or organising it, if we can up with something better.

From the description you give below, the name seems to fit.

...

A "Scenario" (specifically a subclass of exeter.Scenario) is a group of tests with a common set of parameters. In this case UnconfiguredScenario is a bunch of tests about the behaviour of pasta without --config-net. Each of those tests has access to the host and guest sites, the expected interface name, address and gateway in the guest - that is, the contents of an UncofiguredScenario instance.

I'm not sure if I understand this correctly, but if each guest has a single interface, that sounds a bit limiting.

Sorry, to be clear: a Scenario in the general sense can contain whatever parameters you like. This *particular* Scenario - UnconfiguredScenario - has just those things, because those are all that its tests require.

Ah, okay. Still, if I now want to take UnconfiguredScenario and add a couple of dummy interfaces to it for a quick test, I guess I have the choice to either do that with some "external" hack, or... copy and rename it, so that it doesn't affect all the usages?

No. A Scenario instance isn't responsible for managing the simulated environment - that's the setup function - it's just conveying the information about it that the tests need. So, you can make a setup function that adds the dummy interfaces, and still yield an UnconfiguredScenario. It doesn't need to have information about the dummy interfaces because the tests carried by UnconfiguredScenario don't care about them.

Oh, sorry, it's a class, of course, I see now.

...

The scenario mechanism does several things: 1) Groups together some related (parameterized) tests 2) Allows all of those tests to be registered at once 3) Provides a mechanism for providing a bunch of information to those tests (without requiring them each to have a large set of direct parameters)

I'm aware that doing those things with the same construct may be confusing - it's just ways of doing them separately also seem confusing and/or awkward in their own ways. Maybe there's a better way, but I haven't spotted it yet.

It really is confusing to me, but the description above is rather clear so I'll try to propose something once I get to write some kind of setup function and test cases myself.

Makes sense.

...

...

...

...

Actually, I think any abstraction that doesn't offer arbitrary sets of (and relationships between) the objects shown via netlink (or, at least, namespaces, links, routes, addresses, neighbours) might be limiting and not generic enough.

Absolutely, and the abstraction does allow that.

...

...

That instance describes a real (simulated) environment in which we can run those tests.

You use this by supplying a function which sets things up, then yields an UnconfiguredScenario instance describing what it set up. exeter will run all of the UnconfiguredScenario tests on the environment the setup function created, each one as a separate test case.

This part is now clear to me, and I think it's not complicated to grasp the concept vaguely but enough to copy, paste, and modify code doing this.

Ok.

...

It would be even better to hide this entirely, because "yielding a scenario" is a Python thing. In general, there's an imperative part in all this (bordering functional programming, but still, not descriptive) which I struggle to see as beneficial.

Here the tasks at hand are, roughly:

1. represent two network namespaces, with two interfaces each (loopback and non-loopback), with pasta connecting one of the interfaces of the inner one

There's a bit more to it than that - we need to specify the host's routing setup, because that will affect what pasta does. That's what simple_host() is about, creating a host with the single gateway routing that's our easiest / most common case.

Okay, sure, by "interfaces" I meant configured interfaces with addresses and a default route, too. But that doesn't really modify my point, that is:

...

...

2. bring up one of the interfaces

3. compare addresses

...and doing 1. like that is simply not... intuitive, I think.

I'm not really clear on what you're getting at here. There is an unavoidable tradeoff here between obviousness for a single case, versus reuseability for multiple related cases. Is it just that some of the relevant setup is hidden inside simple_host() that's the problem? Or is it something else?

...yes, one part is that it's hidden. Another part are, specifically, these lines:

host: tunbridge.Site guest: tunbridge.Site ifname: str

[...]

@exeter.scenariotest def test_ifname(self) -> None:

[...]

None of these clearly links to "two network namespaces: A, with interface a1 and address x1, ...".

Fair. This needs a docstring explaining the parameters / fields.

That might help a tiny bit but I think the syntax and notations are kind of self-explanatory.

My concern is at a more conceptual level, and it's better summarised below, but here, specifically, we're writing:

host: tunbridge.Site

to say:

give me the "host" network namespace

What's "me" in this sentence?

and to say that, in my ideal world, I would probably go for something on the line(s) of:

A

Not really following, I hope it will make more sense to me below.

...

...

I understand this is probably very close to the bare minimum you can get by modelling this all with actual code, and that's why I think actual (imperative/functional) code is usually not used to model/describe things.

Imperative/functional code as opposed to..?

...declarative.

Ok, I've generally seen functional programming put under the declarative heading. Or as my exam invigilator famously said once "Declaraytive Programming Paradijjums". So, I guess that makes me unclear what does and doesn't count as declarative for you. [snip]

...

...

...

Syntax certainly isn't irrelevant, but so far I haven't grasped what you dislike about the function syntax versus a specialized grammer. Is it: - The irritating silly parentheses? - Longish (qualified) function names? - The indentation from the with syntax? - Something else?

It's *also* the first two

Ok. First I can't easily change. Second can be mitigated by handling the imports differently.

...

(the indentation looks actually convenient),

Ok, good. I also think this is useful because it conveys the lifetime of each object, which will be important once we get to tests where you need to change things part way through.

...

but that's not my main point. My main point is that this isn't fundamentally declarative. You're turning it into something that resembles that, but the syntax is still from an imperative programming language.

And in my mind the main feature of a network (topology) simulator is that you describe the topology (and it will build it for you), not that you... have to build a description?

Using an example that's obviously familiar to you: think of taking a device tree for some system with a couple of USB and I²C busses and a flash controller, and writing all that in Python based on some form of "bus" module/component.

I mean... old school Open Firmware kind of is this, but with Forth instead of Python.

Okay, you can model data structures in Python, obviously, but that wasn't my point. Anyway, it's all in the example below.

...

...

Once one sees how practical device trees are for that, the Python version would look wrong, wouldn't it?

That really depends on the context. If I was making an API for building a device tree, I'd probably come up with something pretty like this.

...an API for building one, yes. But not if you were writing a device tree itself.

...

...

Now, while I think that some single bits of DTS syntax are unnecessarily complicated, conceptually, a "networking" device tree would look more usable to me than the approach you're taking.

Of course, we need the whole "testing" / exeter part as well, and test cases are fundamentally sequential/imperative.

But (sorry, it's been a few years I don't touch these):

namespace@1 { interfaces { lo { address = 127.0.0.1; }; eth0 { address = ...; }; }; routes { /* something simpler than ip -j ro sh ? */ }; }

...

link@... { vxlan { endpoints { a { ns = <&namespace@1>; }; b ...

...

this looks much more natural to me, as an input for a simulator (I would personally make the syntax much more "elastic" by just throwing a link into a namespace but I'm trying to keep it clean just for this example).

Aha, I think I finally get what you're saying. More below.

...

Maybe tunbridge implements this somewhere and I missed it? Or would this be part of a "Scenario" description eventually?

This is entirely unrelated to what Scenario is trying to accomplish. That may cause you to reconsider whether "Scenario" is a good name, which is ok.

So. A declarative way of defining networks would be nice to have.

From my perspective that's fundamental, rather. I gave it for granted.

...

I think doing it with the flexibility we want is much harder than you estimate.

I'll pretend I'm not commenting on this line by... oops. :)

Heh. Well, I'd love to be proved wrong on this one.

...

It looks easy for simple static situations like the examples above, but:

* If you want to describe a topology that changes partway through, that's a huge step up in complexity, and kind of necessarily reintroduces imperative elements.

But you can use JSON or a ton of other formats that allow for ordering of elements.

Just ordering network elements w.r.t. each other isn't enough. If you have a test where you want /this/ network topology - do some stuff - then change the toplogy to /that/ - do some more stuff. I'm not sure how you encode that in a way that isn't imperative, or at least pseudo-imperative.

Alternatively, one could add attributes for lifecycle and timing (think of nftables sets) but it looks much less intuitive than the alternative.

I wasn't previously familiar with nftables sets. I had a quick look at the docs, but I don't see how it relates to the current proposal.

...

Device tree absolutely suffers from this - that's what motivated the godawful runtime overlay mechanism, and right now, I'm struggling to find time to participate in the latest of many discussions about how to better handle devices which can be runtime added and removed.

I'm not suggesting that we use ANS Forth by the way.

Well, we agree on that at least :).

...

* If you want to build complex scenarios out of simpler ones, you need what amounts to a macro system.

There are a ton of ways. You can also use a filesystem and includes. Or simply definitions of blocks, not necessarily macros,

You want to parameterise the blocks - at which point it's basically macros.

and JSON implicitly gives you all that.

No, it doesn't. If you have a JSON sub-object that's repeated several times through a tree, you have to write it out, in full, each time. YAML does allow repeating, as do some other JSON extensions, but bare JSON does not. It still doesn't allow parameterisation of those blocks.

As it's nothing security relevant, I would actually go with something that's in theory more complicated but in practice more digestible such as YAML.

I don't terribly like YAML, because I think there are a bunch of edge cases where it's not obvious reading it whether something is a list or object or somethine else. The typing and back-referencing would be useful for this problem, I'll grant you.

...

Again, a big leap up in complexity. Device tree struggles with this too - it originated primarily as a machine->machine format, where having heaps of repeated information is fine. As it transitioned to being a human->machine format, not so much. Hence /include/, expression support and semi-standard running dts files through cpp before compilation. It's still pretty clunky in this regard.

It absolutely is, but that's because it was designed for a different purpose.

...

Plus.. I think the interpreter for this hypothetical declarative language would need an internal structure pretty similar to what tunbridge, so this is kind of already a first step towards it.

Okay, that's good to know.

I'm estimating I'm currently writing about 5-10 scripts per month, including pasta/iproute2 one-liners, setting up strange stuff, to reproduce / debug issues.

Given that this looks so fundamental for my usage I'm thinking that I could make at least part of this a priority of mine.

I realised I can implement netlink stuff and handling of networking configuration concepts quite fast with Rust and neli, so I'm pondering to write a proof of concept that can parse the example above (minus Forth notations, but with some kind of pointer) and make it create at least namespaces, links, addresses, and routes.

If it helps visualising how that could possibly look like with / in tunbridge itself, I'll take care of it soon rather than later.

The only little problem is that I'm much faster with Rust (because of neli) than I can possibly picture myself with Python, and that doesn't play along with tunbridge. But perhaps as a proof of concept it helps anyway?

It would. At earlier points I did consider writing tunbridge (or whatever I was calling the idea at the time) in Rust. In principle at least, the lifetime tracking would make a very natural way for ensuring you bring up / pull down things in a sensible order. In practice, however, I almost instantly ran into intractable borrow checker problems. Specifically, Rust's notorious difficulty with references between parts of the same structure. That arises almost immediately once you start building composite objects out of smaller components: struct Node { ... } struct Veth<'a> { node0: &'a Node, node1: &'a Node, ... } Then you want something that represents two nodes with a veth between them and you get: struct TwoNodes { node0: Node, node1: Node, veth: Veth<'??> } There are ways around it, of course, but everything I found so far was either hideously unergonomic, depended on obscure crates or both. There are a bunch of language extensions that would help, but while much discussed, none have landed yet. If a way to solve this nicely appears, I'm not against moving tunbridge to Rust. I mean, slightly more against it than I was months ago before I made a start in Python, but still.

In general, do you think there's something in particular I could contribute at this stage, if I want to see my declarative dream come true?

Yes. A bunch of pseudo-code examples - both the network declarations and example tests that might go with them. [snip]

...

...

...

...

...

Path I can easily add. Version would require an extra invocation of pasta, which I don't really want to do.

Ah, right, never mind. The path will be good enough for that.

...

> This part also looks > quite readable and intuitive to me without having looked into tunbridge > recently.

Ok, that's promising.

I mean, I think it's all usable for the moment, and perhaps a starting point for some other kind of... front-end? I'm not sure. As I mentioned I'm a bit worried about the potential for universal intuitiveness and usability.

So am I, but I have to weigh it against being able to re-use both tests and setups without having to re-express both in each case.

I think setups written like that are reusable (or can be made reusable). My usability point is about other project/usages. For passt and pasta themselves, this level or reusability looks enough to me for the foreseeable future.

Even though, one day, I guess we might want to generate pseudo-random (fractal-tree-like?) topologies (and I was recently trying out a pasta-in-pasta-in-pasta-in-pasta-in-pasta setup to reproduce that HTTP/FIN issue). For that, a declarative approach would make things easier, I suppose.

Declarative, or imperative? I actually have something like that in tunbridge's selftests: a function that builds a stack of N nested namespaces. https://gitlab.com/dgibson/tunbridge/-/blob/main/tunbridge/unshare.py#L302

...but they are all the same. Think, for example, of connecting every odd-numbered pair with veth tunnels, and every even-numbered pair with pasta. Say: n1 <-- veth --> n2 <-- pasta --> n3 <-- veth --> n4.

That would certainly be possible. More complex, of course, but not dramtically so.

What's really well suited for this situation, in my experience, is a declarative description format that can be easily generated and manipulated by imperative code.

Ah, so there's both an imperative and declarative component. The idea in tunbridge is that you can do this, but rather than emit the "declarative part" as concrete text for another parser, it's emitted as a data structure (generally a bunch of wrappers around context managers). It is true that as currently designed, tunbridge builds the data structure representation at the same time as building the actual simulated network. With a declarative language approach, building the description (language fragment) is separate from instantiating the simulation. Is that something you see as valuable? Or only a side effect of the other things about the declarative approach you like?

The name of this kind of "indirection" in computer science research currently escapes me, but I'm fairly sure there must be some theory about it. In any case, I can include something like this in my (now planned) proof of concept.

-- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

On Wed, Oct 29, 2025 at 12:17:22AM +0100, Stefano Brivio wrote:

On Fri, 17 Oct 2025 15:30:33 +1100 David Gibson wrote:

...

On Thu, Oct 16, 2025 at 11:31:19PM +0200, Stefano Brivio wrote:

...

On Fri, 10 Oct 2025 13:17:13 +1100 David Gibson wrote:

...

On Fri, Oct 10, 2025 at 01:20:23AM +0200, Stefano Brivio wrote:

...

On Thu, 9 Oct 2025 15:47:01 +1100 David Gibson wrote:

...

On Thu, Oct 09, 2025 at 01:02:48AM +0200, Stefano Brivio wrote: > On Wed, 8 Oct 2025 13:32:27 +1100 > David Gibson wrote: > > > On Tue, Oct 07, 2025 at 10:01:10PM +0200, Stefano Brivio wrote: > > > On Thu, 2 Oct 2025 17:57:08 +1000 > > > David Gibson wrote: > > > > > > > Convert the pasta NDP tests from shell and our own DSL to Python using > > > > the exeter test protocol and tunbridge network simulation library. > > > > > > > > Signed-off-by: David Gibson > > > > --- > > > > test/Makefile | 2 +- > > > > test/pasta/dhcp | 5 ++++ > > > > test/pasta/ndp.py | 59 ++++++++++++++++++++++++++++++++++++++++++ > > > > test/run | 6 +++-- > > > > test/tasst/__init__.py | 4 +++ > > > > test/tasst/pasta.py | 40 ++++++++++++++++++++++++++++ > > > > 6 files changed, 113 insertions(+), 3 deletions(-) > > > > create mode 100755 test/pasta/ndp.py > > > > create mode 100644 test/tasst/pasta.py > > > > > > > > diff --git a/test/Makefile b/test/Makefile > > > > index f66c7e7e..95e3d75e 100644 > > > > --- a/test/Makefile > > > > +++ b/test/Makefile > > > > @@ -67,7 +67,7 @@ ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS) > > > > > > > > EXETER_SH = smoke/smoke.sh build/static_checkers.sh > > > > EXETER_PYPATH = exeter/py3:tunbridge/:. > > > > -EXETER_PYTHON = smoke/smoke.py build/build.py > > > > +EXETER_PYTHON = smoke/smoke.py build/build.py pasta/ndp.py > > > > EXETER_BATS = $(EXETER_SH:%=%.bats) $(EXETER_PYTHON:%=%.bats) > > > > BATS_FILES = $(EXETER_BATS) \ > > > > podman/test/system/505-networking-pasta.bats > > > > diff --git a/test/pasta/dhcp b/test/pasta/dhcp > > > > index e1c66be6..61279fbf 100644 > > > > --- a/test/pasta/dhcp > > > > +++ b/test/pasta/dhcp > > > > @@ -18,6 +18,11 @@ test Interface name > > > > nsout IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname' > > > > check [ -n "__IFNAME__" ] > > > > > > > > +# Bring up the interface > > > > +ns ip link set dev __IFNAME__ up > > > > +# Wait for SLAAC & DAD to complete > > > > +ns while ! ip -j -6 addr show dev __IFNAME__ | jq -e '.[].addr_info.[] | select(.protocol == "kernel_ra")'; do sleep 0.1; done > > > > + > > > > test DHCP: address > > > > ns /sbin/dhclient -4 --no-pid __IFNAME__ > > > > nsout ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local' > > > > diff --git a/test/pasta/ndp.py b/test/pasta/ndp.py > > > > new file mode 100755 > > > > index 00000000..8c7ce31e > > > > --- /dev/null > > > > +++ b/test/pasta/ndp.py > > > > @@ -0,0 +1,59 @@ > > > > +#! /usr/bin/env python3 > > > > +# > > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > > +# > > > > +# test/pasta/ndp.py - pasta NDP functionality > > > > +# > > > > +# Copyright Red Hat > > > > +# Author: David Gibson > > > > + > > > > +import contextlib > > > > +import dataclasses > > > > +from typing import Iterator > > > > + > > > > +import exeter > > > > +import tunbridge > > > > +import tasst > > > > + > > > > + > > > > +@dataclasses.dataclass > > > > +class UnconfiguredScenario(exeter.Scenario): > > > > + """Tests for a pasta instance without --config-net""" > > > > + > > > > + host: tunbridge.Site > > > > + guest: tunbridge.Site > > > > + ifname: str > > > > + addr6: tunbridge.ip.AddrMask6 > > > > + gw6: tunbridge.ip.Addr6 > > > > > > Until this point, it looks like stuff I can happily copy and paste, > > > and grasp, even. But then: > > > > > > > + @exeter.scenariotest > > > > + def test_ifname(self) -> None: > > > > + ifs = tunbridge.ip.ifs(self.guest) > > > > + exeter.assert_eq(set(ifs), {'lo', self.ifname}) > > > > > > ...why does a "Scenario" have a .ifname? > > > > Yeah, the readability of the Scenario mechanism was something I was > > particularly concerned about. I think the concept is valuable, but > > I'm very open to different ways of naming or organising it, if we can > > up with something better. > > From the description you give below, the name seems to fit. > > > A "Scenario" (specifically a subclass of exeter.Scenario) is a group > > of tests with a common set of parameters. In this case > > UnconfiguredScenario is a bunch of tests about the behaviour of pasta > > without --config-net. Each of those tests has access to the host and > > guest sites, the expected interface name, address and gateway in the > > guest - that is, the contents of an UncofiguredScenario instance. > > I'm not sure if I understand this correctly, but if each guest has a > single interface, that sounds a bit limiting.

Sorry, to be clear: a Scenario in the general sense can contain whatever parameters you like. This *particular* Scenario - UnconfiguredScenario - has just those things, because those are all that its tests require.

Ah, okay. Still, if I now want to take UnconfiguredScenario and add a couple of dummy interfaces to it for a quick test, I guess I have the choice to either do that with some "external" hack, or... copy and rename it, so that it doesn't affect all the usages?

No. A Scenario instance isn't responsible for managing the simulated environment - that's the setup function - it's just conveying the information about it that the tests need. So, you can make a setup function that adds the dummy interfaces, and still yield an UnconfiguredScenario. It doesn't need to have information about the dummy interfaces because the tests carried by UnconfiguredScenario don't care about them.

Oh, sorry, it's a class, of course, I see now.

...

The scenario mechanism does several things: 1) Groups together some related (parameterized) tests 2) Allows all of those tests to be registered at once 3) Provides a mechanism for providing a bunch of information to those tests (without requiring them each to have a large set of direct parameters)

I'm aware that doing those things with the same construct may be confusing - it's just ways of doing them separately also seem confusing and/or awkward in their own ways. Maybe there's a better way, but I haven't spotted it yet.

It really is confusing to me, but the description above is rather clear so I'll try to propose something once I get to write some kind of setup function and test cases myself.

Makes sense.

...

...

...

...

> Actually, I think any abstraction that doesn't offer arbitrary sets of > (and relationships between) the objects shown via netlink (or, at > least, namespaces, links, routes, addresses, neighbours) might be > limiting and not generic enough.

Absolutely, and the abstraction does allow that.

> > That instance describes a real (simulated) environment in which we can > > run those tests. > > > > You use this by supplying a function which sets things up, then yields > > an UnconfiguredScenario instance describing what it set up. exeter > > will run all of the UnconfiguredScenario tests on the environment the > > setup function created, each one as a separate test case. > > This part is now clear to me, and I think it's not complicated to grasp > the concept vaguely but enough to copy, paste, and modify code doing > this.

Ok.

> It would be even better to hide this entirely, because "yielding a > scenario" is a Python thing. In general, there's an imperative part in > all this (bordering functional programming, but still, not descriptive) > which I struggle to see as beneficial. > > Here the tasks at hand are, roughly: > > 1. represent two network namespaces, with two interfaces each (loopback > and non-loopback), with pasta connecting one of the interfaces of the > inner one

There's a bit more to it than that - we need to specify the host's routing setup, because that will affect what pasta does. That's what simple_host() is about, creating a host with the single gateway routing that's our easiest / most common case.

Okay, sure, by "interfaces" I meant configured interfaces with addresses and a default route, too. But that doesn't really modify my point, that is:

...

> 2. bring up one of the interfaces > > 3. compare addresses > > ...and doing 1. like that is simply not... intuitive, I think.

I'm not really clear on what you're getting at here. There is an unavoidable tradeoff here between obviousness for a single case, versus reuseability for multiple related cases. Is it just that some of the relevant setup is hidden inside simple_host() that's the problem? Or is it something else?

...yes, one part is that it's hidden. Another part are, specifically, these lines:

host: tunbridge.Site guest: tunbridge.Site ifname: str

[...]

@exeter.scenariotest def test_ifname(self) -> None:

[...]

None of these clearly links to "two network namespaces: A, with interface a1 and address x1, ...".

Fair. This needs a docstring explaining the parameters / fields.

That might help a tiny bit but I think the syntax and notations are kind of self-explanatory.

My concern is at a more conceptual level, and it's better summarised below, but here, specifically, we're writing:

host: tunbridge.Site

to say:

give me the "host" network namespace

What's "me" in this sentence?

The writer of the test case, that is:

So, I'm pretty sure we have a fundamental misunderstanding here, but I'm not sure on whose part. That line is not saying "give me" anything. It's a type declaration saying that an 'UnconfiguredScenario' instance includes a field 'host' of type tunbridge.Site. That field is used to store the host namespace; the docstring should say that, but doesn't yet. The tests (methods of UnconfiguredScenario) can then reference the host namespace as 'self.host'. The setup function is responsible for populating the field - that's the 'host=simh.site' in simh_pasta_setup() below.

...

...

and to say that, in my ideal world, I would probably go for something on the line(s) of:

A

Not really following, I hope it will make more sense to me below.

if "host" is a site in "host: tunbridge.Site", which I'm calling "A" for brevity, I would like to say that with:

A

instead of:

A: tunbridge.Site

In the test? So it's already 'self.host', or 'self.A' if you prefer. You only need the type annotation when defining a new Scenario subclass. Or we could disable mypy and not even need it then, but I think type checking catches enough bugs to make it worthwhile.

...

...

...

...

I understand this is probably very close to the bare minimum you can get by modelling this all with actual code, and that's why I think actual (imperative/functional) code is usually not used to model/describe things.

Imperative/functional code as opposed to..?

...declarative.

Ok, I've generally seen functional programming put under the declarative heading. Or as my exam invigilator famously said once "Declaraytive Programming Paradijjums".

So, I guess that makes me unclear what does and doesn't count as declarative for you.

Hmm yes I see your point. I would have said that "purely functional programming" is declarative, and "functional programming" is a mix of declarative and imperative (in practice).

Eh, true. Kind of depends if you're talking Haskell or, say, Emacs Lisp. And even in Haskell 'do' constructs are quasi-imperative.

In any case, I guess I later clarified this aspect.

...

[snip]

...

...

...

...

Syntax certainly isn't irrelevant, but so far I haven't grasped what you dislike about the function syntax versus a specialized grammer. Is it: - The irritating silly parentheses? - Longish (qualified) function names? - The indentation from the with syntax? - Something else?

It's *also* the first two

Ok. First I can't easily change. Second can be mitigated by handling the imports differently.

...

(the indentation looks actually convenient),

Ok, good. I also think this is useful because it conveys the lifetime of each object, which will be important once we get to tests where you need to change things part way through.

...

but that's not my main point. My main point is that this isn't fundamentally declarative. You're turning it into something that resembles that, but the syntax is still from an imperative programming language.

And in my mind the main feature of a network (topology) simulator is that you describe the topology (and it will build it for you), not that you... have to build a description?

Using an example that's obviously familiar to you: think of taking a device tree for some system with a couple of USB and I²C busses and a flash controller, and writing all that in Python based on some form of "bus" module/component.

I mean... old school Open Firmware kind of is this, but with Forth instead of Python.

Okay, you can model data structures in Python, obviously, but that wasn't my point. Anyway, it's all in the example below.

...

...

Once one sees how practical device trees are for that, the Python version would look wrong, wouldn't it?

That really depends on the context. If I was making an API for building a device tree, I'd probably come up with something pretty like this.

...an API for building one, yes. But not if you were writing a device tree itself.

...

...

Now, while I think that some single bits of DTS syntax are unnecessarily complicated, conceptually, a "networking" device tree would look more usable to me than the approach you're taking.

Of course, we need the whole "testing" / exeter part as well, and test cases are fundamentally sequential/imperative.

But (sorry, it's been a few years I don't touch these):

namespace@1 { interfaces { lo { address = 127.0.0.1; }; eth0 { address = ...; }; }; routes { /* something simpler than ip -j ro sh ? */ }; }

...

link@... { vxlan { endpoints { a { ns = <&namespace@1>; }; b ...

...

this looks much more natural to me, as an input for a simulator (I would personally make the syntax much more "elastic" by just throwing a link into a namespace but I'm trying to keep it clean just for this example).

Aha, I think I finally get what you're saying. More below.

...

Maybe tunbridge implements this somewhere and I missed it? Or would this be part of a "Scenario" description eventually?

This is entirely unrelated to what Scenario is trying to accomplish. That may cause you to reconsider whether "Scenario" is a good name, which is ok.

So. A declarative way of defining networks would be nice to have.

From my perspective that's fundamental, rather. I gave it for granted.

...

I think doing it with the flexibility we want is much harder than you estimate.

I'll pretend I'm not commenting on this line by... oops. :)

Heh. Well, I'd love to be proved wrong on this one.

...

...

It looks easy for simple static situations like the examples above, but:

* If you want to describe a topology that changes partway through, that's a huge step up in complexity, and kind of necessarily reintroduces imperative elements.

But you can use JSON or a ton of other formats that allow for ordering of elements.

Just ordering network elements w.r.t. each other isn't enough. If you have a test where you want /this/ network topology - do some stuff - then change the toplogy to /that/ - do some more stuff. I'm not sure how you encode that in a way that isn't imperative, or at least pseudo-imperative.

Yeah, I'll come up with a proposal, this is rather clear to me: simply interleave declarative setups with imperative test steps.

...

...

Alternatively, one could add attributes for lifecycle and timing (think of nftables sets) but it looks much less intuitive than the alternative.

I wasn't previously familiar with nftables sets. I had a quick look at the docs, but I don't see how it relates to the current proposal.

Sorry, I took this for granted: I meant timeout attributes for, say, elements in sets. That is, making the lifecycle validity an attribute of some declared object, rather than encapsulating declarative blocks in a possibly imperative sequence (if needed).

...

...

...

Device tree absolutely suffers from this - that's what motivated the godawful runtime overlay mechanism, and right now, I'm struggling to find time to participate in the latest of many discussions about how to better handle devices which can be runtime added and removed.

I'm not suggesting that we use ANS Forth by the way.

Well, we agree on that at least :).

...

...

* If you want to build complex scenarios out of simpler ones, you need what amounts to a macro system.

There are a ton of ways. You can also use a filesystem and includes. Or simply definitions of blocks, not necessarily macros,

You want to parameterise the blocks - at which point it's basically macros.

...

and JSON implicitly gives you all that.

No, it doesn't. If you have a JSON sub-object that's repeated several times through a tree, you have to write it out, in full, each time. YAML does allow repeating, as do some other JSON extensions, but bare JSON does not. It still doesn't allow parameterisation of those blocks.

Until you meet RFC 6901.

Only sort of. That's describing a convention for how to reference JSON nodes with a string. It's still not a notion in JSON proper - whatever's consuming the JSON has to know to expect an RFC 6901 pointer and process it accordingly. It doesn't help in the slightest with parameterization.

...

...

As it's nothing security relevant, I would actually go with something that's in theory more complicated but in practice more digestible such as YAML.

I don't terribly like YAML, because I think there are a bunch of edge cases where it's not obvious reading it whether something is a list or object or somethine else. The typing and back-referencing would be useful for this problem, I'll grant you.

...so perhaps YAML as an optional human-barely-tolerating format that we translate to JSON?

Maybe.

Or TOML?

Maybe. I like TOML in general, but I feel like it's specific syntax isn't great for this use case.

...

...

...

Again, a big leap up in complexity. Device tree struggles with this too - it originated primarily as a machine->machine format, where having heaps of repeated information is fine. As it transitioned to being a human->machine format, not so much. Hence /include/, expression support and semi-standard running dts files through cpp before compilation. It's still pretty clunky in this regard.

It absolutely is, but that's because it was designed for a different purpose.

...

Plus.. I think the interpreter for this hypothetical declarative language would need an internal structure pretty similar to what tunbridge, so this is kind of already a first step towards it.

Okay, that's good to know.

I'm estimating I'm currently writing about 5-10 scripts per month, including pasta/iproute2 one-liners, setting up strange stuff, to reproduce / debug issues.

Given that this looks so fundamental for my usage I'm thinking that I could make at least part of this a priority of mine.

I realised I can implement netlink stuff and handling of networking configuration concepts quite fast with Rust and neli, so I'm pondering to write a proof of concept that can parse the example above (minus Forth notations, but with some kind of pointer) and make it create at least namespaces, links, addresses, and routes.

If it helps visualising how that could possibly look like with / in tunbridge itself, I'll take care of it soon rather than later.

The only little problem is that I'm much faster with Rust (because of neli) than I can possibly picture myself with Python, and that doesn't play along with tunbridge. But perhaps as a proof of concept it helps anyway?

It would. At earlier points I did consider writing tunbridge (or whatever I was calling the idea at the time) in Rust. In principle at least, the lifetime tracking would make a very natural way for ensuring you bring up / pull down things in a sensible order.

In practice, however, I almost instantly ran into intractable borrow checker problems. Specifically, Rust's notorious difficulty with references between parts of the same structure. That arises almost immediately once you start building composite objects out of smaller components:

struct Node { ... } struct Veth<'a> { node0: &'a Node, node1: &'a Node, ... }

Then you want something that represents two nodes with a veth between them and you get:

struct TwoNodes { node0: Node, node1: Node, veth: Veth<'??> }

There are ways around it, of course, but everything I found so far was either hideously unergonomic, depended on obscure crates or both. There are a bunch of language extensions that would help, but while much discussed, none have landed yet.

If a way to solve this nicely appears, I'm not against moving tunbridge to Rust. I mean, slightly more against it than I was months ago before I made a start in Python, but still.

Well, let's see how my draft turns out. I think there are obvious marketing reasons for Rust over Python, and a couple of technical ones too (speed, plus what I mentioned about neli), but there are also technical downsides as you point out.

Oh, agreed. At the moment those technical downsides seem pretty fatal to me, though. Oh I missed an option there: hideously unergonomic, depend on obscure crates or neuter the borrow checker even in the cases it would be really useful. Or several of the above.

...

...

In general, do you think there's something in particular I could contribute at this stage, if I want to see my declarative dream come true?

Yes. A bunch of pseudo-code examples - both the network declarations and example tests that might go with them.

[snip]

...

...

...

...

> > Path I can easily add. Version would require an extra invocation of > > pasta, which I don't really want to do. > > Ah, right, never mind. The path will be good enough for that. > > > > This part also looks > > > quite readable and intuitive to me without having looked into tunbridge > > > recently. > > > > Ok, that's promising. > > I mean, I think it's all usable for the moment, and perhaps a starting > point for some other kind of... front-end? I'm not sure. As I mentioned > I'm a bit worried about the potential for universal intuitiveness and > usability.

So am I, but I have to weigh it against being able to re-use both tests and setups without having to re-express both in each case.

I think setups written like that are reusable (or can be made reusable). My usability point is about other project/usages. For passt and pasta themselves, this level or reusability looks enough to me for the foreseeable future.

Even though, one day, I guess we might want to generate pseudo-random (fractal-tree-like?) topologies (and I was recently trying out a pasta-in-pasta-in-pasta-in-pasta-in-pasta setup to reproduce that HTTP/FIN issue). For that, a declarative approach would make things easier, I suppose.

Declarative, or imperative? I actually have something like that in tunbridge's selftests: a function that builds a stack of N nested namespaces. https://gitlab.com/dgibson/tunbridge/-/blob/main/tunbridge/unshare.py#L302

...but they are all the same. Think, for example, of connecting every odd-numbered pair with veth tunnels, and every even-numbered pair with pasta. Say: n1 <-- veth --> n2 <-- pasta --> n3 <-- veth --> n4.

That would certainly be possible. More complex, of course, but not dramtically so.

...

What's really well suited for this situation, in my experience, is a declarative description format that can be easily generated and manipulated by imperative code.

Ah, so there's both an imperative and declarative component. The idea in tunbridge is that you can do this, but rather than emit the "declarative part" as concrete text for another parser, it's emitted as a data structure (generally a bunch of wrappers around context managers).

It is true that as currently designed, tunbridge builds the data structure representation at the same time as building the actual simulated network. With a declarative language approach, building the description (language fragment) is separate from instantiating the simulation. Is that something you see as valuable? Or only a side effect of the other things about the declarative approach you like?

I see that as valuable by itself, mostly because those fragments can be generated much more easily if they're separated from the imperative part.

Ok. I'll look into separating 'construct the describing data structure' from 'instantiating the simulation' in tunbridge. It's not everything you want, but it's a start (and would be a step closer to something that could implement a more declarative approach). Might not be for the next spin though, it's a pretty substantial rework.

...

...

The name of this kind of "indirection" in computer science research currently escapes me, but I'm fairly sure there must be some theory about it. In any case, I can include something like this in my (now planned) proof of concept.

...still planned...

-- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson