August 2023 - dev

[PATCH v3 00/13] Clean up to tap errors and epoll dispatch
by David Gibson 13 Aug '23

13 Aug '23

Getting from an epoll event to the relevant handler function is currently several levels of functions and tests. This series simplifies this to be pretty close to a single switch on a value in the epoll ref dispatching directly to the appropriate handler. Doing this requires some preliminary cleaning up of the handling of errors or disconnects on the tap device. Changes since v2: * Removed a (pre-existing) redundant close() and EPOLL_CTL_DEL in one of the tap reset paths * Fixed incorrect parameter to trace() message Changes since v1: * Give listening TCP sockets their own reference type * Fold "tap reset" series into this one Changes since v2 of tap reset series: * More thorough cleanup of handling error events on the listening Unix socket. Changes since v1 of the tap reset series: * Two extra patches that further clean up the reset path David Gibson (13): tap: Clean up tap reset path tap: Clean up behaviour for errors on listening Unix socket tap: Fold reset handling into tap_handler_pasta() tap: Fold reset handling into tap_handler_passt() epoll: Generalize epoll_ref to cover things other than sockets epoll: Always use epoll_ref for the epoll data variable epoll: Fold sock_handler into general switch on epoll event fd epoll: Split handling of ICMP and ICMPv6 sockets epoll: Tiny cleanup to udp_sock_handler() epoll: Split handling of TCP timerfds into its own handler function epoll: Split handling of listening TCP sockets into their own handler epoll: Split listening Unix domain socket into its own type epoll: Use different epoll types for passt and pasta tap fds icmp.c | 118 ++++++++++++++++++++++++------------------- icmp.h | 9 ++-- passt.c | 90 +++++++++++++++++++-------------- passt.h | 56 ++++++++++++++++----- pasta.c | 8 ++- tap.c | 138 +++++++++++++++++++++++++-------------------------- tap.h | 7 ++- tcp.c | 84 +++++++++++++------------------ tcp.h | 28 +++++++---- tcp_conn.h | 4 +- tcp_splice.c | 8 +-- tcp_splice.h | 2 +- udp.c | 16 +++--- util.c | 27 +++++++--- 14 files changed, 334 insertions(+), 261 deletions(-) -- 2.41.0

2 14

[PATCH 0/6] Flow Table Preliminaries
by David Gibson 11 Aug '23

11 Aug '23

I'm still working on bunch of things to start implementing the generalised flow table. However, I think this set of preliminary clean ups and fixes stand well enough on their own that they're ready for merge now. Based on the epoll patch series. David Gibson (6): tap: Don't clobber source address in tap6_handler() tap: Pass source address to protocol handler functions tcp: More precise terms for addresses and ports tcp: Consistent usage of ports in tcp_seq_init() tcp, udp: Don't include destination address in partially precomputed csums tcp, udp: Don't pre-fill IPv4 destination address in headers icmp.c | 12 ++-- icmp.h | 3 +- passt.c | 10 ++- passt.h | 4 +- pasta.c | 2 +- tap.c | 29 ++++----- tcp.c | 188 ++++++++++++++++++++++++----------------------------- tcp.h | 5 +- tcp_conn.h | 12 ++-- udp.c | 37 ++++------- udp.h | 5 +- util.h | 4 +- 12 files changed, 142 insertions(+), 169 deletions(-) -- 2.41.0

1 7

[PATCH 0/9] Clean up to epoll dispatch
by David Gibson 11 Aug '23

11 Aug '23

Getting from an epoll event to the relevant handler function is currently several levels of functions and tests. This series simplifies this to be pretty close to a single switch on a value in the epoll ref dispatching directly to the appropriate handler. Based on the tap reset series. David Gibson (9): epoll: Generalize epoll_ref to cover things other than sockets epoll: Always use epoll_ref for the epoll data variable epoll: Fold sock_handler into general switch on epoll event fd epoll: Split handling of ICMP and ICMPv6 sockets epoll: Tiny cleanup to udp_sock_handler() epoll: Split handling of TCP timerfds into its own handler function epoll: Split handling of listening TCP sockets into their own handler epoll: Split listening Unix domain socket into its own type epoll: Use different epoll types for passt and pasta tap fds icmp.c | 118 +++++++++++++++++++++++++++++---------------------- icmp.h | 9 ++-- passt.c | 90 +++++++++++++++++++++++---------------- passt.h | 52 ++++++++++++++++++----- pasta.c | 8 +++- tap.c | 63 +++++++++++++-------------- tap.h | 7 ++- tcp.c | 64 ++++++++++------------------ tcp.h | 11 +++-- tcp_conn.h | 4 +- tcp_splice.c | 4 +- udp.c | 16 +++---- util.c | 27 +++++++++--- 13 files changed, 263 insertions(+), 210 deletions(-) -- 2.41.0

2 16

[PATCH v2 00/13] Clean up to tap errors and epoll dispatch
by David Gibson 11 Aug '23

11 Aug '23

Getting from an epoll event to the relevant handler function is currently several levels of functions and tests. This series simplifies this to be pretty close to a single switch on a value in the epoll ref dispatching directly to the appropriate handler. Doing this requires some preliminary cleaning up of the handling of errors or disconnects on the tap device. Changes since v1: * Give listening TCP sockets their own reference type * Fold "tap reset" series into this one Changes since v2 of tap reset series: * More thorough cleanup of handling error events on the listening Unix socket. Changes since v1 of the tap reset series: * Two extra patches that further clean up the reset path David Gibson (13): tap: Clean up tap reset path tap: Clean up behaviour for errors on listening Unix socket tap: Fold reset handling into tap_handler_pasta() tap: Fold reset handling into tap_handler_passt() epoll: Generalize epoll_ref to cover things other than sockets epoll: Always use epoll_ref for the epoll data variable epoll: Fold sock_handler into general switch on epoll event fd epoll: Split handling of ICMP and ICMPv6 sockets epoll: Tiny cleanup to udp_sock_handler() epoll: Split handling of TCP timerfds into its own handler function epoll: Split handling of listening TCP sockets into their own handler epoll: Split listening Unix domain socket into its own type epoll: Use different epoll types for passt and pasta tap fds icmp.c | 118 +++++++++++++++++++++++++-------------------- icmp.h | 9 ++-- passt.c | 90 ++++++++++++++++++++-------------- passt.h | 56 +++++++++++++++++----- pasta.c | 8 +++- tap.c | 133 ++++++++++++++++++++++++++------------------------- tap.h | 7 ++- tcp.c | 84 ++++++++++++++------------------ tcp.h | 28 +++++++---- tcp_conn.h | 4 +- tcp_splice.c | 8 ++-- tcp_splice.h | 2 +- udp.c | 16 +++---- util.c | 27 ++++++++--- 14 files changed, 334 insertions(+), 256 deletions(-) -- 2.41.0

2 17

[PATCH v2 0/4] Clean up reset of tap connections
by David Gibson 07 Aug '23

07 Aug '23

I ran into some mildly confusing stuff in the tap reset path while working on the epoll rework. Here are some fixes for it. Changes since v1: * Two extra patches that further clean up the reset path David Gibson (4): tap: Clean up tap reset path tap: More sensible behaviour for error on listening qemu socket tap: Fold reset handling into tap_handler_pasta() tap: Fold reset handling into tap_handler_passt() tap.c | 94 ++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 51 insertions(+), 43 deletions(-) -- 2.41.0

1 4

[PATCH 0/3] Better report errors failing to open namespace tap device
by David Gibson 04 Aug '23

04 Aug '23

In https://github.com/containers/podman/issues/19428, pasta is failing to open the namespace tap device. Paul Holzinger correctly noted that pasta isn't very helpful in this case, with no information beyond "it failed". He suggested a patch for that, however it wasn't quite sufficient: errno may not be propagated back from the ephemeral thread which enters the namespace, and even if it does the errno alone won't tell us which of the possible failure points actually failed. 2/3 here is a more robust change to address the problem. The other patches are minor cleanups I noticed along the way. Link: https://bugs.passt.top/show_bug.cgi?id=69 David Gibson (3): util: Make ns_enter() a void function and report setns() errors tap: More detailed error reporting in tap_ns_tun() tap: Remove unnecessary global tun_ns_fd conf.c | 3 ++- tap.c | 33 ++++++++++++++++++--------------- udp.c | 6 ++---- util.c | 8 +++----- util.h | 2 +- 5 files changed, 26 insertions(+), 26 deletions(-) -- 2.41.0

2 5

[PATCH 0/2] Clean up reset of tap connections
by David Gibson 04 Aug '23

04 Aug '23

I ran into some mildly confusing stuff in the tap reset path while working on the epoll rework. Here's a couple of fixes for it. David Gibson (2): tap: Clean up tap reset path tap: More sensible behaviour for error on listening qemu socket tap.c | 57 ++++++++++++++++++++++++++++++++------------------------- 1 file changed, 32 insertions(+), 25 deletions(-) -- 2.41.0

1 2

[PATCH v2 00/17] netlink fixes and cleanups
by David Gibson 04 Aug '23

04 Aug '23

We've had several bugs in the past that were quite tricky to debug, but would have been much easier if we'd known that a netlink operation had failed. So, it would be desirable to actually detect and report failures of netlink operations. While working on that, I discovered that there are a number of other issues ranging from very small to medium sized with the way we use netlink. This series addresses many of them. Link: https://bugs.passt.top/show_bug.cgi?id=60 Link: https://bugs.passt.top/show_bug.cgi?id=67 This series was tested as based on the pending patches adding C11 support, though I believe it trivially rebases onto current main. Changes since v1: * Assorted minor fixes based on Stefano's review * Rebased on C11 branch David Gibson (17): netlink: Split up functionality of nl_link() netlink: Split nl_addr() into separate operation functions netlink: Split nl_route() into separate operation functions netlink: Use struct in_addr for IPv4 addresses, not bare uint32_t netlink: Explicitly pass netlink sockets to operations netlink: Make nl_*_dup() use a separate datagram for each request netlink: Start sequence number from 1 instead of 0 netlink: Treat send() or recv() errors as fatal netlink: Fill in netlink header fields from nl_req() netlink: Add nl_do() helper for simple operations with error checking netlink: Clearer reasoning about the netlink response buffer size netlink: Split nl_req() to allow processing multiple response datagrams netlink: Add nl_foreach_oftype to filter response message types netlink: Propagate errors for "set" operations netlink: Always process all responses to a netlink request netlink: Propagate errors for "dump" operations netlink: Propagate errors for "dup" operations conf.c | 66 ++++- netlink.c | 843 ++++++++++++++++++++++++++++++++++-------------------- netlink.h | 27 +- pasta.c | 75 +++-- 4 files changed, 658 insertions(+), 353 deletions(-) -- 2.41.0

2 18

[PATCH 0/3] RFC: Allow C11 extensions in the passt/pasta code
by David Gibson 04 Aug '23

04 Aug '23

As discussed on our recent calls, the C11 standard introduces anonymous structure and union members and static assertions, amongst other things. Both of these could be useful in a few places in passt/pasta to make the code more readable and safer. However, at the moment, the compiler flags we use only allow C99 code. This series allows C11 code, and makes some fairly obvious cleanups by using it. It would be nice to get an opinion on this reasonably quickly, because I have other patches in the works that will look different depending on whether or not they can use C11 features. David Gibson (3): Allow C11 code, not just C99 code Use C11 anonymous members to make poll refs less verbose to use Use static assertion to verify that union epoll_ref is the right size Makefile | 4 ++-- icmp.c | 22 +++++++++++----------- icmp.h | 2 +- passt.c | 8 ++++---- passt.h | 8 ++++++-- tcp.c | 46 +++++++++++++++++++++++----------------------- tcp.h | 2 +- tcp_splice.c | 11 +++++------ udp.c | 50 +++++++++++++++++++++++--------------------------- udp.h | 2 +- util.c | 4 ++-- 11 files changed, 79 insertions(+), 80 deletions(-) -- 2.41.0

2 7

[PATCH 00/17] netlink fixes and cleanups
by David Gibson 03 Aug '23

03 Aug '23

We've had several bugs in the past that were quite tricky to debug, but would have been much easier if we'd known that a netlink operation had failed. So, it would be desirable to actually detect and report failures of netlink operations. While working on that, I discovered that there are a number of other issues ranging from very small to medium sized with the way we use netlink. This series addresses many of them. Link: https://bugs.passt.top/show_bug.cgi?id=60 Link: https://bugs.passt.top/show_bug.cgi?id=67 David Gibson (17): netlink: Split up functionality if nl_link() netlink: Split nl_addr() into separate operation functions netlink: Split nl_route() into separate operation functions netlink: Use struct in_addr for IPv4 addresses, not bare uint32_t netlink: Explicitly pass netlink sockets to operations netlink: Make nl_*_dup() use a separate datagram for each request netlink: Start sequence number from 1 instead of 0 netlink: Treat send() or recv() errors as fatal netlink: Fill in netlink header fields from nl_req() netlink: Add nl_do() helper for simple operations with error checking netlink: Clearer reasoning about the netlink response buffer size netlink: Split nl_req() to allow processing multiple response datagrams netlink: Add nl_foreach_oftype to filter response message types netlink: Propagate errors for "set" operations netlink: Always process all responses to a netlink request netlink: Propagate errors for "dump" operations netlink: Propagate errors for "dup" operations conf.c | 66 ++++- netlink.c | 844 ++++++++++++++++++++++++++++++++++-------------------- netlink.h | 27 +- pasta.c | 75 +++-- 4 files changed, 659 insertions(+), 353 deletions(-) -- 2.41.0

2 34