- dev - passt

[PATCH v3 0/9] Add vhost-user support to passt (part 1)
by Laurent Vivier 08 Mar '24

08 Mar '24

v3: - add a patch that has been extracted from: "tcp: extract buffer management from tcp_send_flag()" -> "tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers()" - see detailed v3 history log in each patch - I didn't address the alignment problem when we provide a pointer to a sub-structure in the internal buffer structure. (for the last patches of the series). v2 comparing to vhost-user full part: - part 1 includes only preliminary patches (checksum, iovec, cleanup) - see detailed v2 history log in each patch. Full series v1 available at: [PATCH 00/24] Add vhost-user support to passt. https://url.corp.redhat.com/passt-vhost-user-v1 Thanks, Laurent Laurent Vivier (9): iov: add some functions to manage iovec pcap: add pcap_iov() checksum: align buffers checksum: add csum_iov() util: move IP stuff from util.[ch] to ip.[ch] checksum: use csum_ip4_header() in udp.c and tcp.c checksum: introduce functions to compute the header part checksum for TCP/UDP tap: make tap_update_mac() generic tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers() Makefile | 12 +-- checksum.c | 171 ++++++++++++++++++++++++++++++----------- checksum.h | 9 ++- conf.c | 1 + dhcp.c | 1 + flow.c | 1 + icmp.c | 1 + iov.c | 171 +++++++++++++++++++++++++++++++++++++++++ iov.h | 29 +++++++ ip.c | 72 +++++++++++++++++ ip.h | 86 +++++++++++++++++++++ ndp.c | 1 + pcap.c | 62 +++++++++++++-- pcap.h | 1 + port_fwd.c | 1 + qrap.c | 1 + tap.c | 14 ++-- tap.h | 2 +- tcp.c | 213 +++++++++++++++++++++++++++++---------------------- tcp_splice.c | 1 + udp.c | 35 +++------ util.c | 55 ------------- util.h | 76 ------------------ 23 files changed, 703 insertions(+), 313 deletions(-) create mode 100644 iov.c create mode 100644 iov.h create mode 100644 ip.c create mode 100644 ip.h -- 2.42.0

3 37

[PATCH] iov: Improve documentation of iov_skip_bytes()
by David Gibson 08 Mar '24

08 Mar '24

As pointed out in review, the documentation comments for iov_skip_bytes() are more confusing than they should be. Reword them, including updating parameter names, to make it clearer. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- iov.c | 28 ++++++++++++++-------------- iov.h | 2 +- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/iov.c b/iov.c index e3312628..8e850180 100644 --- a/iov.c +++ b/iov.c @@ -26,31 +26,31 @@ #include "iov.h" -/* iov_skip_bytes() - Skip the first n bytes into an IO vector +/* iov_skip_bytes() - Skip leading bytes of an IO vector * @iov: IO vector * @n: Number of entries in @iov - * @vec_offset: Total byte offset into the IO vector - * @buf_offset: Offset into a single buffer of the IO vector + * @skip: Number of leading bytes of @iov to skip + * @offset: Offset of first unskipped byte in its @iov entry * - * Return: index I of individual struct iovec which contains the byte at - * @vec_offset bytes into the vector (as though all its buffers were - * contiguous). If @buf_offset is non-NULL, update it to the offset of - * that byte within @iov[I] (guaranteed to be less than @iov[I].iov_len) - * If the whole vector has <= @vec_offset bytes, return @n. + * Return: index I of individual struct iovec which contains the byte at @skip + * bytes into the vector (as though all its buffers were contiguous). + * If @offset is non-NULL, update it to the offset of that byte within + * @iov[I] (guaranteed to be less than @iov[I].iov_len) If the whole + * vector has <= @skip bytes, return @n. */ size_t iov_skip_bytes(const struct iovec *iov, size_t n, - size_t vec_offset, size_t *buf_offset) + size_t skip, size_t *offset) { - size_t offset = vec_offset, i; + size_t off = skip, i; for (i = 0; i < n; i++) { - if (offset < iov[i].iov_len) + if (off < iov[i].iov_len) break; - offset -= iov[i].iov_len; + off -= iov[i].iov_len; } - if (buf_offset) - *buf_offset = offset; + if (offset) + *offset = off; return i; } diff --git a/iov.h b/iov.h index e1becdea..6058af77 100644 --- a/iov.h +++ b/iov.h @@ -19,7 +19,7 @@ #include <string.h> size_t iov_skip_bytes(const struct iovec *iov, size_t n, - size_t vec_offset, size_t *buf_offset); + size_t skip, size_t *offset); size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, const void *buf, size_t bytes); size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, -- 2.44.0

2 1

[PATCH v2 0/9] vhost-user part 1, v6
by David Gibson 06 Mar '24

06 Mar '24

When I rebased by own code, I hit the problems Stefano mentioned with static checker regressions in the vhost-user series. Laurent's most recent vhost-user part 1 series, but with fixes applied for the cppcheck and clang-tidy regressions introduced. Changes since 'v5 + static checker fixes' * Added my own alteration notes and S-o-b Changes since v5: * Fixed a number of static checker regressions Laurent Vivier (9): pcap: add pcap_iov() checksum: align buffers checksum: add csum_iov() util: move IP stuff from util.[ch] to ip.[ch] udp: little cleanup in udp_update_hdrX() to prepare future changes checksum: use csum_ip4_header() in udp.c and tcp.c checksum: introduce functions to compute the header part checksum for TCP/UDP tap: make tap_update_mac() generic tcp: Introduce tcp_fill_headers4()/tcp_fill_headers6() Makefile | 10 +-- checksum.c | 173 ++++++++++++++++++++++++++++++---------- checksum.h | 12 ++- conf.c | 1 + dhcp.c | 1 + flow.c | 1 + fwd.c | 1 + icmp.c | 1 + inany.c | 1 + iov.c | 1 - ip.c | 72 +++++++++++++++++ ip.h | 86 ++++++++++++++++++++ ndp.c | 1 + pcap.c | 27 ++++++- pcap.h | 1 + qrap.c | 1 + tap.c | 13 +-- tap.h | 2 +- tcp.c | 221 +++++++++++++++++++++++++++++---------------------- tcp_splice.c | 1 + udp.c | 82 ++++++++----------- util.c | 55 ------------- util.h | 76 ------------------ 23 files changed, 505 insertions(+), 335 deletions(-) create mode 100644 ip.c create mode 100644 ip.h -- 2.44.0

2 10

[PATCH 0/9] vhost-user part 1, v5 + static checker fixes
by David Gibson 06 Mar '24

06 Mar '24

Stefano, you may have already done this, but I hit the same issue trying to rebase my own code on the vhost-user stuff. This is Laurent's most recent vhost-user part 1 series, but with fixes applied for the cppcheck and clang-tidy regressions introduced. Laurent Vivier (9): pcap: add pcap_iov() checksum: align buffers checksum: add csum_iov() util: move IP stuff from util.[ch] to ip.[ch] udp: little cleanup in udp_update_hdrX() to prepare future changes checksum: use csum_ip4_header() in udp.c and tcp.c checksum: introduce functions to compute the header part checksum for TCP/UDP tap: make tap_update_mac() generic tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers() Makefile | 10 +-- checksum.c | 173 ++++++++++++++++++++++++++++++---------- checksum.h | 12 ++- conf.c | 1 + dhcp.c | 1 + flow.c | 1 + fwd.c | 1 + icmp.c | 1 + inany.c | 1 + iov.c | 1 - ip.c | 72 +++++++++++++++++ ip.h | 86 ++++++++++++++++++++ ndp.c | 1 + pcap.c | 27 ++++++- pcap.h | 1 + qrap.c | 1 + tap.c | 13 +-- tap.h | 2 +- tcp.c | 221 +++++++++++++++++++++++++++++---------------------- tcp_splice.c | 1 + udp.c | 82 ++++++++----------- util.c | 55 ------------- util.h | 76 ------------------ 23 files changed, 505 insertions(+), 335 deletions(-) create mode 100644 ip.c create mode 100644 ip.h -- 2.44.0

2 12

[PATCH v5 0/9] Add vhost-user support to passt (part 1)
by Laurent Vivier 06 Mar '24

06 Mar '24

v5: - add a patch to cleanup before change: udp: little cleanup in udp_update_hdrX() to prepare future changes - see detailed v5 history log in each patch v4: - rebase - see detailed v4 history log in each patch v3: - add a patch that has been extracted from: "tcp: extract buffer management from tcp_send_flag()" -> "tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers()" - see detailed v3 history log in each patch - I didn't address the alignment problem when we provide a pointer to a sub-structure in the internal buffer structure. (for the last patches of the series). v2 comparing to vhost-user full part: - part 1 includes only preliminary patches (checksum, iovec, cleanup) - see detailed v2 history log in each patch. Full series v1 available at: [PATCH 00/24] Add vhost-user support to passt. https://url.corp.redhat.com/passt-vhost-user-v1 Thanks, Laurent Laurent Vivier (9): pcap: add pcap_iov() checksum: align buffers checksum: add csum_iov() util: move IP stuff from util.[ch] to ip.[ch] udp: little cleanup in udp_update_hdrX() to prepare future changes checksum: use csum_ip4_header() in udp.c and tcp.c checksum: introduce functions to compute the header part checksum for TCP/UDP tap: make tap_update_mac() generic tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers() Makefile | 10 +-- checksum.c | 174 +++++++++++++++++++++++++++++----------- checksum.h | 10 ++- conf.c | 1 + dhcp.c | 1 + flow.c | 1 + fwd.c | 1 + icmp.c | 1 + inany.c | 1 + ip.c | 72 +++++++++++++++++ ip.h | 86 ++++++++++++++++++++ ndp.c | 1 + pcap.c | 26 +++++- pcap.h | 1 + qrap.c | 1 + tap.c | 13 +-- tap.h | 2 +- tcp.c | 221 +++++++++++++++++++++++++++++---------------------- tcp_splice.c | 1 + udp.c | 82 ++++++++----------- util.c | 55 ------------- util.h | 76 ------------------ 22 files changed, 503 insertions(+), 334 deletions(-) create mode 100644 ip.c create mode 100644 ip.h -- 2.42.0

3 14

[PATCH v4 0/8] Add vhost-user support to passt (part 1)
by Laurent Vivier 03 Mar '24

03 Mar '24

v4: - rebase - see detailed v4 history log in each patch v3: - add a patch that has been extracted from: "tcp: extract buffer management from tcp_send_flag()" -> "tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers()" - see detailed v3 history log in each patch - I didn't address the alignment problem when we provide a pointer to a sub-structure in the internal buffer structure. (for the last patches of the series). v2 comparing to vhost-user full part: - part 1 includes only preliminary patches (checksum, iovec, cleanup) - see detailed v2 history log in each patch. Full series v1 available at: [PATCH 00/24] Add vhost-user support to passt. https://url.corp.redhat.com/passt-vhost-user-v1 Thanks, Laurent Laurent Vivier (8): pcap: add pcap_iov() checksum: align buffers checksum: add csum_iov() util: move IP stuff from util.[ch] to ip.[ch] checksum: use csum_ip4_header() in udp.c and tcp.c checksum: introduce functions to compute the header part checksum for TCP/UDP tap: make tap_update_mac() generic tcp: Introduce ipv4_fill_headers()/ipv6_fill_headers() Makefile | 10 +-- checksum.c | 173 ++++++++++++++++++++++++++++++----------- checksum.h | 9 ++- conf.c | 1 + dhcp.c | 1 + flow.c | 1 + fwd.c | 1 + icmp.c | 1 + inany.c | 1 + ip.c | 72 +++++++++++++++++ ip.h | 86 +++++++++++++++++++++ ndp.c | 1 + pcap.c | 26 ++++++- pcap.h | 1 + qrap.c | 1 + tap.c | 14 ++-- tap.h | 2 +- tcp.c | 214 +++++++++++++++++++++++++++++---------------------- tcp_splice.c | 1 + udp.c | 38 ++++----- util.c | 55 ------------- util.h | 76 ------------------ 22 files changed, 475 insertions(+), 310 deletions(-) create mode 100644 ip.c create mode 100644 ip.h -- 2.42.0

2 16

[PATCH v2 0/7] Allow more use of iovecs in pcap and tap interfaces
by David Gibson 01 Mar '24

01 Mar '24

While working on other stuff I stumbled across some patches I wrote quite a while back (part of a larger series on indefinite hiatus). These make it easier to use the pcap and tap functions with frames that aren't in a single contiguous buffer. This overlaps with some of the work in Laurent's vhost-user series, so I've included the first of his patches and integrated it with my changes. There will The first two patches have some overlap with the preliminary iovec patches in the vhost-user series (and will certainly conflict). I do think the pcap interface here is slightly better than the one in the vhost-user series, although I did ack that. Stefano, if you've already applied / run tests for Laurent's series then go ahead with it; I'll rework this on top of those. Changes since v1: * Stefano correctly pointed out that my iov_offset() function was unclear in its naming and description. Rename it to iov_skip_bytes() and change the wording around it to improve this. * Incorporate Laurent's iov helper patch, and also use iov_skip_bytes() to slightly simplify its functions. * Fix commit message for write_remainder() so that it correctly describes handling "short writes", not "sort writes", whatever that might mean. * Adjust parameter names and descriptions for pcap_multiple() for clarity. David Gibson (6): iov: Add helper to find skip over first n bytes of an io vector pcap: Update pcap_frame() to take an iovec and offset util: Add write_remainder() helper pcap: Handle short writes in pcap_frame() pcap: Allow pcap_frame() and pcap_multiple() to take multi-buffer frames tap: Use write_remainder() in tap_send_frames_passt() Laurent Vivier (1): iov: add some functions to manage iovec Makefile | 8 +-- iov.c | 198 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ iov.h | 31 +++++++++ pcap.c | 56 ++++++++-------- pcap.h | 3 +- tap.c | 41 +++--------- util.c | 33 ++++++++++ util.h | 1 + 8 files changed, 307 insertions(+), 64 deletions(-) create mode 100644 iov.c create mode 100644 iov.h -- 2.43.2

2 11

[PATCH v3 00/20] More flow table preliminaries: address handling improvements
by David Gibson 29 Feb '24

29 Feb '24

Here's another batch of cleanups and tweaks in preparation for the flow table. This set focuses on improved helpers for handling addresses, particularly in the TCP splice path. Based on my other series adding more iovecs to the tap and pcap code, however the only conflicts should be trivial Makefile collisions. Changes since v2: * Minor stylistic and formatting changes based on review * Some clarifying changes to the theory of operation notes on flow lifecycle * Rebased on top of new series cleaning up socket pool error handling. This removes a couple of patches from this series. * Small edits to commit message for improved clarity Changes since v1: * Rebased, and reordered in a way I hope is clearer * Add patch to rename port_fwd.[ch] * Added doc comments to clarify flow life cycle * Added uniform logging of flow start / end to match that lifecycle * union inany_addr typed special address constants * inany based tests for unspecified and multicast addresses, as well as loopback * Dropped patch allowing NULL parameter to inany_from_af(), turned out not to be that useful * Dropped sockaddr_any_init function, turned out not to be very useful in that form * Added patch enforcing no loopback addresses on tap interface * Added logic to sanity check TCP endpoint addresses * Moved socket creation into tcp_splice_connect() * Moved epoll ref parsing into tcp_listen_handler() * Allowed IN4_IS_*() helpers to work on void * addresses David Gibson (20): inany: Helper to test for various address types inany: Add inany_ntop() helper inany: Provide more conveniently typed constants for special addresses inany: Introduce union sockaddr_inany util: Allow IN4_IS_* macros to operate on untyped addresses tcp, udp: Don't precompute port remappings in epoll references flow: Add helper to determine a flow's protocol tcp_splice: Simplify clean up logic tcp_splice: Don't use flow_trace() before setting flow type flow: Clarify flow entry life cycle, introduce uniform logging tcp_splice: More specific variable names in new splice path tcp_splice: Merge tcp_splice_new() into its caller tcp_splice: Make tcp_splice_connect() create its own sockets tcp_splice: Improve error reporting on connect path tcp_splice: Improve logic deciding when to splice tcp, tcp_splice: Parse listening socket epoll ref in tcp_listen_handler() tcp: Validate TCP endpoint addresses tap: Disallow loopback addresses on tap interface port_fwd: Fix copypasta error in port_fwd_scan_udp() comments fwd: Rename port_fwd.[ch] and their contents Makefile | 12 ++-- conf.c | 8 +-- flow.c | 84 ++++++++++++++++++++++- flow.h | 9 +++ port_fwd.c => fwd.c | 32 ++++----- port_fwd.h => fwd.h | 24 +++---- icmp.c | 18 ++--- inany.c | 50 ++++++++++++++ inany.h | 96 ++++++++++++++++++++++---- passt.h | 2 +- tap.c | 19 ++++++ tcp.c | 119 +++++++++++++++++++++++--------- tcp.h | 6 +- tcp_splice.c | 162 +++++++++++++++++++++++++------------------- tcp_splice.h | 7 +- udp.c | 32 +++++---- udp.h | 10 +-- util.h | 8 +-- 18 files changed, 502 insertions(+), 196 deletions(-) rename port_fwd.c => fwd.c (83%) rename port_fwd.h => fwd.h (62%) create mode 100644 inany.c -- 2.43.2

2 21

[PATCH 0/6] udp: Small cleanups
by David Gibson 29 Feb '24

29 Feb '24

Following on from the recent set of small fixes for UDP, here are a number of small cleanups to the UDP code, which will simplify later more complex fixes and improvements. Laurent, I expect this will have some conflicts with part 2 of your vhost-user work, sorry. David Gibson (6): udp: Refactor udp_sock[46]_iov_init() udp: Tweak interface to udp_update_hdr[46] udp: Clarify setting of addresses inin udp_update_hdr[46]() udp: Consistent port variable names in udp_update_hdr[46] udp: Avoid unnecessary pointer in udp_update_hdr4() udp: Clean up UDP checksum generation for inbound IPv6 packets udp.c | 235 +++++++++++++++++++++++++++------------------------------- 1 file changed, 108 insertions(+), 127 deletions(-) -- 2.44.0

1 6

[PATCH v2 0/5] Assorted small fixes for UDP
by David Gibson 29 Feb '24

29 Feb '24

Partly in looking at flow table support, partly for other reasons, I've spotted a number of small fixes that can be made to the UDP code (in addition to the large fixes I'm still working on). These are ready to go now, so here they are. They'll go before the flow related addressing cleanups. Changes since v1: * Removed patch to unconditionally remove ACT flags, which was Just Plain Wrong. * Removed patch to sort out incorrect handling of port flags. While there's a real bug there, it interacts with at least two more bugs that I hadn't spotted, complicating the picture. I'll fix those some other day. David Gibson (5): udp: Don't attempt to translate a 0.0.0.0 source address udp: Set pif in epoll reference for ephemeral host sockets udp: Small streamline to udp_update_hdr4() udp: Fix incorrect usage of IPv6 state in IPv4 path udp: Remove unnecessary test for unspecified addr_out udp.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) -- 2.43.2

2 6