- dev - passt

[PATCH v2 0/3] conf, log: Fix regression in usage() printing
by David Gibson 05 Jun '24

05 Jun '24

Erik Sjölund pointed out a regression where we're no long printing the --help message to stdout, but to stderr instead. I'm not actually sure what caused the regression, but it was made possible by the fact that the original fix was pretty strange to begin with. It introduced some hacks to force the logging subsystem to go to stdout, which have presumably failed in some subtle way. But, there's really no reason to run usage() through the logging subsystem in the first place - it just adds complexity. Link: https://bugs.passt.top/show_bug.cgi?id=90 Link: https://bugs.passt.top/show_bug.cgi?id=52 Changes since v1: * Use multi-line strings heavily in 2/3 David Gibson (3): conf: Remove unhelpful usage() wrapper conf: Don't print usage via the logging subsystem log: Remove log_to_stdout option conf.c | 337 ++++++++++++++++++++++++++++----------------------------- log.c | 8 +- log.h | 1 - 3 files changed, 170 insertions(+), 176 deletions(-) -- 2.45.1

2 4

[PATCH v8] tcp: move seq_to_tap update to when frame is queued
by Jon Maloy 05 Jun '24

05 Jun '24

commit a469fc393fa1 ("tcp, tap: Don't increase tap-side sequence counter for dropped frames") delayed update of conn->seq_to_tap until the moment the corresponding frame has been successfully pushed out. This has the advantage that we immediately can make a new attempt to transmit a frame after a failed trasnmit, rather than waiting for the peer to later discover a gap and trigger the fast retransmit mechanism to solve the problem. This approach has turned out to cause a problem with spurious sequence number updates during peer-initiated retransmits, and we have realized it may not be the best way to solve the above issue. We now restore the previous method, by updating the said field at the moment a frame is added to the outqueue. To retain the advantage of having a quick re-attempt based on local failure detection, we now scan through the part of the outqueue that had do be dropped, and restore the sequence counter for each affected connection to the most appropriate value. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v2: - Re-spun loop in tcp_revert_seq() and some other changes based on feedback from Stefano Brivio. - Added paranoid test to avoid that seq_to_tap becomes lower than seq_ack_from_tap. v3: - Identical to v2. Called v3 because it was embedded in a series with that version. v4: - In tcp_revert_seq(), we read the sequence number from the TCP header instead of keeping a copy in struct tcp_buf_seq_update. - Since the only remaining field in struct tcp_buf_seq_update is a pointer to struct tcp_tap_conn, we eliminate the struct altogether, and make the tcp6/tcp3_buf_seq_update arrays into arrays of said pointer. - Removed 'paranoid' test in tcp_revert_seq. If it happens, it is not fatal, and will be caught by other code anyway. - Separated from the series again. v5: - Changed way to index array within tcp_revert_seq() v6-v7: No changes. v8: Fixed missing indexing of tcp4/6_frame_conns array in tcp_payload_flush(). --- tcp.c | 61 ++++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/tcp.c b/tcp.c index 06acb41..89a5b19 100644 --- a/tcp.c +++ b/tcp.c @@ -408,16 +408,6 @@ static int tcp_sock_ns [NUM_PORTS][IP_VERSIONS]; */ static union inany_addr low_rtt_dst[LOW_RTT_TABLE_SIZE]; -/** - * tcp_buf_seq_update - Sequences to update with length of frames once sent - * @seq: Pointer to sequence number sent to tap-side, to be updated - * @len: TCP payload length - */ -struct tcp_buf_seq_update { - uint32_t *seq; - uint16_t len; -}; - /* Static buffers */ /** * struct tcp_payload_t - TCP header and data to send segments with payload @@ -459,7 +449,8 @@ static struct tcp_payload_t tcp4_payload[TCP_FRAMES_MEM]; static_assert(MSS4 <= sizeof(tcp4_payload[0].data), "MSS4 is greater than 65516"); -static struct tcp_buf_seq_update tcp4_seq_update[TCP_FRAMES_MEM]; +/* References tracking the owner connection of frames in the tap outqueue */ +static struct tcp_tap_conn *tcp4_frame_conns[TCP_FRAMES_MEM]; static unsigned int tcp4_payload_used; static struct tap_hdr tcp4_flags_tap_hdr[TCP_FRAMES_MEM]; @@ -481,7 +472,8 @@ static struct tcp_payload_t tcp6_payload[TCP_FRAMES_MEM]; static_assert(MSS6 <= sizeof(tcp6_payload[0].data), "MSS6 is greater than 65516"); -static struct tcp_buf_seq_update tcp6_seq_update[TCP_FRAMES_MEM]; +/* References tracking the owner connection of frames in the tap outqueue */ +static struct tcp_tap_conn *tcp6_frame_conns[TCP_FRAMES_MEM]; static unsigned int tcp6_payload_used; static struct tap_hdr tcp6_flags_tap_hdr[TCP_FRAMES_MEM]; @@ -1257,25 +1249,51 @@ static void tcp_flags_flush(const struct ctx *c) tcp4_flags_used = 0; } +/** + * tcp_revert_seq() - Revert affected conn->seq_to_tap after failed transmission + * @conns: Array of connection pointers corresponding to queued frames + * @frames: Two-dimensional array containing queued frames with sub-iovs + * @num_frames: Number of entries in the two arrays to be compared + */ +static void tcp_revert_seq(struct tcp_tap_conn **conns, struct iovec (*frames)[TCP_NUM_IOVS], + int num_frames) +{ + int i; + + for (i = 0; i < num_frames; i++) { + struct tcp_tap_conn *conn = conns[i]; + struct tcphdr *th = frames[i][TCP_IOV_PAYLOAD].iov_base; + uint32_t seq = ntohl(th->seq); + + if (SEQ_LE(conn->seq_to_tap, seq)) + continue; + + conn->seq_to_tap = seq; + } +} + /** * tcp_payload_flush() - Send out buffers for segments with data * @c: Execution context */ static void tcp_payload_flush(const struct ctx *c) { - unsigned i; size_t m; m = tap_send_frames(c, &tcp6_l2_iov[0][0], TCP_NUM_IOVS, tcp6_payload_used); - for (i = 0; i < m; i++) - *tcp6_seq_update[i].seq += tcp6_seq_update[i].len; + if (m != tcp6_payload_used) { + tcp_revert_seq(&tcp6_frame_conns[m], &tcp6_l2_iov[m], + tcp6_payload_used - m); + } tcp6_payload_used = 0; m = tap_send_frames(c, &tcp4_l2_iov[0][0], TCP_NUM_IOVS, tcp4_payload_used); - for (i = 0; i < m; i++) - *tcp4_seq_update[i].seq += tcp4_seq_update[i].len; + if (m != tcp4_payload_used) { + tcp_revert_seq(&tcp4_frame_conns[m], &tcp4_l2_iov[m], + tcp4_payload_used - m); + } tcp4_payload_used = 0; } @@ -2129,10 +2147,11 @@ static int tcp_sock_consume(const struct tcp_tap_conn *conn, uint32_t ack_seq) static void tcp_data_to_tap(const struct ctx *c, struct tcp_tap_conn *conn, ssize_t dlen, int no_csum, uint32_t seq) { - uint32_t *seq_update = &conn->seq_to_tap; struct iovec *iov; size_t l4len; + conn->seq_to_tap = seq + dlen; + if (CONN_V4(conn)) { struct iovec *iov_prev = tcp4_l2_iov[tcp4_payload_used - 1]; const uint16_t *check = NULL; @@ -2142,8 +2161,7 @@ static void tcp_data_to_tap(const struct ctx *c, struct tcp_tap_conn *conn, check = &iph->check; } - tcp4_seq_update[tcp4_payload_used].seq = seq_update; - tcp4_seq_update[tcp4_payload_used].len = dlen; + tcp4_frame_conns[tcp4_payload_used] = conn; iov = tcp4_l2_iov[tcp4_payload_used++]; l4len = tcp_l2_buf_fill_headers(c, conn, iov, dlen, check, seq); @@ -2151,8 +2169,7 @@ static void tcp_data_to_tap(const struct ctx *c, struct tcp_tap_conn *conn, if (tcp4_payload_used > TCP_FRAMES_MEM - 1) tcp_payload_flush(c); } else if (CONN_V6(conn)) { - tcp6_seq_update[tcp6_payload_used].seq = seq_update; - tcp6_seq_update[tcp6_payload_used].len = dlen; + tcp6_frame_conns[tcp6_payload_used] = conn; iov = tcp6_l2_iov[tcp6_payload_used++]; l4len = tcp_l2_buf_fill_headers(c, conn, iov, dlen, NULL, seq); -- 2.42.0

3 6

[PATCH 0/3] conf, log: Fix regression in usage() printing
by David Gibson 05 Jun '24

05 Jun '24

Erik Sjölund pointed out a regression where we're no long printing the --help message to stdout, but to stderr instead. I'm not actually sure what caused the regression, but it was made possible by the fact that the original fix was pretty strange to begin with. It introduced some hacks to force the logging subsystem to go to stdout, which have presumably failed in some subtle way. But, there's really no reason to run usage() through the logging subsystem in the first place - it just adds complexity. Link: https://bugs.passt.top/show_bug.cgi?id=90 Link: https://bugs.passt.top/show_bug.cgi?id=52 David Gibson (3): conf: Remove unhelpful usage() wrapper conf: Don't print usage via the logging subsystem log: Remove log_to_stdout option conf.c | 329 ++++++++++++++++++++++++++++----------------------------- log.c | 8 +- log.h | 1 - 3 files changed, 163 insertions(+), 175 deletions(-) -- 2.45.1

2 5

[PATCH v4 0/3] Support for SO_PEEK_OFF socket option
by Jon Maloy 04 Jun '24

04 Jun '24

Latest changes based on feedback. Notably, removed fast retransmit when peer advertises zero-window. Jon Maloy (3): tcp: move seq_to_tap update to when frame is queued tcp: leverage support of SO_PEEK_OFF socket option when available tcp: allow retransmit when peer receive window is zero tcp.c | 149 ++++++++++++++++++++++++++++++++++++++++------------- tcp_conn.h | 2 + 2 files changed, 116 insertions(+), 35 deletions(-) -- 2.42.0

3 17

[PATCH v4 00/10] Add vhost-user support to passt (part 2)
by Laurent Vivier 03 Jun '24

03 Jun '24

Extract buffers management code from tcp.c and move it to tcp_buf.c tcp.c keeps all the generic code and will be also used by the vhost-user functions. Also compare mode to MODE_PASTA, as we will manage vhost-user mode (MODE_VU) like MODE_PASST. v4: - remove "tcp: extract buffer management from tcp_conn_tap_mss()" as the MSS size can be the same between socket and vhost-user. - rename tcp_send_flag() and tcp_data_from_sock() to tcp_buf_send_flag() and tcp_buf_data_from_sock() v3: - add 3 new patches tap: use in->buf_size rather than sizeof(pkt_buf) tcp: remove tap_hdr parameter iov: remove iov_copy() v2: - compare to MODE_PASTA in conf_open_files() too - move taph out of udp_update_hdr4()/udp_update_hdr6() Laurent Vivier (10): tcp: inline tcp_l2_buf_fill_headers() tcp: extract buffer management from tcp_send_flag() tcp: move buffers management functions to their own file tap: export pool_flush()/tapX_handler()/packet_add() udp: move udpX_l2_buf_t and udpX_l2_mh_sock out of udp_update_hdrX() udp: rename udp_sock_handler() to udp_buf_sock_handler() vhost-user: compare mode MODE_PASTA and not MODE_PASST iov: remove iov_copy() tcp: remove tap_hdr parameter tap: use in->buf_size rather than sizeof(pkt_buf) Makefile | 5 +- conf.c | 14 +- iov.c | 39 ---- iov.h | 3 - isolation.c | 10 +- passt.c | 4 +- tap.c | 119 +++++----- tap.h | 7 + tcp.c | 604 +++---------------------------------------------- tcp_buf.c | 536 +++++++++++++++++++++++++++++++++++++++++++ tcp_buf.h | 16 ++ tcp_internal.h | 85 +++++++ udp.c | 68 +++--- udp.h | 2 +- 14 files changed, 792 insertions(+), 720 deletions(-) create mode 100644 tcp_buf.c create mode 100644 tcp_buf.h create mode 100644 tcp_internal.h -- 2.44.0

2 19

[PATCH v3 00/11] Add vhost-user support to passt (part 2)
by Laurent Vivier 31 May '24

31 May '24

Extract buffers management code from tcp.c and move it to tcp_buf.c tcp.c keeps all the generic code and will be also used by the vhost-user functions. Also compare mode to MODE_PASTA, as we will manage vhost-user mode (MODE_VU) like MODE_PASST. v3: - add 3 new patches tap: use in->buf_size rather than sizeof(pkt_buf) tcp: remove tap_hdr parameter iov: remove iov_copy() v2: - compare to MODE_PASTA in conf_open_files() too - move taph out of udp_update_hdr4()/udp_update_hdr6() Laurent Vivier (11): tcp: inline tcp_l2_buf_fill_headers() tcp: extract buffer management from tcp_send_flag() tcp: extract buffer management from tcp_conn_tap_mss() tcp: move buffers management functions to their own file tap: export pool_flush()/tapX_handler()/packet_add() udp: move udpX_l2_buf_t and udpX_l2_mh_sock out of udp_update_hdrX() udp: rename udp_sock_handler() to udp_buf_sock_handler() vhost-user: compare mode MODE_PASTA and not MODE_PASST iov: remove iov_copy() tcp: remove tap_hdr parameter tap: use in->buf_size rather than sizeof(pkt_buf) Makefile | 5 +- conf.c | 14 +- iov.c | 39 ---- iov.h | 3 - isolation.c | 10 +- passt.c | 4 +- tap.c | 119 +++++----- tap.h | 7 + tcp.c | 609 +++---------------------------------------------- tcp_buf.c | 553 ++++++++++++++++++++++++++++++++++++++++++++ tcp_buf.h | 17 ++ tcp_internal.h | 77 +++++++ udp.c | 68 +++--- udp.h | 2 +- 14 files changed, 798 insertions(+), 729 deletions(-) create mode 100644 tcp_buf.c create mode 100644 tcp_buf.h create mode 100644 tcp_internal.h -- 2.44.0

2 12

[PATCH v7 0/3] Support for SO_PEEK_OFF
by Jon Maloy 31 May '24

31 May '24

v7: Only patch #3 updated. Jon Maloy (3): tcp: move seq_to_tap update to when frame is queued tcp: leverage support of SO_PEEK_OFF socket option when available tcp: allow retransmit when peer receive window is zero tcp.c | 164 +++++++++++++++++++++++++++++++++++++++-------------- tcp_conn.h | 2 + 2 files changed, 123 insertions(+), 43 deletions(-) -- 2.45.0

2 5

[PATCH v2 0/8] Add vhost-user support to passt (part 2)
by Laurent Vivier 28 May '24

28 May '24

Extract buffers management code from tcp.c and move it to tcp_buf.c tcp.c keeps all the generic code and will be also used by the vhost-user functions. Also compare mode to MODE_PASTA, as we will manage vhost-user mode (MODE_VU) like MODE_PASST. v2: - compare to MODE_PASTA in conf_open_files() too - move taph out of udp_update_hdr4()/udp_update_hdr6() Laurent Vivier (8): tcp: inline tcp_l2_buf_fill_headers() tcp: extract buffer management from tcp_send_flag() tcp: extract buffer management from tcp_conn_tap_mss() tcp: move buffers management functions to their own file tap: export pool_flush()/tapX_handler()/packet_add() udp: move udpX_l2_buf_t and udpX_l2_mh_sock out of udp_update_hdrX() udp: rename udp_sock_handler() to udp_buf_sock_handler() vhost-user: compare mode MODE_PASTA and not MODE_PASST Makefile | 5 +- conf.c | 14 +- isolation.c | 10 +- passt.c | 4 +- tap.c | 109 +++++---- tap.h | 7 + tcp.c | 603 +++---------------------------------------------- tcp_buf.c | 543 ++++++++++++++++++++++++++++++++++++++++++++ tcp_buf.h | 17 ++ tcp_internal.h | 79 +++++++ udp.c | 68 +++--- udp.h | 2 +- 12 files changed, 786 insertions(+), 675 deletions(-) create mode 100644 tcp_buf.c create mode 100644 tcp_buf.h create mode 100644 tcp_internal.h -- 2.44.0

1 8

[PATCH 0/8] Add vhost-user support to passt (part 2)
by Laurent Vivier 27 May '24

27 May '24

Extract buffers management code from tcp.c and move it to tcp_buf.c tcp.c keeps all the generic code and will be also used by the vhost-user functions. Also compare mode to MODE_PASTA, as we will manage vhost-user mode (MODE_VU) like MODE_PASST. Laurent Vivier (8): tcp: inline tcp_l2_buf_fill_headers() tcp: extract buffer management from tcp_send_flag() tcp: extract buffer management from tcp_conn_tap_mss() tcp: move buffers management functions to their own file tap: export pool_flush()/tapX_handler()/packet_add() udp: move udpX_l2_buf_t and udpX_l2_mh_sock out of udp_update_hdrX() udp: rename udp_sock_handler() to udp_buf_sock_handler() vhost-user: compare mode MODE_PASTA and not MODE_PASST Makefile | 5 +- conf.c | 12 +- isolation.c | 10 +- passt.c | 4 +- tap.c | 109 +++++---- tap.h | 7 + tcp.c | 603 +++---------------------------------------------- tcp_buf.c | 543 ++++++++++++++++++++++++++++++++++++++++++++ tcp_buf.h | 17 ++ tcp_internal.h | 79 +++++++ udp.c | 72 +++--- udp.h | 2 +- 12 files changed, 787 insertions(+), 676 deletions(-) create mode 100644 tcp_buf.c create mode 100644 tcp_buf.h create mode 100644 tcp_internal.h -- 2.44.0

1 8

passt: new version 2024_05_23.765eb0b available
by Stefano Brivio 23 May '24

23 May '24

The new version with tag 2024_05_23.765eb0b includes the following changes: 765eb0b apparmor: Fix comments after PID file and AF_UNIX socket creation refactoring 0608ec4 conf, passt.h: Rename pid_file in struct ctx to pidfile c9b2413 conf, passt, tap: Open socket and PID files before switching UID/GID ba23b05 passt, util: Move opening of PID file to its own function 57d8aa8 util: Rename write_pidfile() to pidfile_write() cbca08c tap: Split tap_sock_unix_init() into opening and listening parts fcfb592 passt, tap: Don't use -1 as uninitialised value for fd_tap_listen d02bb6c tap: Move all-ones initialisation of mac_guest to tap_sock_init() 45b8632 conf: Don't lecture user about starting us as root 3f917b3 netlink, test: Ignore deprecated addresses cc801fb tcp: Remove interim 'tapside' field from connection 8a2accb flow: Record the pifs for each side of each flow 4357185 flow: Make side 0 always be the initiating side 0060acd flow: Clarify and enforce flow state transitions a631998 inany: Better helpers for using inany and specific family addrs together 7a832a8 flow: Properly type callbacks to protocol specific handlers 1a20370 util, tcp: Add helper to display socket addresses 3ff3a8a apparmor: Fix passt abstraction 6cdc9fd apparmor: allow netns paths on /tmp 80f7ff2 clang-tidy: Suppress macro to enum conversion warnings 29bd08f conf: Fix clang-tidy warning about using an undefined enum value 26c71db passt.c: explicitly include libgen.h for basename 623c2fd netlink: Don't duplicate routes referring to unrelated host interfaces https://passt.top/passt/log/?qt=range&q=2024_05_10.7288448..2024_05_23.765e… Packages: - Alpine Linux: https://pkgs.alpinelinux.org/packages?name=passt - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Chimera: https://pkgs.chimera-linux.org/packages?name=passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/7482357/ permanent mirror: https://passt.top/builds/copr/0^20240523.g765eb0b/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Gentoo versions: https://packages.gentoo.org/packages/net-misc/passt - GNU Guix: https://packages.guix.gnu.org/packages/passt/ - openSUSE: https://software.opensuse.org/package/passt - PLD Linux: https://git.pld-linux.org/cgi-bin/gitweb.cgi?p=packages/passt.git - Solus: https://github.com/getsolus/packages/tree/main/packages/p/passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-g765eb0b-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_765eb0b-1_all.deb -- Stefano

1 0