On Thu, 22 Jun 2023 22:12:27 -0400
Jon Maloy <jmaloy(a)redhat.com> wrote:
When reading received messages with MSG_PEEK, we
sometines have to read
the leading bytes of the stream several times, only to reach the bytes
we really want. This is clearly non-optimal.
What we would want is something similar to pread/preadv(), but working
even for tcp sockets. At the same time, we obviously don't want to add
any new arguments to the recv/recvmsg() calls.
In this commit, we allow the user to set iovec.iov_base in the first
vector entry to NULL. This tells the socket to skip the first entry,
hence making the iov_len field of that entry indicate the offset value.
This way, there is no need to add any new arguments.
Ah-ha! I'm glad you
found an acceptable way to pass a NULL pointer
there. :)
This change is simple and non-intrusive, and
should be safe addition to
the socket API. We have measured it to give a throughput improvement of
...it
would be nice to also do a bit of profiling with perf(1) --
that's where I originally noticed we were wasting cycles on filling up
tcp_buf_discard. Plus, sure, there's also some value in dropping a
useless 16 MiB buffer.
If you need examples/inspiration: the pasta (automated) demo shows
that, skip at 9:20 in:
https://passt.top/passt/about/#pasta_2 (the one
on the left)
and that's simply done like this:
https://passt.top/passt/tree/test/demo/pasta#n163 passt
containers.
Those are virtual machines in
containers -- but other than KubeVirt,
passt is generally supported in libvirt (>= 9.2) with QEMU (>= 7.2)
Signed-off-by: Jon Maloy
<jmaloy(a)redhat.com>
works with original msghdr
Signed-off-by: Jon Maloy <jmaloy(a)redhat.com>
---
net/ipv4/tcp.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 33f559f491c8..1d89337e89b6 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2428,6 +2428,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg,
size_t len,
struct tcp_sock *tp = tcp_sk(sk);
int copied = 0;
u32 peek_seq;
+ u32 peek_offset;
Kernel networking code observes the reverse-Christmas tree
notation (at
least for new additions), this would need to go after *tp:
struct tcp_sock *tp = tcp_sk(sk);
u32 peek_offset;
u32 *seq;
unsigned long used;
int err;
@@ -2435,7 +2436,6 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg,
size_t len,
long timeo;
struct sk_buff *skb, *last;
u32 urg_hole = 0;
-
Probably not intended.
err = -ENOTCONN;
if (sk->sk_state == TCP_LISTEN)
goto out;
@@ -2469,6 +2469,14 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg,
size_t len,
if (flags & MSG_PEEK) {
peek_seq = tp->copied_seq;
seq = &peek_seq;
+ if (msg->msg_iter.iov[0].iov_base == NULL) {
+ peek_offset = msg->msg_iter.iov[0].iov_len;
+ msg->msg_iter.iov = &msg->msg_iter.iov[1];
+ msg->msg_iter.nr_segs -= 1;
Do you also need to make sure that nr_segs >
1 if iov[0].iov_base is
NULL? I'm not sure if we need to check explicitly that msg_iter.iov[1]
is valid here (but I haven't followed the whole path from the syscall
handler).
> + msg->msg_iter.count -= peek_offset;
> + len -= peek_offset;
> + *seq += peek_offset;
> + }
> }
>
> target = sock_rcvlowat(sk, flags & MSG_WAITALL, len);