Hi, as reported by Stefano there is an asymmetry in the throughput between host and guest with vhost-user. I've tested the following kernel patch from Jason to see if it can improve the performance: ------------------------------------------------------------------------------ commit e13b6da7045f997e1a5a5efd61d40e63c4fc20e8 Author: Jason Wang Date: Tue Feb 18 10:39:08 2025 +0800 virtio-net: tweak for better TX performance in NAPI mode There are several issues existed in start_xmit(): - Transmitted packets need to be freed before sending a packet, this introduces delay and increases the average packets transmit time. This also increase the time that spent in holding the TX lock. - Notification is enabled after free_old_xmit_skbs() which will introduce unnecessary interrupts if TX notification happens on the same CPU that is doing the transmission now (actually, virtio-net driver are optimized for this case). So this patch tries to avoid those issues by not cleaning transmitted packets in start_xmit() when TX NAPI is enabled and disable notifications even more aggressively. Notification will be since the beginning of the start_xmit(). But we can't enable delayed notification after TX is stopped as we will lose the notifications. Instead, the delayed notification needs is enabled after the virtqueue is kicked for best performance. Performance numbers: 1) single queue 2 vcpus guest with pktgen_sample03_burst_single_flow.sh (burst 256) + testpmd (rxonly) on the host: - When pinning TX IRQ to pktgen VCPU: split virtqueue PPS were increased 55% from 6.89 Mpps to 10.7 Mpps and 32% TX interrupts were eliminated. Packed virtqueue PPS were increased 50% from 7.09 Mpps to 10.7 Mpps, 99% TX interrupts were eliminated. - When pinning TX IRQ to VCPU other than pktgen: split virtqueue PPS were increased 96% from 5.29 Mpps to 10.4 Mpps and 45% TX interrupts were eliminated; Packed virtqueue PPS were increased 78% from 6.12 Mpps to 10.9 Mpps and 99% TX interrupts were eliminated. 2) single queue 1 vcpu guest + vhost-net/TAP on the host: single session netperf from guest to host shows 82% improvement from 31Gb/s to 58Gb/s, %stddev were reduced from 34.5% to 1.9% and 88% of TX interrupts were eliminated. Signed-off-by: Jason Wang Acked-by: Michael S. Tsirkin Signed-off-by: David S. Miller ------------------------------------------------------------------------------ systemctl stop firewalld.service || service iptables stop || iptables -Ft /sbin/sysctl -w net.core.rmem_max=536870912 /sbin/sysctl -w net.core.wmem_max=536870912 ____ I made my test using 6.14-rc7 kernel: From guest: iperf3 -c 10.6.68.254 -P2 -Z -t5 -l 1M -w 16M [SUM] 0.00-5.00 sec 14.5 GBytes 24.9 Gbits/sec 0 sender [SUM] 0.00-5.00 sec 14.5 GBytes 24.9 Gbits/sec receiver From host: iperf3 -c localhost -P2 -Z -t5 -p 10001 -l 1M -w 16M [SUM] 0.00-5.00 sec 28.9 GBytes 49.6 Gbits/sec 0 sender [SUM] 0.00-5.03 sec 28.8 GBytes 49.2 Gbits/sec receiver ____ The results with a 6.14-rc7 + e13b6da7045f: From guest: iperf3 -c 10.6.68.254 -P2 -Z -t5 -l 1M -w 16M [SUM] 0.00-5.00 sec 14.8 GBytes 25.4 Gbits/sec 0 sender [SUM] 0.00-5.01 sec 14.8 GBytes 25.4 Gbits/sec receiver From host: iperf3 -c localhost -P2 -Z -t5 -p 10001 -l 1M -w 16M [SUM] 0.00-5.00 sec 28.5 GBytes 48.9 Gbits/sec 0 sender [SUM] 0.00-5.03 sec 28.4 GBytes 48.6 Gbits/sec receiver We have only a 2% improvement. Thanks, Laurent