On Tue, 18 Mar 2025 16:21:58 +1100
David Gibson
Continued investigating the problem with migration failing across a bridge.
Good news is I've found the problem... or at least one problem.
\o/
Bad news is we'll have to change the migration stream format to fix it.
Whoops, sorry, my bad. And now, RFC 7323, section 3.2, contrary to RFC
1323 (also section 3.2), requires that we keep sending timestamps if we
negotiated them:
Once TSopt has been successfully negotiated, that is both <SYN> and
The packets are being dropped in tcp_validate_incoming() due to a failed PAWS check (skb drop reason "TCP_RFC7323_PAWS"). That in turn looks to be because we don't preserve TCP timestamp state across the migration. We preserve _whether_ TCP timestamps are active on the connection (TCPOPT_TIMESTAMP entry in TCP_REPAIR_OPTIONS), but we don't preserve the current timestamp values (TCP_TIMESTAMP socket option). The equivalent CRIU code is
https://github.com/checkpoint-restore/criu/blob/d18912fc88f3dc7bde5fdfa35756...
and
https://github.com/checkpoint-restore/criu/blob/d18912fc88f3dc7bde5fdfa35756...
I'll work on writing a fix tomorrow.
Not yet sure why we didn't hit this with a local migration. I'm guessing some part of being a local connection means we're bypassing the PAWS check.
The TCP_TIMESTAMP option is documented... not where it should be documented, grr: https://criu.org/index.php?title=TCP_connection#Timestamp and I _guess_ that two guests using kvm-clock as clock source might actually have the same jiffies, and from this description, same jiffies, same timestamps. Perhaps in your nested case not all guests are using kvm-clock, or there's something else to it. -- Stefano