Hi, I've been using pasta as a network driver for rootless docker and I've been running into a couple of issues for a while now. I hope this is where I can find some help troubleshooting. The issue is that when I use pasta as the network driver as opposed to slirp4netns, I'm unable to access the internet through rootless docker or use ping (or traceroute) through its containers. So if I run "docker pull <image-name>" I get a timeout error:
Using default tag: latest Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 10.0.2.3:53: read udp 169.254.2.1:58905 ->10.0.2.3:53: i/o timeout
I'm running pasta version 0.0~git20250217.a1e48a0-1 on Ubuntu 24.04.2 LTS with docker v27.3.1 build ce12230. Please let me know what other details I could provide to troubleshoot this issue. Thanks and regards, Ayon Tarafdar
Hi Ayon,
On Sat, 10 May 2025 21:26:29 -0230
Ayon T
Hi,
I've been using pasta as a network driver for rootless docker and I've been running into a couple of issues for a while now. I hope this is where I can find some help troubleshooting.
The issue is that when I use pasta as the network driver as opposed to slirp4netns, I'm unable to access the internet through rootless docker or use ping (or traceroute) through its containers. So if I run "docker pull <image-name>" I get a timeout error:
Using default tag: latest Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 10.0.2.3:53: read udp 169.254.2.1:58905 ->10.0.2.3:53: i/o timeout
I'm running pasta version 0.0~git20250217.a1e48a0-1 on Ubuntu 24.04.2 LTS with docker v27.3.1 build ce12230.
I suspect you might be hitting this: https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2077158 ...which is fixed on Ubuntu 24.10 and later versions. As a workaround, I guess you can create the AppArmor profile for pasta manually, from: https://passt.top/passt/tree/contrib/apparmor/usr.bin.pasta or set /proc/sys/kernel/unprivileged_userns_apparmor_policy to 0, see also: https://github.com/kubevirt/kubevirt/issues/12333 Let me know if you still hit the issue. -- Stefano
Hi,
Thanks for responding back to me. I was trying these solutions, when I
realised that my /etc/apparmor.d/ directory already contains usr.bin.pasta.
I believe this is because I downloaded the latest package from the
launchpad. Using aa-status, it seems like ' /usr/bin/pasta.avx2 (1366)
pasta' is running in enforce mode. So I suspect there is no point trying to
do this over.
On Mon, May 12, 2025 at 11:36 AM Stefano Brivio
Hi Ayon,
On Sat, 10 May 2025 21:26:29 -0230 Ayon T
wrote: Hi,
I've been using pasta as a network driver for rootless docker and I've been running into a couple of issues for a while now. I hope this is where I can find some help troubleshooting.
The issue is that when I use pasta as the network driver as opposed to slirp4netns, I'm unable to access the internet through rootless docker or use ping (or traceroute) through its containers. So if I run "docker pull <image-name>" I get a timeout error:
Using default tag: latest Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 10.0.2.3:53: read udp 169.254.2.1:58905 ->10.0.2.3:53: i/o timeout
I'm running pasta version 0.0~git20250217.a1e48a0-1 on Ubuntu 24.04.2 LTS with docker v27.3.1 build ce12230.
I suspect you might be hitting this:
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2077158
...which is fixed on Ubuntu 24.10 and later versions. As a workaround, I guess you can create the AppArmor profile for pasta manually, from:
https://passt.top/passt/tree/contrib/apparmor/usr.bin.pasta
or set /proc/sys/kernel/unprivileged_userns_apparmor_policy to 0, see also:
https://github.com/kubevirt/kubevirt/issues/12333
Let me know if you still hit the issue.
-- Stefano
On Mon, 12 May 2025 12:18:17 -0230
Ayon T
Hi,
Thanks for responding back to me. I was trying these solutions, when I realised that my /etc/apparmor.d/ directory already contains usr.bin.pasta. I believe this is because I downloaded the latest package from the launchpad.
...wait, so it's not 0.0~git20250217.a1e48a0-1 anymore? What version of passt are you using now? And what version of rootlesskit are you running? Does it contain this fix: https://github.com/rootless-containers/rootlesskit/pull/458 it's not directly related to your issue, but it changes the behaviour significantly. I don't remember how you would pass pasta options through moby / rootlesskit, but ideally you should try running pasta with --debug and --log-file. Another important bit of information would be if container connectivity works with an existing container (on 'docker run', not 'docker pull'), and, if it doesn't, whether pasta is running (check with 'ps ax') while the container is running. -- Stefano
Hi,
I'm sorry for the delayed response. No, I am still using
0.0~git20250217.a1e48a0-1. I just meant that I had not used apt for
installing passt.
I had an upgrade planned to Ubuntu 24.10 and after that upgrade, my
connections to port 53 are being refused when I'm using pasta, leaving me
unable to use a DNS resolver. I'm not sure why this is. It works fine with
the default network driver.
This is not the issue I contacted you with and that's why I'm trying to
figure out why this is happening myself (albeit with little luck).
Regards,
Ayon
On Mon, May 12, 2025 at 12:29 PM Stefano Brivio
On Mon, 12 May 2025 12:18:17 -0230 Ayon T
wrote: Hi,
Thanks for responding back to me. I was trying these solutions, when I realised that my /etc/apparmor.d/ directory already contains usr.bin.pasta. I believe this is because I downloaded the latest package from the launchpad.
...wait, so it's not 0.0~git20250217.a1e48a0-1 anymore? What version of passt are you using now?
And what version of rootlesskit are you running? Does it contain this fix:
https://github.com/rootless-containers/rootlesskit/pull/458
it's not directly related to your issue, but it changes the behaviour significantly.
I don't remember how you would pass pasta options through moby / rootlesskit, but ideally you should try running pasta with --debug and --log-file. Another important bit of information would be if container connectivity works with an existing container (on 'docker run', not 'docker pull'), and, if it doesn't, whether pasta is running (check with 'ps ax') while the container is running.
-- Stefano
Hi Ayon,
On Tue, 20 May 2025 12:09:46 -0230
Ayon T
Hi,
I'm sorry for the delayed response. No, I am still using 0.0~git20250217.a1e48a0-1. I just meant that I had not used apt for installing passt.
...where did you take that package from, though? If it's something like alvistack (https://github.com/alvistack/passt-top-passt/), there might be other patches or modified / missing AppArmor profiles, and I can't really look into that as well (I already maintain official packages, at least for Debian).
I had an upgrade planned to Ubuntu 24.10 and after that upgrade, my connections to port 53 are being refused when I'm using pasta, leaving me unable to use a DNS resolver. I'm not sure why this is. It works fine with the default network driver.
This is not the issue I contacted you with and that's why I'm trying to figure out why this is happening myself (albeit with little luck).
I would suggest to start passing the --pcap option to pasta, say: podman run --net=pasta:--pcap,/tmp/port53.pcap ... and then have a look at the resulting packet capture (with Wireshark / tshark, for example). As we keep fixing bugs, and we fixed quite a lot of things with UDP flows since February, an updated package, or even a build from source, (git clone git://passt.top && cd passt && make && sudo make install) might be worth a try. It takes a few seconds to build / install, and can be removed cleanly with 'sudo make uninstall'. -- Stefano
Hi, I have managed to fix the new connection refusal problem I was struggling with. Apparently my unprivileged port start setting had changed after the upgrade, even though it was set correctly in /etc/sysctl.conf. I fixed it by adding an entry in /etc/sysctl.d/. And what version of rootlesskit are you running? Does it contain this
fix:
https://github.com/rootless-containers/rootlesskit/pull/458
it's not directly related to your issue, but it changes the behaviour significantly.
To answer your questions, I'm using v2.3.1, which, I assume, includes this fix.
Another important bit of information would be if container connectivity works with an existing container (on 'docker run', not 'docker pull'), and, if it doesn't, whether pasta is running (check with 'ps ax') while the container is running.
I used docker exec to get into the containers that are already running and
I cannot ping/traceroute from the containers, but I can, for instance, use
wget google.com and download the webpage. pasta is running, checked using
ps ax.
...where did you take that package from, though?
I installed it using the deb package in
https://launchpad.net/ubuntu/+source/passt.
I know you have been asking me to run pasta with arguments with docker, but
I'm not sure how to do this (pardon my inexperience). I use an
override.conf file to set the default network and port driver of docker,
and that's how I use pasta with docker. I have tried looking up how to do
it in a different way that gives me more control over the arguments that go
in, but I haven't been able to find it. Could you guide me regarding this
or point me to a resource?
Thanks and regards,
Ayon
On Tue, May 20, 2025 at 1:26 PM Stefano Brivio
Hi Ayon,
On Tue, 20 May 2025 12:09:46 -0230 Ayon T
wrote: Hi,
I'm sorry for the delayed response. No, I am still using 0.0~git20250217.a1e48a0-1. I just meant that I had not used apt for installing passt.
...where did you take that package from, though? If it's something like alvistack (https://github.com/alvistack/passt-top-passt/), there might be other patches or modified / missing AppArmor profiles, and I can't really look into that as well (I already maintain official packages, at least for Debian).
I had an upgrade planned to Ubuntu 24.10 and after that upgrade, my connections to port 53 are being refused when I'm using pasta, leaving me unable to use a DNS resolver. I'm not sure why this is. It works fine with the default network driver.
This is not the issue I contacted you with and that's why I'm trying to figure out why this is happening myself (albeit with little luck).
I would suggest to start passing the --pcap option to pasta, say:
podman run --net=pasta:--pcap,/tmp/port53.pcap ...
and then have a look at the resulting packet capture (with Wireshark / tshark, for example).
As we keep fixing bugs, and we fixed quite a lot of things with UDP flows since February, an updated package, or even a build from source, (git clone git://passt.top && cd passt && make && sudo make install) might be worth a try.
It takes a few seconds to build / install, and can be removed cleanly with 'sudo make uninstall'.
-- Stefano
participants (2)
-
Ayon T
-
Stefano Brivio