feat: run OCI containers as lab devices - #45
Open
Frando wants to merge 3 commits into
Open
Conversation
Integration tests using patchbay often need auxiliary services that are easier to run as a container than to embed: an ACME test server (Pebble), a database, a real DNS server. Add a way to run these as lab nodes. A container is a device. `Lab::add_container(name, image)` returns a builder that mirrors the device builder for wiring (uplink, iface, default_via, mtu) and adds container config (env, args, run_arg, ready_tcp, runtime). It creates the device namespace and veth uplink like any device, then runs the image joined to that namespace, so the service is reachable from other lab devices at the container's lab IP. The returned Container derefs to Device and adds exec/logs/stop; it is removed when the handle drops. Containers run with podman, which must be on PATH. Podman is daemonless, so the `podman run` process forked from inside the device's namespace worker joins that namespace via --network=host. Docker cannot do this rootlessly: its daemon runs in a separate namespace. Making rootless podman work inside patchbay's user namespace needed three things: - userns: map the full subuid/subgid range via newuidmap/newgidmap when the host provides them (/etc/subuid, /etc/subgid entries), falling back to the previous single-uid map otherwise. A single uid cannot chown image files or mount overlay; the range can. This is the standard rootless-podman prerequisite and is generally useful for any nested container runtime. - Tell podman it is the already-configured rootless namespace (_CONTAINERS_USERNS_CONFIGURED, _CONTAINERS_ROOTLESS_UID/GID), since patchbay maps the user to uid 0 and podman would otherwise use unwritable rootful paths. This also makes it reuse the user's rootless image store. - Run the image pull and the non-run podman commands on a thread with its own mount namespace, because podman makes its storage mounts private and the plain blocking-pool threads share the host mount namespace. The pull runs on the host (the device namespace has no route to a registry); the run is forked from the device worker, which already has a private mount namespace. Tested: unit tests for run-arg construction and name sanitizing, and an integration test that runs nginx:alpine and fetches it from a client device (gated on podman, skips where absent). The full lib suite passes with the range-map userns, so existing behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integration services usually need their config or data handed in from the host: a Pebble config file, a seeded database directory. Add builder methods for it, following the testcontainers shape (bind mounts with an access mode, plus tmpfs): - `volume(host, container)` / `volume_ro(host, container)`: bind-mount a host path read-write or read-only. - `tmpfs(container)`: mount a fresh tmpfs, for a writable scratch dir on an otherwise read-only image. These render to `podman run --volume host:container[:ro]` / `--tmpfs` before the image. `run_arg` remains the escape hatch for exotic cases such as a `:z`/`:Z` SELinux label. Also documents why the `_CONTAINERS_*` rootless environment variables are used: podman keys "am I rootless" off euid with no containers.conf override, `--userns=ns:path` is broken for rootless, and a supported flag is an open request (podman-container-tools/podman#7774). They are internal but stable. Tested: unit test for mount rendering, and an integration test that bind-mounts a host file read-only and reads it back from inside the container (gated on podman). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add methods to move files between the host and a running container,
following testcontainers' shape:
- copy_to(host, container) / copy_from(container, host): copy a file or
directory in or out (podman cp).
- write_file(path, bytes) / read_file(path): write or read a single file
without a bind mount, staged through a host temp file so binary content
and any image (no shell needed) work.
Also records, on runtime_command, why podman must run as uid 0 with the
_CONTAINERS_* variables rather than as a non-zero uid: a non-zero uid
makes podman detect rootless and try to create its own nested user
namespace, which fails inside patchbay's ("cannot clone"), so
_CONTAINERS_USERNS_CONFIGURED is required regardless, and a non-root uid
would break patchbay's own uid-0 assumptions. Verified by reproducing
both cases under `unshare`.
Tested: an integration test round-trips write_file/read_file and
copy_to/copy_from against an alpine container (gated on podman).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Integration tests written against patchbay often need an auxiliary service running alongside the code under test: an ACME test server like Pebble, a database, a real DNS server. These are usually easier to run as a container than to embed. This lets patchbay run one as an ordinary lab node.
A container is a device.
Lab::add_container(name, image)returns a builder that mirrors the device builder for wiring (uplink,iface,default_via,mtu) and adds container configuration on top. It creates the device namespace and veth uplink like any device, then runs the image joined to that namespace, so the service is reachable from other lab devices at the container's lab IP.The returned
Containerdereferences toDevice, so the device accessors work directly, and it adds container methods:exec,logs,stop,write_file/read_file, andcopy_to/copy_from. Volumes come in read-write, read-only, and tmpfs forms. The container is removed when the handle is dropped.Runtime and requirements
Containers run with podman, which must be on
PATH. Podman is daemonless, so thepodman runprocess, forked from inside the device's namespace worker, joins that namespace with--network=hostand lands the container on the device's lab IP. Docker cannot do this rootlessly: its daemon runs in a separate namespace, so a docker container would join the daemon's namespace rather than the device's.Running a container inside patchbay's rootless user namespace needs a full subuid/subgid range, the standard rootless-podman setup:
newuidmap/newgidmaponPATHand an/etc/subuidand/etc/subgidentry for the user. The user-namespace bootstrap now maps that range vianewuidmap/newgidmapwhen the host provides it, and falls back to the previous single-uid map otherwise. A single uid cannot unpack image layers or mount overlay; the range can. Router and device networking work either way, so this changes nothing for labs that do not use containers.Why it took some plumbing
patchbay maps the invoking user to uid 0, so podman would otherwise treat itself as rootful and use unwritable system paths. Two internal but stable environment variables (
_CONTAINERS_USERNS_CONFIGURED,_CONTAINERS_ROOTLESS_UID/GID) tell it it is the already-configured rootless namespace, so it uses the user's rootless storage and reuses already-pulled images. The image is pulled on the host before the container starts, because the device namespace has no route to a registry, and the pull runs on a thread with its own mount namespace, since podman makes its storage mounts private and the plain worker threads share the host mount namespace.Notes and dead ends
Two alternatives to the environment-variable approach turned out not to be worth it, and the reasons are recorded next to the code:
--uidmap 0:0:65536does work with--network=host, but it is redundant, since patchbay's namespace already maps the full range the container needs.Making "run in an already-configured user namespace as uid 0" a first-class podman flag is an open upstream request (podman-container-tools/podman#7774).
The API borrows its shape from testcontainers-rs (bind mounts with an access mode, tmpfs, copy in and out) but stays smaller on purpose:
ready_tcpcovers the common readiness case rather than a full set of wait strategies, which can be added when a service needs them.Testing
Unit tests cover the run-argument and mount rendering and the container-name sanitizing. Integration tests, gated on podman so they skip where it is absent, run nginx and fetch it from a separate client device, check that a read-only bind mount is visible, and round-trip
write_file/read_fileandcopy_to/copy_from. The full library suite passes with the range-map user namespace, so existing behavior is unchanged.🤖 Generated with Claude Code