feat: Test isolation prototype — namespaces and NixOS VMs#1324
Open
feat: Test isolation prototype — namespaces and NixOS VMs#1324
Conversation
4 tasks
scripts/isolated-test.sh wraps environment tests in Linux namespaces (network + PID) to prevent port conflicts and orphaned services between concurrent test runs. scripts/ISOLATION.md documents the approaches considered (namespaces, systemd-nspawn, Firecracker, QEMU) and how to integrate with the existing CI workflow. This is a prototype — the Nix-based VM approach using NixOS test infrastructure may be a better fit given the builders are already Nix-capable machines.
Three approaches documented for Nix-native test isolation: 1. unshare in mkFloxEnvPkg: add util-linux/iproute2 to the Nix derivation, wrap flox activate with unshare --net --pid. Zero CI changes. Nix caches the derivation. 2. NixOS VM tests: full QEMU VMs via nixos/tests framework. Strongest isolation, Nix caches VM images. ~10-30s boot. 3. Hybrid (recommended): unshare for non-service envs, NixOS VMs for service envs (postgres, mysql, etc.), current approach for macOS. Includes concrete flake.nix patch prototype showing the minimal change for approach 1.
Renamed from "Nix-native proposal" to "Test Isolation Solutions" covering all three approaches equally: 1. Linux namespaces (standalone wrapper) — simplest, ~0ms overhead, plain bash, no Nix dependency 2. Nix+unshare (flake-integrated) — same isolation, Nix provides deps and caches the derivation 3. NixOS VM tests — full kernel isolation via QEMU, Nix caches VM images, strongest but ~10-30s boot Includes comparison table and phased recommendation.
Confirmed macOS has no equivalent to unshare --net. Darwin kernel does not support network namespaces. sandbox-exec can deny network but cannot isolate port spaces. Documented three macOS mitigation options: 1. Accept no isolation (current behavior, conflicts rare) 2. Randomize ports (fragile workaround) 3. Tart VMs on Mac minis (phase 3, full isolation) Updated recommendation: Nix+unshare for Linux (phase 1), Tart for macOS (phase 3 when needed).
32d297b to
6257386
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prototype and research for isolated test execution (ST-006 in the
environment health testing effort). Addresses the core problem:
concurrent test runs on shared builders fight over ports (MySQL,
Elasticsearch, Postgres) and leave orphaned services.
Three solutions documented and compared:
Recommended: Nix+unshare as default — bake namespace
isolation into
mkFloxEnvPkginflake.nix. Zero CI changes,Nix provides all deps and caches the derivation. Needs further
investigation for Darwin equivalent to achieve cross-platform
isolation.
What's included
scripts/isolated-test.sh— Standalone namespace wrapper(Solution 1). Usage:
./scripts/isolated-test.sh postgres --start-servicesscripts/flake-unshare-patch.nix— Concrete flake.nix patchshowing the Nix+unshare approach (Solution 2)
scripts/nix-isolation-proposal.md— Unified comparison ofall three solutions with pros/cons and phased rollout plan
scripts/ISOLATION.md— Original research notesKey findings
MySQL 13306, Elasticsearch 19200)
unshare --net --pid --forkgives each test its own loopbackand port space — conflicts impossible
Nix derivation, zero CI changes, ~0ms overhead
if namespaces prove insufficient
macOS, but sandbox-exec or Virtualization.framework may work)
Open questions
(
sandbox-exec, Seatbelt, Virtualization.framework?)(
sysctl kernel.unprivileged_userns_clone)service environments?
Related
efforts/2026/02-environment-health-testing/(ST-006, TH-004)Test plan
./scripts/isolated-test.sh postgres --start-servicesruns without port conflicts on Linux
Via Forge (interactive) • 97e38ba