IPv6

Every jail gets a real IPv6 address

No NAT, no router advertisements, no SLAAC, no helper daemon to install. JailOps writes the address into the jail's spec, configures it, answers for it on the network when it has to, and puts a firewall in front of it. It works the same on a host that has a whole prefix routed to it and on a cloud server that only shares a /64 with its neighbours, and the second case is the one that usually needs extra machinery.

Works with how your provider hands out IPv6

Two models, one tool.

Providers hand out IPv6 in one of two ways. Hetzner is an example of the first: the /64 is routed to your server, so you use routed mode and no proxy is needed. Netcup is an example of the second: the /64 sits on a shared segment, so you use proxy mode and JailOps answers neighbour discovery for each jail address automatically. Both work on the same host, with the same CLI and the same API; the difference is one flag, --ipv6-mode proxy.

The two ways providers hand out IPv6
Hetzner-styleNetcup-style
Who routes the /64The provider routes it to your serverIt sits on a shared segment; the provider's router asks who has each address
JailOps modeRouted, no proxySwitched, with the built-in proxy
What you configure--ipv6 PREFIX::1/80 (a /80 or /96 per network)--ipv6-mode proxy, plus one approval line in priv.conf; a /64 or a smaller carved prefix such as a /80
Inbound pf.conf rulepass in quick on vtnet0 inet6 proto tcp from any to JAIL-PREFIX port 80 flags S/SA keep stateThe same rule

What has been verified: routed mode on a routed test host, and switched (proxy) mode on a test host with a switched /64. Those tests did not run at the named providers.

The model

Three ideas, the same in both modes.

  • Addresses are allocated, not discovered. A network has an IPv6 subnet and an explicit allocation range. JailOps picks the next free address, stores it in the jail's spec and configures it statically. The address a jail has is the address the API shows, the firewall renders and the backups record.
  • One network is one /80 (or any prefix you choose). A /64 holds 65,536 of them, each with 248 addresses. Because nothing uses SLAAC, a prefix longer than /64 is no problem.
  • Dual stack by default. A network can carry an IPv4 subnet behind NAT and an IPv6 subnet without it. A jail attached to that network gets an address of each family, the default route, and resolv.conf from the network.
  • Every jail has its own PF anchor, for IPv6 as for IPv4, and a project's jails cannot talk to another project's unless a rule you wrote opens it. See networking.

Routed mode

The provider routes a prefix to your host.

Implemented, fakesVerified on FreeBSD 15.1 (arm64)

Most dedicated-server and many cloud providers send a whole /64 to your machine's address. The upstream router knows that every address in it lives behind the host, so the host only has to forward. JailOps carves a /80 out of it for a network, puts the first address on the bridge as the gateway, and nothing else is needed: no proxy, no neighbor discovery tricks. This mode never depends on the proxy.

a dual-stack network in a routed prefix
# 2001:db8:1:2::/64 is routed to this host
$ jailops network create web \
    --ipv4 10.88.0.1/24 --nat \
    --ipv6 2001:db8:1:2:1::1/80 \
    --uplink vtnet0
network/web created

$ jailops project create shop
$ jailops project set shop --network web --storage default
$ jailops jail create site --project shop --image freebsd-15@stable \
    --network web --storage default
$ jailops jail start site --project shop
$ jailops jail list --project shop
NAME   STATE     IMAGE            GEN   ADDRESSES                          AGE
site   running   freebsd-15@stable   g1    10.88.0.2,2001:db8:1:2:1::2        5s
  • The host has to forward IPv6 (net.inet6.ip6.forwarding=1). JailOps sets it while it runs and jailops doctor tells you to make it permanent with ipv6_gateway_enable="YES" in /etc/rc.conf; it never edits that file.
  • Give the next project the next /80: 2001:db8:1:2:2::1/80, and so on. Networks are never shared between projects.
  • Outbound IPv6 from a jail, the host reaching the jail, and Alpine and Debian jails with the same addresses were run on FreeBSD 15.1. Traffic into a jail from the internet over a routed prefix has not been tested end to end yet; see what is verified.

Switched mode

The provider gives you addresses on a shared segment.

Implemented, fakesVerified on FreeBSD 15.1 (amd64)

On many cloud servers there is no routed prefix. You get a /64 on the same Ethernet segment as other customers, the provider's router believes every address in it is on-link, and it finds a machine by asking neighbor discovery who has the address. A jail that lives behind your host never hears that question, so it is unreachable. The usual answers are to give the jails the host's MAC address, to bridge them onto the provider's network (which the provider often forbids), or to give up on IPv6.

JailOps has a built-in neighbor discovery proxy instead. It is written in Go, part of the daemon, and knows nothing generic: it only knows the addresses JailOps has allocated.

  • It answers the router's neighbor solicitation for every address of a running jail with the host's own MAC address, at once, and announces each address when it appears and again whenever the capture is reopened, so the first packets are not left waiting for the router's timers.
  • It installs a /128 host route through the network's gateway interface for each address and removes it, and the host's neighbor cache entry, when the jail stops. A new jail that gets an old address is reachable at once.
  • It never answers for the host's own addresses (the kernel does that), never answers duplicate address detection, rate-limits itself, and ignores everything outside the addresses it allocated.
  • It runs for every network in proxy mode. There is no experimental switch to forget. jailops doctor fails, with the line to add, when a proxy network is not answering.
a /80 inside a shared, switched /64
# 2001:db8:1:2::/64 is on the host's uplink and shared with others.
# The operator approves the proxy once, in the root-owned priv.conf:
$ echo 'ndp-proxy = vtnet0=2001:db8:1:2:100::/80' | sudo tee -a /usr/local/etc/jailops/priv.conf
$ sudo service jailops_priv restart

$ jailops network create web --ipv4 10.94.0.1/24 --nat --uplink vtnet0 \
    --ipv6 2001:db8:1:2:100::/80 --ipv6-mode proxy
network/web created

$ jailops jail create site --project shop --image freebsd-15@stable \
    --network web --storage default
$ jailops jail start site --project shop
$ jailops jail list --project shop
NAME   STATE     IMAGE            GEN   ADDRESSES                       AGE
site   running   freebsd-15@stable   g1    10.94.0.2,2001:db8:1:2:100::10  7s
  • The operator decides what the proxy may do. The daemon does not run as root. The helper that does only sends advertisements for addresses inside the prefixes you wrote in priv.conf, on the uplinks you named, at a bounded rate. A compromised daemon cannot sniff other interfaces or forge other frames.
  • The jail's gateway is a link-local address on the bridge (fe80::1 above), so nothing in the provider's segment ever needs to know about it. Use a prefix longer than /64 for the jails (a /80 here); the provider still sees addresses of its own /64.
  • On an amd64 FreeBSD 15.1 host with a shared, switched IPv6 /64, jails reached the internet over IPv6, another machine on the internet reached them, stopping and deleting a jail cleaned up its route and cache entry, and the host's own address kept answering from the kernel.

Which one do I use?

Choosing between routed and switched IPv6
RoutedSwitched (proxy)
The provider gives youA prefix routed to the host's addressA /64 on the shared segment of the uplink
Who answers neighbor discoveryNobody has to; the upstream router forwards to the hostThe JailOps proxy, for each allocated address
Network prefix/80 carved out of the routed prefix/80 inside the shared /64, approved in priv.conf
Jail gatewayA real address on the bridgeA link-local address on the bridge
Extra host workForwardingForwarding, and one priv.conf line
Depends on the proxyNeverAlways
Routed and switched IPv6 In routed mode the provider's router sends a prefix to the host, which forwards to the bridge and the jails. In switched mode the router shares a /64 with other machines and asks who has each jail address; the JailOps proxy on the uplink answers with the host's MAC address, host routes lead to the bridge, and the operator's approval in priv.conf gates what the proxy may do. Routedthe prefix is routed to the host Provider router 2001:db8:1:2::/64 via the host Host, uplink vtnet0forwards; no proxy involved Bridge, gateway ::1 of a /80jails get ::2, ::3 ... written down Switcheda /64 shared on the provider's segment Provider router "who has ...:100::10?" Host, uplink vtnet0proxy answers with the host's MAC priv.conf approval /128 route per jail Bridge, gateway fe80::1jails get ...:100::10, ::11 ... Both modes: static addresses, no SLAAC, one PF anchor per jail.

Both modes use the same commands, the same firewall, the same API and the same web console. A fleet can mix them: jailops-sync only declares a network's spec, and the mode is one field of it.

Segmentation on a cloud VM

One /64, many networks.

On a cloud server you usually cannot segment your own workloads: one NIC, one /64, no VLANs, no private layer 2 between your services. JailOps gives you the segments in software. Carve the single /64 into networks, for example a /80 per project: one /64 holds 65,536 of them.

  • One network per project, never shared. Every network belongs to exactly one project.
  • Cross-project traffic is closed by default. Only a rule you write opens it.
  • Jails are bound to their own addresses. The per-jail PF anchor allows a jail's own addresses as source and its own link-local address; per-jail firewall policies live in the same anchor, and a project's policy is a floor that a jail cannot weaken.
  • No extra moving parts. No additional NICs, VLANs, virtual machines or overlay networks.

Verified on the test hosts: a carved /80 on a switched /64, and cross-project traffic closed by default (the PF test matrix). Jails of one network share a layer 2 segment, but each holds its address as a /128 and reaches everything, the other jails included, through the host. The host keeps a permanent neighbour entry for every jail address and no jail may send a neighbour advertisement, so a jail cannot change where the host or another jail sends traffic (verified with forged packets in proxy mode and in routed mode). IPv4 on one network is not covered: jails still share a /24 and can forge ARP replies to each other, so jails that must not trust each other belong on separate networks.

Host firewall: letting traffic in

A default-deny host needs one rule from you.

JailOps writes a PF anchor per jail and never touches the administrator's own ruleset. If your pf.conf blocks inbound traffic by default, which it should, packets from the internet to a jail's address are dropped by your rules before the jail's anchor can matter. Admitting them is the operator's decision. Add a rule for the jail prefix and the ports you want to expose, for example for HTTP:

/etc/pf.conf, example
# admit inbound HTTP to the jails' /80 (replace the interface and prefix with yours)
pass in quick on vtnet0 inet6 proto tcp from any to 2001:db8:1:2:100::/80 port 80 flags S/SA keep state

Use your own uplink interface and your jail network's prefix (a table or macro works too). This rule was used on a test host with a default-deny ruleset. Keep it narrow: addresses are public, so each port you open is reachable from everywhere.

Cloud setups

What it looks like on a rented server.

  • One server, one prefix, many tenants. Give each project a /80 network of its own and a PF policy, and every customer or service has its own reachable IPv6 addresses, isolated from the others without a single NAT rule. Addresses are public, so a firewall policy is the access control; the default for a new jail's anchor is closed to other projects.
  • A fleet of cheap servers. Declare the networks, the projects and the jails once in a jailops-sync repository, with the sealed secrets next to them, and roll the same declaration out to every host. Each host carries its own prefix in its own file; the jails and policies stay the same.
  • No load balancer needed for IPv6. Each service has a public address of its own. Point a DNS record at it. IPv4 can sit behind NAT and port forwards on the same network, so the same jail answers on both families.
  • Dual stack for outbound traffic. A jail's outbound IPv6 leaves with its own address, which makes allow-lists at the other end possible without sharing the host's address.
  • Move a jail between hosts by restoring an encrypted backup on the new host (replication between hosts is planned) and giving it an address from the new host's network. The jail's identity is its spec; the address follows the network.
one declaration for many hosts, example
# hosts/fra1/networks/web.yaml
apiVersion: jailops/v1
kind: Network
metadata:
  name: web
spec:
  bridge: jo30
  uplink: vtnet0
  ipv4: {subnet: 10.94.0.0/24, gateway: 10.94.0.1, range: 10.94.0.2-10.94.0.254, nat: true}
  ipv6:
    mode: proxy
    subnet: "2001:db8:1:2:100::/80"
    gateway: "fe80::1"
    range: "2001:db8:1:2:100::10-2001:db8:1:2:100::ff"

$ jailops-sync plan --host fra1
$ jailops-sync sync --apply --host fra1

What is verified

  • Verified on FreeBSD 15.1 arm64, routed: dual-stack managed bridge, jails reaching the internet over IPv4 and IPv6, the host reaching the jails, FreeBSD, Alpine and Debian jails.
  • Verified on FreeBSD 15.1 amd64, switched: the proxy answering the upstream routers, jails reaching the internet and being reached from another host for long ping runs in both directions, route and cache cleanup when a jail is deleted, the host's own address untouched, and the whole PF isolation test matrix (183 probes) passing while the proxy was running.
  • Verified on FreeBSD 15.1, switched /64: inbound TCP from the internet into a jail, with an HTTP 200 answered by a web server in the jail to a request from another host, after the pf.conf rule above admitted it on a default-deny host.
  • Not yet verified: Cloudflare in front of a jail, traffic into a jail from the internet on a routed prefix (the first test host has no IPv6 route to it), providers that filter by MAC address or that limit neighbor discovery, and many hundreds of proxied addresses on one uplink.

The status page lists everything else, and the design document describes the proxy's state machine, the helper's checks and the measurements.