Get started

From nothing to a service that answers

Pick the way in that fits you. Each one ends at the same place: nginx running in a jail of its own, and a page it served to you.

These instructions are for JailOps on FreeBSD 15.1 or later, amd64 or arm64.

What you need for B and C

Check these before you run anything. jailops doctor checks most of them again once JailOps is installed.

  • The machine

    • FreeBSD 15.1 or later. Nothing older is supported, and jailops doctor says so on an older release.
    • amd64 or arm64. The packages are built for the FreeBSD 15 ABI on these two architectures.
    • A ZFS pool JailOps may put a dataset on. The FreeBSD installer's ZFS option makes one called zroot.
    • PF as the firewall, and the kernel modules jailops doctor names.
  • You

    • Root on the host. FreeBSD has no sudo in the base system: log in as root or use su -. The command blocks below say when they run as root.
    • A browser, only if you turn the console on or choose appliance mode: it must reach the host on TCP port 39110.
    • A machine you can rebuild. Start on a server or pool you can afford to experiment with.
  • What the host fetches

    • The JailOps packages and the app catalog from pkg.jailops.net, signed.
    • FreeBSD images from pkg.freebsd.org (pkgbase), and app packages from the FreeBSD package repository your host already uses.
    • Only if you choose them: a certificate from a public CA (ACME), and a backup node of your own.

Values in this style are examples: replace them with yours. The copy buttons copy the commands without the prompt.

A · No install

Try the console

There is no hosted demo to log in to. What you can do without installing anything: walk through the real console, captured in demo mode, and read what each app of the catalog installs. Everything shown is demo data.

  1. Tour the Apps page

    Browse, read a definition, preview the steps, watch an installation and read its report: seven screens, in order.

  2. See the host side

    The host name, the firewall change that reverts itself unless confirmed, and the logs: host and firewall, who may reach an app, logs.

  3. Pick your first app

    The catalog says, for every app, which packages, ports, data paths and health check it has, before you install anything.

If you build software: the console also runs on canned demo data from the source repository, without a server; web/README.md says how.

B · A clean machine

Install on a clean FreeBSD machine

Install FreeBSD on ZFS, as the installer offers, then run two commands as root. jailops-setup does the rest of the host, and the command line works as root at once: no user, no token and no browser.

  1. Trust the package repository

    Once per host: the steps and the key fingerprint are in the jailops-packages README.

  2. Install and set the host up

    As root on the host
    # pkg install jailops
    # jailops-setup -n   # what it would do; changes nothing
    # jailops-setup

    jailops-setup loads the kernel modules, sets up the pool (the user jailops, the datasets and the two configuration files), turns on forwarding, makes the host resolver answer for the names of the jails, adds the one anchor line to /etc/pf.conf, starts jailopsd and runs jailops doctor. Every action is printed on one line before it runs, and running it again changes nothing. A host with more than one pool needs -pool zroot.

    The jailops package brings the command line (jailops-client), the daemon with its root helper, and the app catalog (jailops-apps). jailops-backupd is only for a backup node.

  3. Check the host

    As root on the host
    # jailops doctor

    Nothing should say FAIL. Each check that fails names what to do; the help page has more.

  4. A network and a project

    There is no default network or project: you name them. The storage default exists already. Replace vtnet0 with the host's uplink interface (ifconfig -l lists them).

    As root on the host
    # jailops network create lan --ipv4 10.77.0.1/24 --uplink vtnet0 --nat
    # jailops project create web
    # jailops network grant lan --project web
    # jailops storage grant default --project web
    # jailops project switch web

    For IPv6 add --ipv6 PREFIX::1/80 to network create (see IPv6). To keep all of this in git, see the provisioning example. Then continue with your first service.

The web console, if you want it

Run jailops-setup --web, or answer yes to Serve the web console (port 39110)?. The setup then offers to create the first administrator, and the console answers at https://HOST:39110/ with a self-signed certificate; your firewall must let your machine reach port 39110. Root on the host stays the administrator of the local socket, which is the way back in if a password is lost.

Or run the host from the browser: appliance mode

Appliance mode hands parts of the host to JailOps and sets it up from the console, with a one-time setup token and a wizard. Choose it when nobody will work in a shell on the machine. One switch in /etc/rc.conf, jailops_appliance, does it. Read what it takes over first.

What the appliance switch takes over on the host

  • On the first start: a user and group jailops, the datasets zroot/jailops and zroot/jailops/state on the pool of the root file system (or the pool jailops_pool names), and the files /usr/local/etc/jailops/priv.conf and jailopsd.yaml. It refuses a pool that already holds another JailOps setup, and every command it runs goes to syslog.
  • The host's identity, once you apply it: the host name, /etc/resolv.conf as a whole file, a marked block in /etc/ntp.conf, and the rc.conf variables hostname, ntpd_enable and ntpd_sync_on_start.
  • The host firewall, once you apply it: /etc/pf.conf as a whole file, and pf_enable and pf_rules. Your old pf.conf is kept in a snapshot and shown in the diff of the first change.
  • The console on TCP port 39110, over TLS, with a self-signed certificate until you replace it.

Every change shows a diff first, keeps a copy of exactly what it changes (the last 20 are kept, jailops host restore puts one back), and a hand edit is reported as drift, never overwritten in silence. A firewall change reverts itself unless you confirm it. To keep your own pf.conf, add jailops_appliance_modules="identity" (the default is every module) and use the anchor line of route C.

  1. Kernel modules and forwarding

    The three modules jailops doctor checks, loaded now and at every boot, and forwarding, set so that it survives a reboot.

    As root on the host
    # kldload -n if_epair pf zfs
    # sysrc -f /boot/loader.conf if_epair_load=YES pf_load=YES zfs_load=YES
    # sysrc gateway_enable=YES ipv6_gateway_enable=YES
    # sysctl net.inet.ip.forwarding=1 net.inet6.ip6.forwarding=1
  2. Four commands

    As root on the host
    # pkg install jailops
    # sysrc jailopsd_enable=YES jailopsd_web=YES jailops_appliance=YES
    # service jailopsd start
    # service jailopsd setuptoken

    To name the pool, add jailops_pool=zroot to the sysrc line. The start of jailopsd also prints the token and the console's address on the system console and in /var/log/auth.log. A token works once and for 24 hours; service jailopsd setuptoken prints a new one.

  3. Open the console and run the wizard

    Open https://HOST:39110/. The browser warns about the self-signed certificate until the wizard's certificate step replaces it. Enter the token, then:

    1. Administrator: the first account, made with the token. This step is the only one you cannot skip.
    2. Host name: the name, the resolvers and the time servers.
    3. Certificate: ACME when the host's DNS name points at it from the internet (it renews itself; ports 80 or 443 are redirected to JailOps, and wildcard names are not possible), a certificate of your own CA inside a network (nothing renews it; jailops doctor warns 14 days before it expires), or keep the self-signed one.
    4. Firewall: ssh only from the address your browser connects from, the console from anywhere, everything else inbound blocked. It reverts itself after a countdown unless you confirm it, so a mistake that locks you out undoes itself.
    5. Storage and network: the storage default, a network (lan, 10.77.0.0/24 unless you change it, with your uplink interface for NAT) and a project (main), for whatever does not exist yet.
    6. Base image: the FreeBSD image of the host's own release and architecture, named after the release (freebsd-15-1 on 15.1-RELEASE), created and built in the step. The build downloads the base system packages from pkg.freebsd.org and takes a few minutes; a failed build says what to check and can be tried again. One checkbox turns on the nightly upgrade of the image; it is off until you tick it. The step can be skipped.
    7. Done: Install your first app when an image exists, Create the base image when the step was skipped. Continue with your first service.

    Every step after the account can be done later from the console. If you skip the firewall step, the host needs the anchor line of route C in its own pf.conf.

  4. Check the host

    As root on the host
    # jailops doctor

    appliance should be ok and nothing should say FAIL. Each check that fails names what to do; the help page has more.

C · Your own host, by hand

I already run PF and jails

JailOps adds its own anchors to your firewall and its own datasets to your pool. It does not take over pf.conf, rc.conf or any other file of the host on this route.

  1. Install and set JailOps up

    Trust the package repository once (the jailops-packages README has the steps and the key fingerprint), then:

    As root on the host
    # pkg install jailops
    # jailops-setup -n
    # jailops-setup
    # jailops doctor

    -n shows what jailops-setup would do and changes nothing. It never changes or moves a line of your pf.conf: it inserts what is missing where pf wants it (set state-policy if-bound and a scrub rule only when there are none, and it says so), keeps the old file as /etc/pf.conf.jailops-setup-date, checks the new text with pfctl -nf before it writes anything and loads it with pfctl -f, which keeps your ssh session. It never runs service pf restart. jailops doctor should show no FAIL.

    Using bash? Add pkg install bash-completion and its line in ~/.bashrc, and Tab completes the names of your jails, networks and firewall policies: Tab completion for jailops.

  2. Rather add the line yourself?

    Do the steps of docs/install.md (sections 1, 3 and 4: the kernel modules, this line and forwarding) and jailops-setup finds them done. The anchor holds the filter rules of the jails and the NAT and port forwards of their networks, so it belongs among your filter rules, before any broad quick pass. Put scrub in all fragment reassemble near the top too.

    /etc/pf.conf, the lines to add
    # ... your options and scrub, then among your filter rules:
    anchor "jailops/*"
    As root on the host
    # pfctl -nf /etc/pf.conf
    # sysrc pf_enable=YES
    # pfctl -f /etc/pf.conf

    pfctl -nf checks the file without loading it. If pf is not running yet, start it with service pf onestart instead of the last line. Over ssh, never service pf restart: it flushes the states and cuts your session. A pf.conf that blocks by default and NATs the jails needs set state-policy if-bound at the top; docs/install.md explains why.

  3. A network and a project

    The same as route B: create a network and a project, then continue with your first service.

B and C · The end of both

Your first service: nginx, end to end

Done means a page served by nginx reached you, not a green label. If you came through the wizard, the project is main (or the name you gave it): run jailops project switch main first.

  1. An image and the catalog

    Apps and jails start from an image; the app catalog came with the jailops package. One image is one FreeBSD release, named after it: freebsd-15-1 is 15.1-RELEASE, and its generations are the patch levels of that release. If the wizard's Base image step built it, skip the first two commands; on another release, use the host's own (freebsd-version -k). Building fetches the base system from pkg.freebsd.org.

    As root on the host
    # jailops image create freebsd-15-1 --release 15.1-RELEASE
    # jailops image build freebsd-15-1
    # jailops app info nginx

    To keep the image and its jails on the newest patch level, turn on its nightly upgrade (off until you do; the checkbox Upgrade automatically every night on the console's Images page does the same):

    As root on the host
    # jailops image set freebsd-15-1 --auto-upgrade --upgrade-at 03:00

    What it does and leaves alone: Jail updates without cron.

  2. Preview, then install

    As root on the host
    # jailops app install nginx --dry-run
    # jailops app install nginx

    The dry run lists the thirteen steps and creates nothing. The installation then creates a jail named nginx, installs the package, starts the service and checks that it answers: GET http://<first address>:80/ must answer 200, retried for 30s. Only then does it close the jail and take the snapshot app-installed. The report looks like this (your version and address will differ):

    The report of jailops app install nginx
    App nginx installed as jail nginx (running)
      Version:   1.30.6,3
      Addresses: 10.77.0.2
      Ports:     http 80/tcp at 10.77.0.2:80 (expose: forwarding is not set up yet) HTTP, the default site.
                 https 443/tcp at 10.77.0.2:443 HTTPS; not configured until a certificate and a server block are added.
      Access:    closed: nothing reaches it until jailops app allow nginx --from SOURCE
      Data:      /usr/local/www The sites; the default site is nginx, copied from nginx-dist.
      Config:    /usr/local/etc/nginx/nginx.conf (restart nginx) The server configuration; ...
      Logs:      access /var/log/nginx/access.log One line per request.
                 error /var/log/nginx/error.log Errors and warnings of the server.
      Snapshot:  app-installed (jailops jail rollback nginx --all --snapshot app-installed goes back to it)
      Steps:     13 done

    If a step fails after the jail has started, the error names the step, and the jail is stopped and kept for you to look into.

  3. Let the host in, and ask nginx for a page

    Installed is not the same as reachable: the jail is closed to everyone until you allow a source. Allow the host, then fetch the page from it, with the address of the report.

    As root on the host
    # jailops app allow nginx --from host
    # fetch -qo - http://10.77.0.2/ | grep title
    <title>Welcome to nginx!</title>

    That line is your success signal: nginx answered a request. No answer: jailops app access nginx shows who may connect, and jailops jail logs nginx lists the jail's log files.

  4. See it in your own browser

    The jail's address is on a network behind the host, so reach it through an ssh tunnel to the host. The tunnel connects from the host, which you allowed.

    On your computer
    $ ssh -L 8080:10.77.0.2:80 you@HOST

    Open http://localhost:8080/: the nginx welcome page. Your request is now the last line of the access log: jailops jail logs nginx nginx/access.log --tail 5.

  5. The same in the console

    Apps, then nginx and Install; Preview steps shows the plan, and the report follows the installation. On the jail's page, the Access tab has Allow (pick the host), and the Logs tab reads its log files. See the tour.

Where to go from here

  • Your own site: put it under /usr/local/www/nginx in the jail. /usr/local/www is a dataset of its own: snapshotted, rolled back and backed up with the jail, and kept when the jail moves to a new image.
  • A shell in the jail: jailops shell nginx, or the Shell button on the jail in the console. A new jail comes with bash and a light blue prompt, so you can tell it from the host. Jail search and shell.
  • Start at boot: a jail starts when the host boots only when it has autostart: jailops config set nginx boot.autostart true, or --autostart when you create a jail. jailops list -c name,state,autostart shows which jails have it and jailops boot shows the order they start in and any that will not start, with the reason.
  • Back to the fresh install: jailops jail rollback nginx --all --snapshot app-installed. It discards everything written since, so it asks first.
  • Reaching it from the internet: forwarding the ports an app marks to expose is planned, not built. Today you admit the jail's address with a rule of your own on the host, or give each jail a public IPv6 address (IPv6 and cloud setups).
  • Other sources: another jail, an address or an address set: app access.

Before you put real data on it

Recovery you can verify

A snapshot, a rollback and a backup are three different things. Only the third survives the loss of the host.

What a snapshot, a rollback and an off-host backup protect against
WhatWhere it livesWhat it is forWhat it does not cover
SnapshotThe same ZFS pool as the jailGoing back after a bad change. JailOps takes one before a jail update, before an image upgrade moves a jail, and after an app install (app-installed); you take more with jailops jail snapshot create.A lost disk, pool or host takes the snapshots with it. Not every change takes a snapshot.
RollbackUses a snapshotScopes: roll back the system and keep what the jail wrote since, or roll back everything, data included.A rollback of everything discards the data written since the snapshot: it shows how much and asks for the jail's name.
Off-host backupA backup node on other hardwareEncrypted on the host before it leaves; the node holds no key. Verify by reading every chunk, by restoring into a scratch directory, or by booting the restored jail with no network.Host mounts, images (rebuilt from pkgbase on restore) and the host's state unless the policy includes it. A locked encrypted volume is skipped.
  • Keep these off the host

    • The recovery key (JOR1-...), printed once by jailops backup key create and kept nowhere by JailOps.
    • The host's backup.key file. Restoring on a new host with the recovery key alone is not possible from the command line yet, so keep this file in a backup of your own.
    • secrets.key (opens sealed secrets) and your TLS material: they are not in a JailOps backup.
    • The passphrases of encrypted volumes: there is no recovery for them.
  • If you lose ...

    • The host: the jails come back from the backup node onto a new host, with the old backup.key.
    • The backup node: its backups go with it; the jails on the host keep running. Back up to a new node.
    • Every copy of the key: the backups cannot be decrypted, by design. Escrow the key to an age recipient you hold (jailops backup key escrow).
  • Read more

    • docs/backup-restore.md: the node, the key, policies, verify and restore.
    • docs/storage.md: what is backed up and what is not.
    • Hosts that share a backup node share its key and can read each other's backups: give hosts that do not trust each other their own node.

Stuck somewhere?

The help page lists symptoms, causes and fixes. Every error code of the command line is explained by jailops explain.