A firewall you can read
How the JailOps firewall works today: one policy per jail rendered into a PF anchor, address sets, jails by name with jail:web, and a comment on every set, entry, policy and rule, so that a reviewer can read why each line is there.
The model
One anchor per jail.
A firewall policy is a list of rules with a default for inbound and a default for outbound traffic. JailOps renders the policies of a jail into one PF anchor for that jail, jailops/PROJECT/JAIL. It never edits the host's pf.conf: that file stays yours, and the anchors hang under it.
- A project can have mandatory policies, a list. They are a floor: a jail's own policies can add rules but cannot weaken the project's defaults.
- A policy always says its defaults:
default_inanddefault_out,allowordeny, are required, so nothing depends on what a default would have been. - A jail without a policy is open to its network, and
jailops jail infosays so (Firewall: open: no firewall policy). A jail with--default-in denyadmits only what a rule names. An app installed from the catalog starts closed, with a managed policy of its own (see app access). - Rules, sets and policies are objects of the project, like jails and networks. They can be made with the command line, in the console, or from files in a repository that a fleet applies.
References
Always kind:name.
A rule, an address set and app allow name who they mean with a kind, a colon and a name. There is nothing else to learn:
| Write | It means | Where |
|---|---|---|
jail:web | a jail of the project, by its addresses, read at every render | rules, sets, app allow, firewall test |
set:office | an address set of the project | rules, sets, app allow |
tag:backend | every jail of the project with that tag | rules |
192.0.2.7, 192.0.2.0/24 | an address or a prefix | everywhere |
any | every address | rules |
host | the host itself | app allow, firewall test |
The kinds are separate, so a jail and an address set may have the same name. A jail: that does not exist matches nobody, and jailops doctor says which rule names it. An address set with a single address is a perfectly good alias: give it a name, and say in its comment what it is.
Address sets
Names for who.
An address set is a named list of addresses. A rule says from set:office instead of repeating prefixes, and when the office gets a new address you change one set, not every policy.
$ jailops firewall address-set create office --comment "the office" $ jailops firewall address-set add office 192.0.2.0/24 --comment "the office, IPv4" $ jailops firewall address-set add office 2001:db8:a::/48 --comment "the office, IPv6"
A set starts empty, and a rule that names an empty set matches nobody. A member is added with add and taken away with remove; each carries its own comment, so an IPv4 and an IPv6 network of the same office are two entries. Many entries at once come from a file, or from jailops firewall address-set edit:
apiVersion: jailops/v1
kind: AddressSet
metadata:
name: office
spec:
comment: Where the staff connect from
addresses:
- address: 192.0.2.0/24
comment: the office, IPv4
- address: 2001:db8:a::/48
comment: the office, IPv6
- address: 198.51.100.7
comment: the home connection of the person on callA member of a set can also be jail:name: a jail of the project, or another set. A list such as git-clients can hold the office and the jails that push to the git server, and be added to later. The same words mean the same everywhere a firewall names a source: an address or a prefix, jail:name, and in a rule tag:name for the jails with a tag.
A policy
Rules, and why.
A rule is added to a policy with a command that names each part: jailops firewall rule add POLICY in|out, then any of --from, --to, --port, --proto, --log and a --comment that says why. A rule allows unless you say --action deny, and a port on its own means tcp. A source or destination is a prefix, an any, jail:name, set:name or tag:name.
$ jailops firewall policy create web --default-in deny --default-out allow \ --comment "the public web server" $ jailops firewall rule add web in --port 80,443 \ --comment "the site" $ jailops firewall rule add web in --from set:office --port 22 \ --comment "ssh from the office" $ jailops create web --image freebsd-15-1 --firewall-policy web --start --autostart
jail:web is the jail itself, read at every render: a rule that names a jail follows it when its addresses change. A jail and an address set of one project never share a name, so jail:web means one thing. A jail that a rule or a set names cannot be renamed or deleted until the reference is changed, so a rule never names a jail that is gone and a jail made later never takes the place of one.
Several policies
A list, in order.
A jail has a list of policies, and so has a project. Each policy is a block you can name on as many jails as you like, so a rule that many jails need is written once. The anchor of a jail holds them in this order: the project's policies, then the policies of the jail, each in the order of its list. Each one is a section of the anchor under its own heading, with its own rule numbers, so what policy show prints is what the reviewer reads in the ruleset.
$ jailops project set shop --firewall-policy default $ jailops firewall address-set create postgresql-clients --comment "who may reach PostgreSQL" $ jailops firewall address-set add postgresql-clients jail:web --comment "the web jail" $ jailops firewall policy create postgres --default-in deny --default-out allow $ jailops firewall rule add postgres in --from set:postgresql-clients \ --port 5432 --comment "who may reach PostgreSQL" $ jailops firewall policy create ssh-office --default-in deny --default-out allow $ jailops firewall rule add ssh-office in --from set:office \ --port 22 --comment "ssh from the office" $ jailops config set db firewall.policies postgres,ssh-office
Every jail of the project gets default (inbound closed, ping allowed). The database jail gets postgres and ssh-office on top, and the jail that runs something else names only the blocks it needs. jailops jail info db lists the policies in order, the project's marked as such.
- The strictest default wins. If any policy in play denies inbound, inbound is denied. A policy of a jail can open a port, never turn the project's deny into allow.
- A jail does not name the project's policies, since they are rendered for every jail already, and it does not name one twice. A policy that a list names cannot be deleted, and it is renamed in every list.
- The order does not decide between an allow and a deny: a deny always wins (below).
--firewall-policycan be given more than once atjailops create, and an empty--firewall-policy ""clears the list of a project.
Deny wins, and rules can be off
Order is not a trap.
A deny rule wins over every allow rule, whatever the order of the policies. The anchor of a jail holds the deny rules of all its policies first and the allow rules after them, each group in the order you wrote them and under the heading and number of its policy, so you can still find rule 3 of postgres. You never have to ask which policy is listed first.
$ jailops firewall policy create quiet --default-in deny --default-out allow $ jailops firewall rule add quiet in --action deny --proto tcp --from jail:web --port 5432 \ --comment "keep web out for now" $ jailops config set db firewall.policies postgres,quiet $ jailops firewall test --from jail:web --to jail:db --port 5432
The test says the connection is blocked by rule 1 of quiet, and lists the allow rule of postgres that it won over.
A rule can be switched off without deleting it. Add it with --disabled, or switch one off later with jailops firewall rule disable POLICY NUMBER. It keeps its number, policy show marks it, and the anchor has it as a comment line so a reviewer sees what is off, but it is not loaded.
$ jailops firewall rule add ssh-office in --from set:office --port 22 \
--disabled --comment "ssh from the office, off until the audit"Across projects
Another project is an external host.
A project is a boundary. Another project is treated exactly like an external host: only addresses and address sets cross it, never a jail reference. Do two things need to talk to each other as one system? Then they belong in the same project. Across projects, write the addresses explicitly on both sides: the project that serves allows the addresses in, and the project that connects allows the addresses out.
$ jailops firewall address-set create blog --address 10.30.0.5 \ --comment "the web jail of project blog" $ jailops firewall policy create db --default-in deny --default-out allow $ jailops firewall rule add db in --from set:blog --port 5432 \ --comment "the web jail of blog"
$ jailops firewall policy create out-db --default-in deny --default-out allow $ jailops firewall rule add out-db out --to 10.20.0.10 --port 5432 \ --comment "the shop database"
Each project's firewall can be read and reviewed on its own, and no command writes into a project it was not asked about.
The reasons, and when to use one project instead of two, are in Projects are boundaries.
Ping and ICMP
A policy that denies inbound, and still answers ping.
With --default-in deny a jail does not answer ping, not even from the host: echo is policy. What JailOps always lets through, whatever a policy says, is what IPv6 cannot work without: neighbor discovery and the ICMPv6 errors (destination unreachable, packet too big, time exceeded, parameter problem). Path MTU discovery depends on them.
To answer ping, say so in a rule, once, in the policy the jails share. Every jail that uses the policy gets it, and the comments say why:
$ jailops firewall policy create default --default-in deny --default-out allow $ jailops firewall rule add default in --proto icmp --comment "ping and ICMP errors" $ jailops firewall rule add default in --proto icmp6 --comment "ping6" $ jailops firewall policy show default Firewall policy default Default in: deny Default out: allow # RULE COMMENT 1 allow in icmp ping and ICMP errors 2 allow in icmp6 ping6
A jail that should not answer ping uses a policy without these two rules. The reviewer sees which one it is by reading the policy, not by guessing what a default does.
Would it pass?
Ask the firewall before you try.
jailops firewall test says whether a new connection would pass, and which rule decides at each anchor it meets. It walks the same anchors JailOps loads into pf, not a second copy of your policies, so the answer is about the rules as they are rendered.
$ jailops firewall test --from jail:web --to jail:db --port 5432 $ jailops firewall test --from jail:web --to jail:db --port 5433 $ jailops firewall test --from jail:other --to jail:db --port 5432
With the project policy default (deny in, ping allowed) and postgres on the database jail (allow in tcp from set:postgresql-clients port 5432, the set holding jail:web), the first passes and names the rule: policy postgres rule 1 "who may reach PostgreSQL", with the pf line. The second is blocked by the default of the anchor, and so is the third. Each hop prints what decided it: a policy and the number of the rule, a project policy, the managed policy of an app, the isolation of projects, or the default of the anchor.
- Both hops. What a jail sends is judged on its own port. What reaches the destination leaves the host on the gateway interface of its network, so a connection between two jails has to pass the source's anchor and the destination's.
- The ends. A jail of the project (
jail:web), one address, orhost. Another project is an external host: its jails are addresses here, and only the other end's anchor is evaluated. - Exit status 0 pass, 1 block, 8 when it cannot say.
-o jsonhas the same answer for a script, and for a console later.
What it does not know, and says so. A raw PF fragment in front of the decision (the answer is unknown, naming the policy), a connection that is already open (pf keeps its state, so the answer is for a new one), the host's own pf.conf, and NAT. Pass means "as far as these allow", and the lines under Not covered say which.
The console draws the same answer as a flow, with a form for the two ends: See a connection before it happens.
How to read the answer, use it in a script and check it with a real connection: Test the firewall before you trust it.
Reading it back
For the person who reviews it.
Every part has a comment: the set, each entry, the policy, each rule. It is one line of printable text, at most 200 characters, without double quotes or backslashes, because it becomes a comment line of the ruleset and is never part of a rule.
jailops firewall address-set show officeandjailops firewall policy show weblist the comments in a table next to the addresses and rules.- The rendered anchor of a jail carries them:
# set git-clients: 198.51.100.7 the office,# rule 1: namewith the reason on the next line,# policy: .... Someone who reads the PF ruleset on the host reads the same explanations. - The console has a comment field on a set, a policy and every rule, and takes the comment of a member at the end of its line:
198.51.100.0/24 # the office.
jailops firewall validate -f FILE checks files offline, before anything is applied.
What it will not let you do
Refusals with a reason.
- A name that is neither an address set nor a jail of the project is refused when the policy or the set is made.
- Two names that PF would read as one table (
web-1andweb_1) cannot both exist in a project. - A policy that an app installation manages (
app-JAIL) cannot be edited through the firewall commands; usejailops app allowanddeny. - Router advertisements are not passed, and ICMPv6 errors are never blocked, whatever a policy says: path MTU discovery depends on them.
Where next
Worked examples.
More articles
Jail updates without cron
Replace the nightly script that upgrades every jail: automatic jail upgrades per image, with a health check, a rollback and a record of every run, what they leave alone, and how a new FreeBSD release comes in.
Read the article: Jail updates without cronJail search and shell
Ctrl+K, a few letters, Shift+Enter: find a jail by name, label or app and open its shell. Labels that filter the jail list, bash in new jails, and the jailops command next to every step.
Read the article: Jail search and shellGetting started with the JailOps CLI
Root on the host, the command line and nothing else: install JailOps, build an image, create a network and a jail, reach it by name, snapshot it, open a port and install an app.
Read the article: Getting started with the JailOps CLIJailOps on a netcup server, with IPv6 that stays up
A /64, a router pair and a FreeBSD default that drops their probes. How every jail gets its own IPv6 address on netcup, and the one sysctl that ends the random unreachable alerts.
Read the article: JailOps on a netcup server, with IPv6 that stays upOne small cloud server, many services, solid IPv6
Run your database, web tier and workers on one low-cost server, each jail with its own IPv6 address, and fix the alerts a shared /64 causes with one sysctl.
Read the article: One small cloud server, many services, solid IPv6Write your own app: memcached, from package to catalog
Turn a small FreeBSD package into a JailOps app: the definition line by line, the offline check, a local install, the five traps, and the pull request that puts it in the catalog.
Read the article: Write your own app: memcached, from package to catalogFrom catalog to running service in one command
Install nginx from the app catalog with one command, see each of its steps, and find the data, the configuration, the logs and the snapshot afterwards.
Read the article: From catalog to running service in one commandPostgreSQL and git, closed except for who needs them
Close a database and a git server, then let in exactly the web jail and the office, with a comment on every entry.
Read the article: PostgreSQL and git, closed except for who needs themProjects are boundaries
A project is a boundary. Another project is an external host, so only addresses cross it, written out on both sides. Why that is simpler to review, and what a project does not isolate.
Read the article: Projects are boundariesThe life of a jail
A jail starts at boot only when you declared it with --autostart. Why JailOps does it this way, how the order is decided, and how to see it with jailops boot.
Read the article: The life of a jailTab completion for jailops
Press Tab and get the names of your jails, networks and firewall policies, even on another host. How to install and turn on completion in bash, zsh and fish.
Read the article: Tab completion for jailopsTest the firewall before you trust it
Ask the firewall whether a connection would pass, and why, before you try it. Read the answer, put it in a script, and know what it cannot tell you.
Read the article: Test the firewall before you trust itSee a connection before it happens
Pick a From and a To, a protocol and a port, press Run, and see the connection flow through the firewall: what passes, what stops it, and which rule decides.
Read the article: See a connection before it happensLimit what a jail can use
Give a jail a ceiling for memory, processes, CPU, disk, firewall states and bandwidth, one command each, and know what happens when it reaches it.
Read the article: Limit what a jail can useStop every jail cleanly when the host shuts down
A host that reboots should stop its databases before it stops the disks. How JailOps stops the jails in order, and how to start them in order again.
Read the article: Stop every jail cleanly when the host shuts downOne line in pf.conf
The host's firewall stays yours. JailOps asks for one line, and everything for the jails lives below it.
Read the article: One line in pf.confWhat the host tells you by itself
A pool that reports an error, an uplink that goes down, a jail that reaches a limit: they show up in the event stream by themselves.
Read the article: What the host tells you by itselfNetworks the host controls
No bridge, no DHCP. The host hands out the addresses, gives each jail a link of its own and is the router, so jails cannot see or impersonate each other.
Read the article: Networks the host controlsVolumes: data that outlives a jail
Keep the data in a volume and the jail disposable. One volume can be attached to several jails, has its own snapshots and its own backup.
Read the article: Volumes: data that outlives a jailBack up volumes once
One copy of a volume, however many jails use it. Leave volumes out when you want to, bring them back with the jail.
Read the article: Back up volumes onceA private Docker registry
A registry of your own with users in a file: one jail, docker-registry and docker-auth, htpasswd for the passwords, the images on a volume.
Read the article: A private Docker registryNames for jails
A jail is reached by its name inside its project: db.shop.internal, or just db. Create a jail and its name is there; delete it and the name is gone.
Read the article: Names for jails
Try it on one server.
Try the console from the screenshots, install on a clean FreeBSD 15.1 machine, or add JailOps to a host you already run.