FreeBSD VNET Jail with OpenVPN Kill Switch and qBittorrent

[ccb056]

FreeBSD VNET Jail with OpenVPN Kill Switch and qBittorrent

I set up a FreeBSD VNET jail named jailbox to run qBittorrent through an OpenVPN tunnel to a seedbox VPN provider. The goal was:

• Use a thick FreeBSD jail.
• Keep the jail on DHCP like my other VNET jails.
• Run OpenVPN inside the jail.
• Run qBittorrent-nox inside the jail.
• Force torrent traffic through the VPN.
• Use a host-side PF kill switch so the jail cannot leak traffic through the normal WAN if the VPN drops.
• Keep the Web UI reachable from the LAN.

Final design:

Code: Select all

FreeBSD host
    -> bridge0
    -> PF enabled
    -> bridge member filtering enabled
    -> epair host-side interface for jailbox
    -> jailbox VNET jail
        -> DHCP LAN address
        -> OpenVPN client to seedbox server
        -> tun0 VPN interface
        -> qBittorrent-nox
        -> Web UI on port 8080
        -> torrent listener bound to tun0

This was done on FreeBSD 15.x using VNET jails.

1. Create the thick jail

Create the jail:

Code: Select all

bsdinstall jail /usr/local/jails/jailbox

Copy DNS configuration:

Code: Select all

cp /etc/resolv.conf /usr/local/jails/jailbox/etc/resolv.conf

2. Configure the VNET jail

Edit:

Code: Select all

/etc/jail.conf

Add a jail block. Adjust the epair number if needed. In this example I use epair14.

Code: Select all

jailbox {
    host.hostname = "jailbox";
    path = "/usr/local/jails/jailbox";

    vnet;
    vnet.interface = "jailbox0";

    exec.clean;
    exec.system_user = "root";
    exec.jail_user = "root";

    exec.prestart = "ifconfig epair14 create";
    exec.prestart += "ifconfig epair14a descr jailbox-host";
    exec.prestart += "ifconfig epair14a up";
    exec.prestart += "ifconfig bridge0 addm epair14a";
    exec.prestart += "ifconfig epair14b name jailbox0";

    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";

    exec.poststop = "ifconfig bridge0 deletem epair14a";
    exec.poststop += "ifconfig epair14a destroy";

    mount.devfs;
    devfs_ruleset = 14;

    persist;

    allow.raw_sockets = 0;
    allow.set_hostname = 0;
    allow.sysvipc = 0;

    allow.mount = 0;
    allow.mount.devfs = 0;
    allow.mount.nullfs = 0;
    allow.mount.procfs = 0;
    allow.mount.tmpfs = 0;

    allow.chflags = 0;
    enforce_statfs = 2;
    children.max = 0;
}

Important interface mapping:

Code: Select all

epair14a = host side
epair14b = renamed to jailbox0 inside the jail

The host-side epair interface is important because the PF kill switch will target epair14a, not the jail’s DHCP address. This allows the jail to remain dynamic/DHCP-based.

3. Configure devfs rules for DHCP and OpenVPN

Because the jail uses DHCP, it needs bpf. Because OpenVPN runs inside the jail, it needs tun.

Edit:

Code: Select all

/etc/devfs.rules

Add:

Code: Select all

[devfsrules_jailbox=14]
add include $devfsrules_jail

# Needed for DHCP inside the VNET jail.
add path 'bpf*' unhide

# Needed for OpenVPN inside the jail.
add path 'tun' unhide
add path 'tun*' unhide

Reload devfs:

Code: Select all

service devfs restart

4. Configure rc.conf inside the jail

Edit:

Code: Select all

/usr/local/jails/jailbox/etc/rc.conf

Use:

Code: Select all

hostname="jailbox"

ifconfig_jailbox0="SYNCDHCP"

sshd_enable="NO"
sendmail_enable="NONE"
syslogd_flags="-ss"
clear_tmp_enable="YES"

dumpdev="NO"
moused_nondefault_enable="NO"

Add the jail to the host jail list:

Code: Select all

sysrc jail_list+=" jailbox"

Start the jail:

Code: Select all

service jail start jailbox

Verify:

Code: Select all

jls
jexec jailbox hostname
jexec jailbox ifconfig jailbox0
jexec jailbox ls -l /dev/bpf*

5. Verify TUN support on FreeBSD 15.x

On FreeBSD 15.x, the TUN/TAP driver is usually if_tuntap.ko.

Check for it:

Code: Select all

ls /boot/kernel/if_tuntap.ko

Try loading it:

Code: Select all

kldload if_tuntap

If it says:

Code: Select all

module already loaded or in kernel

that is fine.

Test tun creation on the host:

Code: Select all

ifconfig tun0 create
ifconfig tun0
ls -l /dev/tun*
ifconfig tun0 destroy

Then test tun creation inside the jail:

Code: Select all

jexec jailbox ifconfig tun0 create
jexec jailbox ifconfig tun0
jexec jailbox ls -l /dev/tun*
jexec jailbox ifconfig tun0 destroy

If that works, the jail can create and use tun0, which OpenVPN needs.

6. Install OpenVPN inside jailbox

Enter the jail:

Code: Select all

jexec jailbox /bin/sh

Install OpenVPN and useful tools:

Code: Select all

pkg update
pkg install openvpn ca_root_nss curl nano

Create the OpenVPN directory:

Code: Select all

mkdir -p /usr/local/etc/openvpn
chmod 700 /usr/local/etc/openvpn

Exit back to the host:

Code: Select all

exit

Copy your seedbox provider’s OpenVPN profile into the jail.

Example:

Code: Select all

cp /home/YOURUSER/Seedbox.ovpn /usr/local/jails/jailbox/usr/local/etc/openvpn/seedbox.conf

Secure it:

Code: Select all

jexec jailbox chmod 600 /usr/local/etc/openvpn/seedbox.conf

7. Create the OpenVPN auth file

Enter the jail:

Code: Select all

jexec jailbox /bin/sh

Create:

Code: Select all

cat > /usr/local/etc/openvpn/seedbox.auth <<'EOF'
SEEDBOX_USERNAME
SEEDBOX_PASSWORD
EOF

chmod 600 /usr/local/etc/openvpn/seedbox.auth

Edit the OpenVPN profile:

Code: Select all

nano /usr/local/etc/openvpn/seedbox.conf

Find:

Code: Select all

auth-user-pass

Change it to:

Code: Select all

auth-user-pass /usr/local/etc/openvpn/seedbox.auth

If your config uses IPv6 or automatic protocol selection and you want a simpler IPv4-only setup, change:

Code: Select all

proto udp

to:

Code: Select all

proto udp4

Your remote line should look something like this, using your provider’s actual seedbox/VPN server:

Code: Select all

remote SEEDBOX_SERVER_HOSTNAME 1194

or:

Code: Select all

remote SEEDBOX_SERVER_IP 1194

Optional cleanup:

Code: Select all

auth-nocache

Do not uncomment route ignore lines such as:

Code: Select all

# pull-filter ignore 'route '
# pull-filter ignore redirect-gateway

For this design, you want OpenVPN to push default routes through the VPN.

8. Enable and start OpenVPN

Inside or from the host:

Code: Select all

jexec jailbox sysrc openvpn_enable=YES
jexec jailbox sysrc openvpn_configfile="/usr/local/etc/openvpn/seedbox.conf"

Start OpenVPN:

Code: Select all

jexec jailbox service openvpn start

Check logs:

Code: Select all

jexec jailbox tail -50 /var/log/messages

Look for:

Code: Select all

Initialization Sequence Completed

Check tun0:

Code: Select all

jexec jailbox ifconfig tun0

Check routes:

Code: Select all

jexec jailbox netstat -rn

Expected OpenVPN split default routes usually look like:

Code: Select all

0.0.0.0/1        via tun0
128.0.0.0/1      via tun0
SEEDBOX_SERVER   via LAN gateway

Test the public IP from inside the jail:

Code: Select all

jexec jailbox curl -4 https://ifconfig.me

It should show the seedbox/VPN public IP, not your residential WAN IP.

9. Set up PF kill switch on the host

The jail uses DHCP, so I did not target the jail IP address. Instead, I targeted the host-side epair interface:

Code: Select all

epair14a

Check that PF exists:

Code: Select all

which pfctl
ls -l /etc/rc.d/pf

Load PF if needed:

Code: Select all

kldload pf

Create:

Code: Select all

/etc/pf.conf

Example PF rules:

Code: Select all

#
# /etc/pf.conf
# Host-side PF rules for jailbox OpenVPN kill switch
#

jailbox_if = "epair14a"
lan_net = "192.168.11.0/24"

seedbox_vpn_ip = "SEEDBOX_SERVER_IP"
seedbox_vpn_port = "1194"

set skip on lo0

#
# Allow DHCP for jailbox.
#
pass in quick on $jailbox_if proto udp from port 68 to port 67 keep state
pass out quick on $jailbox_if proto udp from port 67 to port 68 keep state

#
# Allow jailbox DNS to LAN resolvers.
#
pass in quick on $jailbox_if proto { udp tcp } from any to $lan_net port 53 keep state

#
# Allow jailbox to connect directly only to the seedbox OpenVPN endpoint.
#
pass in quick on $jailbox_if proto udp from any to $seedbox_vpn_ip port $seedbox_vpn_port keep state

#
# Allow jailbox to talk to local LAN.
#
pass in quick on $jailbox_if from any to $lan_net keep state

#
# Allow LAN clients to reach services inside jailbox.
# This allows the qBittorrent Web UI.
#
pass out quick on $jailbox_if from $lan_net to any keep state

#
# Kill switch:
# Block all other direct non-VPN traffic from jailbox.
#
block in quick on $jailbox_if from any to any

Replace:

Code: Select all

SEEDBOX_SERVER_IP

with the actual IP address of your seedbox/VPN server.

You can resolve the server hostname with:

Code: Select all

host SEEDBOX_SERVER_HOSTNAME

Validate PF syntax:

Code: Select all

pfctl -nf /etc/pf.conf

Enable PF:

Code: Select all

sysrc pf_enable=YES
sysrc pf_rules="/etc/pf.conf"
service pf start

Verify loaded rules:

Code: Select all

pfctl -sr

10. Enable bridge member filtering

Because the jail is connected through bridge0, PF needs to inspect traffic on the bridge member interface.

Enable live:

Code: Select all

sysctl net.link.bridge.pfil_member=1
sysctl net.link.bridge.pfil_bridge=0

Make persistent:

Code: Select all

cat >> /etc/sysctl.conf <<'EOF'

# PF filtering for VNET jail bridge member interfaces
net.link.bridge.pfil_member=1
net.link.bridge.pfil_bridge=0
EOF

Verify:

Code: Select all

grep -n 'net.link.bridge.pfil' /etc/sysctl.conf

11. Test the kill switch

With OpenVPN running:

Code: Select all

jexec jailbox curl -4 https://ifconfig.me

Expected result:

Code: Select all

SEEDBOX_PUBLIC_IP

Stop OpenVPN:

Code: Select all

jexec jailbox service openvpn stop

Try internet access:

Code: Select all

jexec jailbox curl -4 --connect-timeout 10 https://ifconfig.me

Expected result:

Code: Select all

curl: (28) Connection timed out

Restart OpenVPN:

Code: Select all

jexec jailbox service openvpn start
sleep 5
jexec jailbox curl -4 https://ifconfig.me

Expected result:

Code: Select all

SEEDBOX_PUBLIC_IP

Check PF counters:

Code: Select all

pfctl -vvsr

The final block rule should increment if anything tries to bypass the VPN:

Code: Select all

block drop in quick on epair14a all

12. Install qBittorrent-nox

Install:

Code: Select all

jexec jailbox pkg install qbittorrent-nox

The package may automatically create:

Code: Select all

user:  qbittorrent
group: qbittorrent

Create torrent directories:

Code: Select all

jexec jailbox mkdir -p /srv/torrents/downloads
jexec jailbox mkdir -p /srv/torrents/incomplete
jexec jailbox mkdir -p /srv/torrents/watch
jexec jailbox mkdir -p /srv/torrents/completed

jexec jailbox chown -R qbittorrent:qbittorrent /srv/torrents
jexec jailbox chmod -R 775 /srv/torrents

Verify:

Code: Select all

jexec jailbox sh -c 'ls -ld /srv/torrents /srv/torrents/*'

13. Configure qBittorrent service

Check the rc script:

Code: Select all

jexec jailbox grep -n 'qbittorrent_' /usr/local/etc/rc.d/qbittorrent

Create config directory:

Code: Select all

jexec jailbox mkdir -p /var/db/qbittorrent/conf
jexec jailbox chown -R qbittorrent:qbittorrent /var/db/qbittorrent

Configure rc variables:

Code: Select all

jexec jailbox sysrc qbittorrent_enable=YES
jexec jailbox sysrc qbittorrent_user=qbittorrent
jexec jailbox sysrc qbittorrent_group=qbittorrent
jexec jailbox sysrc qbittorrent_conf_dir="/var/db/qbittorrent/conf"
jexec jailbox sysrc qbittorrent_download_dir="/srv/torrents/downloads"
jexec jailbox sysrc qbittorrent_flags="--confirm-legal-notice"

Start qBittorrent:

Code: Select all

jexec jailbox service qbittorrent start

Verify:

Code: Select all

jexec jailbox service qbittorrent status
jexec jailbox pgrep -lf qbittorrent
jexec jailbox sockstat -4 -l | grep -i qbit

The Web UI should listen on:

Code: Select all

*:8080

Access it from the LAN:

Code: Select all

http://jailbox:8080/

or:

Code: Select all

http://JAILBOX_DHCP_IP:8080/

14. qBittorrent Web UI notes

Default username is usually:

Code: Select all

admin

qBittorrent may generate a temporary password on first launch.

If you need to see the temporary password, stop the daemon and run qBittorrent in the foreground:

Code: Select all

jexec jailbox service qbittorrent stop

jexec jailbox su -m qbittorrent -c "qbittorrent-nox --profile=/var/db/qbittorrent/conf --save-path=/srv/torrents/downloads --confirm-legal-notice"

Copy the temporary password from the terminal output.

Then stop it with Ctrl+C and restart the service:

Code: Select all

jexec jailbox service qbittorrent start

After logging in, immediately change the Web UI password.

Do not change the Web UI to port 80 unless you are using a reverse proxy. qBittorrent runs as the unprivileged qbittorrent user, so binding directly to port 80 can fail. Keep it on:

Code: Select all

8080

15. Recommended qBittorrent settings

In the Web UI:

Connection

Code: Select all

Listening port: 51413
Use UPnP / NAT-PMP: disabled
Use different port on each startup: disabled

Advanced

Code: Select all

Network interface: tun0

Do not bind to a specific 10.x VPN address unless you are prepared to update it after reconnects. Binding to tun0 is better.

Downloads

Code: Select all

Default save path: /srv/torrents/downloads
Incomplete path:   /srv/torrents/incomplete
Watched folder:    /srv/torrents/watch

16. Verify qBittorrent is bound to the VPN

Run:

Code: Select all

jexec jailbox sockstat -4 -l | grep -i qbit

After setting qBittorrent to use tun0, the torrent listener should appear on the VPN address, for example:

Code: Select all

qbittorrent ... tcp4 10.x.x.x:51413
qbittorrent ... udp4 10.x.x.x:51413
qbittorrent ... tcp46 *:8080

The Web UI can still listen on *:8080 so it remains reachable from the LAN.

17. Final validation

With OpenVPN up:

Code: Select all

jexec jailbox curl -4 https://ifconfig.me

Expected:

Code: Select all

SEEDBOX_PUBLIC_IP

Stop OpenVPN:

Code: Select all

jexec jailbox service openvpn stop

Test for leaks:

Code: Select all

jexec jailbox curl -4 --connect-timeout 10 https://ifconfig.me

Expected:

Code: Select all

curl: (28) Connection timed out

Restart OpenVPN:

Code: Select all

jexec jailbox service openvpn start
sleep 5
jexec jailbox curl -4 https://ifconfig.me

Expected:

Code: Select all

SEEDBOX_PUBLIC_IP

18. Final state

Code: Select all

FreeBSD host
    PF loaded and enabled
    /etc/pf.conf active
    bridge member filtering enabled
    epair14a used as jailbox kill-switch interface

jailbox
    thick VNET jail
    DHCP address from LAN
    OpenVPN to seedbox server
    tun0 active
    qBittorrent-nox running as qbittorrent user
    Web UI on port 8080
    torrent listener bound to tun0
    torrent storage in /srv/torrents

Expected traffic behavior:

Code: Select all

OpenVPN up:
    jailbox exits through the seedbox VPN

OpenVPN down:
    jailbox cannot reach the internet

LAN access:
    browser can still reach http://jailbox:8080/

19. Useful status commands

Code: Select all

jexec jailbox service openvpn status
jexec jailbox service qbittorrent status
jexec jailbox curl -4 https://ifconfig.me
jexec jailbox sockstat -4 -l | grep -i qbit
pfctl -vvsr

Expected:

Code: Select all

curl returns the seedbox VPN public IP
qBittorrent torrent port is on 10.x.x.x:51413
qBittorrent Web UI is on *:8080
PF block rule increments only when something tries to bypass the VPN

[ccb056]

Add-on: Allow pfSense OpenVPN Road Warriors to Reach jailbox

This add-on applies to the original FreeBSD VNET jail/OpenVPN/qBittorrent kill-switch setup where:

Code: Select all

LAN subnet:             192.168.11.0/24
pfSense OpenVPN subnet: 10.0.11.0/24
jail name:              jailbox
jail VNET interface:    jailbox0
host-side epair:        epair14a
pfSense LAN IP:         192.168.11.1

Problem

LAN clients on 192.168.11.0/24 can ping jailbox and access services such as:

Code: Select all

http://JAILBOX_IP:8080/
http://JAILBOX_IP:7474/

However, road-warrior clients connected to pfSense OpenVPN on 10.0.11.0/24 cannot reach jailbox, even though they can reach other jails on the LAN.

Cause

There are two common issues.

• The host-side PF kill switch rules may only allow access from the LAN subnet 192.168.11.0/24, not the OpenVPN road-warrior subnet 10.0.11.0/24.
• Because jailbox runs its own OpenVPN client, the jail may have VPN-pushed default or split-default routes through tun0. Without a more specific route, replies from jailbox to road-warrior clients may try to exit through the seedbox VPN instead of going back to pfSense.

The fix is:

1. Allow the road-warrior subnet through the host PF rules on the jail’s host-side epair interface.
2. Add a static route inside jailbox so 10.0.11.0/24 goes back to pfSense via the LAN interface.

1. Add the road-warrior subnet to PF

Edit the FreeBSD host PF rules:

Code: Select all

/etc/pf.conf

Add a variable for the pfSense OpenVPN road-warrior subnet:

Code: Select all

vpn_net = "10.0.11.0/24"

If you already have:

Code: Select all

lan_net = "192.168.11.0/24"

then keep it.

Add explicit rules to allow LAN and road-warrior clients to reach only the services you want on jailbox.

For example, to allow ping, qBittorrent Web UI on 8080, and another service on 7474:

Code: Select all

#
# Allow LAN and pfSense OpenVPN road warriors to ping jailbox.
#
pass out quick on epair14a inet proto icmp from { $lan_net $vpn_net } to any keep state

#
# Allow LAN and pfSense OpenVPN road warriors to access selected jailbox services.
#
pass out quick on epair14a inet proto tcp from { $lan_net $vpn_net } to any port { 7474 8080 } flags S/SA keep state

These rules should appear before the final block rule, for example before:

Code: Select all

block in quick on epair14a from any to any

or, if using a stricter ruleset:

Code: Select all

block quick on epair14a all

Example access section

Code: Select all

lan_net = "192.168.11.0/24"
vpn_net = "10.0.11.0/24"

#
# Allow LAN and road warriors to ping jailbox.
#
pass out quick on epair14a inet proto icmp from { $lan_net $vpn_net } to any keep state

#
# Allow LAN and road warriors to access jailbox services.
#
pass out quick on epair14a inet proto tcp from { $lan_net $vpn_net } to any port { 7474 8080 } flags S/SA keep state

Reload PF:

Code: Select all

pfctl -nf /etc/pf.conf
service pf reload

Verify the loaded rules:

Code: Select all

pfctl -sr

You should see rules allowing traffic from:

Code: Select all

10.0.11.0/24

toward epair14a.

2. Add a static route inside jailbox for the road-warrior subnet

Because jailbox uses its own OpenVPN client, it may route unknown destinations through tun0. The pfSense OpenVPN subnet needs a specific route back through pfSense on the LAN.

Assuming pfSense LAN IP is:

Code: Select all

192.168.11.1

add the route inside the jail:

Code: Select all

jexec jailbox route add -net 10.0.11.0/24 192.168.11.1

Verify the route:

Code: Select all

jexec jailbox route -n get 10.0.11.10

Replace 10.0.11.10 with an actual road-warrior OpenVPN client IP if desired.

Expected output should show:

Code: Select all

gateway: 192.168.11.1
interface: jailbox0

Example:

Code: Select all

route to: 10.0.11.10
destination: 10.0.11.0
       mask: 255.255.255.0
    gateway: 192.168.11.1
  interface: jailbox0
      flags: <UP,GATEWAY,DONE,STATIC>

This confirms that replies to road-warrior clients will return through pfSense instead of through the seedbox VPN tunnel.

3. Make the route persistent

Inside the jail, make the route survive jail restarts:

Code: Select all

jexec jailbox sysrc static_routes+=" roadwarriors"
jexec jailbox sysrc route_roadwarriors="-net 10.0.11.0/24 192.168.11.1"

Verify:

Code: Select all

jexec jailbox sysrc static_routes
jexec jailbox sysrc route_roadwarriors

Expected:

Code: Select all

static_routes: roadwarriors
route_roadwarriors: -net 10.0.11.0/24 192.168.11.1

4. Test from a road-warrior client

From a machine connected to pfSense OpenVPN, test:

Code: Select all

ping JAILBOX_IP

Then test the Web UI and any other allowed service:

Code: Select all

http://JAILBOX_IP:8080/
http://JAILBOX_IP:7474/

If testing from a shell:

Code: Select all

curl -I http://JAILBOX_IP:8080/
curl -I http://JAILBOX_IP:7474/

5. Troubleshooting

If road warriors still cannot connect, check whether packets reach the host-side epair.

On the FreeBSD host:

Code: Select all

tcpdump -ni epair14a host 10.0.11.X

Replace 10.0.11.X with the road-warrior client IP.

Inside the jail:

Code: Select all

jexec jailbox tcpdump -ni jailbox0 host 10.0.11.X

Interpretation:

• If packets appear on epair14a but not inside jailbox, PF is still blocking or the bridge filtering rules need review.
• If packets appear inside jailbox but no replies return, check the route to 10.0.11.0/24.
• If packets and replies appear inside jailbox, check pfSense OpenVPN firewall rules and client-side firewall settings.

Check the route inside the jail:

Code: Select all

jexec jailbox route -n get 10.0.11.10

Correct result:

Code: Select all

gateway: 192.168.11.1
interface: jailbox0

Incorrect result would be anything using tun0 for 10.0.11.0/24.

Check that the services are listening on reachable addresses:

Code: Select all

jexec jailbox sockstat -4 -l | egrep '(:7474|:8080)'

Good examples:

Code: Select all

*:8080
192.168.11.x:8080
*:7474
192.168.11.x:7474

Bad examples:

Code: Select all

127.0.0.1:8080
127.0.0.1:7474

If a service only listens on 127.0.0.1, neither LAN clients nor road-warrior clients can reach it directly.

6. Final expected behavior

After this add-on is applied:

Code: Select all

LAN clients on 192.168.11.0/24:
    Can reach jailbox on allowed ports.

pfSense OpenVPN road warriors on 10.0.11.0/24:
    Can reach jailbox on allowed ports.

jailbox internet traffic:
    Still exits through the seedbox VPN.

Seedbox VPN down:
    Host-side PF kill switch still blocks direct internet leaks.

Replies from jailbox to road warriors:
    Return through pfSense at 192.168.11.1 using jailbox0.

Final working route inside jailbox

Code: Select all

10.0.11.0/24 via 192.168.11.1 dev jailbox0

Final working PF concept

Code: Select all

Allow jailbox to reach only the VPN endpoint directly.
Allow LAN and road-warriors to reach selected local services.
Block all other direct non-VPN traffic from jailbox.

[ccb056]

This creates a ZFS pool named tank, mounts it at /tank, and makes part of it available inside jailbox as /storage.

Warning: Confirm that the external drive is da0. Do not run the partitioning commands against the FreeBSD system disk, which is commonly ada0.

Final layout:

Code: Select all

USB drive:          /dev/da0
GPT partition:      /dev/gpt/zfs6tb
ZFS pool:           tank
Host mount:         /tank
Host jail storage:  /tank/jailbox
Path inside jail:   /storage

1. Identify the USB drive

Code: Select all

usbconfig list
camcontrol devlist
sysctl kern.disks
diskinfo -v /dev/da0

Confirm that /dev/da0 is the external drive and that its reported capacity is correct.

2. Create the GPT partition

Code: Select all

gpart create -s GPT da0
gpart add -a 1M -t freebsd-zfs -l zfs6tb da0

This creates an aligned ZFS partition with the persistent label /dev/gpt/zfs6tb.

Code: Select all

gpart show -lp da0
ls -l /dev/gpt/zfs6tb

Verify that zfs6tb occupies nearly the entire external drive.

3. Preview the ZFS pool creation

Code: Select all

zpool create -n -o ashift=12 -O compression=zstd -O atime=off -O mountpoint=/tank tank /dev/gpt/zfs6tb

The dry run should show only gpt/zfs6tb.

4. Create the ZFS pool

Code: Select all

zpool create -o ashift=12 -O compression=zstd -O atime=off -O mountpoint=/tank tank /dev/gpt/zfs6tb

This creates the pool named tank and mounts it at /tank.

5. Verify the pool

Code: Select all

zpool status -v tank
zpool list tank
zfs list tank
df -h /tank

The pool should be ONLINE with no known data errors.

Code: Select all

zpool get ashift tank
zfs get compression,atime,mountpoint tank

Expected values are ashift=12, compression=zstd, atime=off, and mountpoint=/tank.

6. Configure automatic import at boot

Code: Select all

sysrc zfs_enable=YES
zpool set cachefile=/boot/zfs/zpool.cache tank

This enables ZFS during startup and records tank in the ZFS pool cache.

Code: Select all

zpool get cachefile tank
ls -l /boot/zfs/zpool.cache

Confirm that tank uses /boot/zfs/zpool.cache.

7. Create the jail storage directories

Code: Select all

mkdir -p /tank/jailbox
mkdir -p /usr/local/jails/jailbox/storage

The host directory /tank/jailbox will appear inside the jail as /storage.

8. Test the nullfs mount

Code: Select all

mount -t nullfs /tank/jailbox /usr/local/jails/jailbox/storage

This temporarily exposes the host directory inside jailbox.

Code: Select all

mount | grep '/usr/local/jails/jailbox/storage'
jexec jailbox df -h /storage
jexec jailbox touch /storage/test
ls -l /tank/jailbox/test
rm /tank/jailbox/test

These commands verify that the jail can see and write to the external storage.

9. Create the persistent nullfs configuration

Create /etc/fstab.jailbox:

Code: Select all

echo '/tank/jailbox  /usr/local/jails/jailbox/storage  nullfs  rw  0  0' > /etc/fstab.jailbox

Verify the file:

Code: Select all

cat /etc/fstab.jailbox

Expected content:

Code: Select all

/tank/jailbox  /usr/local/jails/jailbox/storage  nullfs  rw  0  0

10. Update /etc/jail.conf

Add this line inside the jailbox block, immediately after the path line:

Code: Select all

mount.fstab = "/etc/fstab.jailbox";

Example:

Code: Select all

jailbox {
    host.hostname = "jailbox";
    path = "/usr/local/jails/jailbox";

    mount.fstab = "/etc/fstab.jailbox";

    vnet;
    vnet.interface = "jailbox0";

    # Keep the remaining jailbox configuration here.
}

Keep the existing mount restrictions unchanged:

Code: Select all

allow.mount = 0;
allow.mount.nullfs = 0;

The FreeBSD host performs the nullfs mount, so the jail itself does not need permission to mount filesystems.

11. Restart and verify jailbox

Stop the jail:

Code: Select all

service jail stop jailbox

Check whether the temporary test mount still exists:

Code: Select all

mount | grep '/usr/local/jails/jailbox/storage'

If it is still mounted, unmount it:

Code: Select all

umount /usr/local/jails/jailbox/storage

Start the jail:

Code: Select all

service jail start jailbox

Verify that the persistent mount was created automatically:

Code: Select all

jls -j jailbox
mount | grep '/usr/local/jails/jailbox/storage'
jexec jailbox df -h /storage

Expected mount:

Code: Select all

/tank/jailbox on /usr/local/jails/jailbox/storage (nullfs, local)

12. Test the persistent mount

Code: Select all

jexec jailbox sh -c 'echo "persistent mount works" > /storage/persistent-test.txt'
cat /tank/jailbox/persistent-test.txt
rm /tank/jailbox/persistent-test.txt

The file created inside the jail should also be visible on the FreeBSD host.

13. Give qBittorrent permission

Check the qBittorrent UID and GID:

Code: Select all

jexec jailbox id qbittorrent

Example result:

Code: Select all

uid=850(qbittorrent) gid=850(qbittorrent) groups=850(qbittorrent)

If both values are 850, set ownership on the host:

Code: Select all

chown 850:850 /tank/jailbox
chmod 775 /tank/jailbox

Use the actual numeric UID and GID returned by the previous command if they differ.

Verify the numeric ownership inside the jail:

Code: Select all

jexec jailbox ls -ldn /storage

Test access as qBittorrent:

Code: Select all

jexec -U qbittorrent jailbox touch /storage/qbittorrent-write-test
jexec jailbox ls -ln /storage/qbittorrent-write-test
jexec -U qbittorrent jailbox rm /storage/qbittorrent-write-test

If these commands succeed, qBittorrent can write to the external drive.

14. Create the download directories

Code: Select all

jexec jailbox mkdir -p /storage/downloads
jexec jailbox mkdir -p /storage/incomplete
jexec jailbox mkdir -p /storage/completed
jexec jailbox mkdir -p /storage/watch

Assign the directories to qBittorrent:

Code: Select all

jexec jailbox chown -R qbittorrent:qbittorrent /storage
jexec jailbox chmod 2775 /storage
jexec jailbox chmod 2775 /storage/downloads
jexec jailbox chmod 2775 /storage/incomplete
jexec jailbox chmod 2775 /storage/completed
jexec jailbox chmod 2775 /storage/watch

The setgid permission keeps new files and directories associated with the qbittorrent group.

15. Test the download directories

Code: Select all

jexec -U qbittorrent jailbox touch /storage/downloads/test
jexec -U qbittorrent jailbox touch /storage/incomplete/test
jexec -U qbittorrent jailbox touch /storage/completed/test
jexec -U qbittorrent jailbox touch /storage/watch/test

Remove the test files:

Code: Select all

jexec -U qbittorrent jailbox rm /storage/downloads/test
jexec -U qbittorrent jailbox rm /storage/incomplete/test
jexec -U qbittorrent jailbox rm /storage/completed/test
jexec -U qbittorrent jailbox rm /storage/watch/test

If all commands succeed, qBittorrent has working permissions throughout the storage area.

16. Configure qBittorrent

In the qBittorrent Web UI, open:

Code: Select all

Tools -> Options -> Downloads

Set the default save path to:

Code: Select all

/storage/downloads

If the incomplete-torrent option is enabled, use:

Code: Select all

/storage/incomplete

The additional directories are available for categories or automation:

Code: Select all

/storage/completed
/storage/watch

Restart qBittorrent after changing its configuration:

Code: Select all

jexec jailbox service qbittorrent restart
jexec jailbox pgrep -laf qbittorrent

17. Verify after reboot

Code: Select all

zpool status tank
zfs get mounted,mountpoint tank
mount | grep '/usr/local/jails/jailbox/storage'
jexec jailbox df -h /storage

The expected startup order is:

Code: Select all

FreeBSD starts
tank imports and mounts at /tank
jailbox starts
/etc/fstab.jailbox is processed
/tank/jailbox appears as /storage inside jailbox
qBittorrent starts and can write to /storage

18. Safely disconnect the USB drive

Before unplugging the drive:

Code: Select all

service jail stop jailbox
zpool export tank

After reconnecting the drive:

Code: Select all

zpool import tank
service jail start jailbox

Verify the pool and jail storage:

Code: Select all

zpool status tank
mount | grep '/usr/local/jails/jailbox/storage'
jexec jailbox df -h /storage

Important: This is a single-disk ZFS pool. It provides checksums, compression, snapshots, and corruption detection, but it cannot survive a complete drive failure. Keep another copy of important data.

[ccb056]

Enable OpenVPN DCO in a FreeBSD VNET Jail

I run OpenVPN and qBittorrent inside a FreeBSD 15.x VNET jail. Download speeds were limited to roughly 400 Mbps, and the traditional OpenVPN process consumed most of one CPU core.

OpenVPN already supported DCO:

Code: Select all

jexec JAIL_NAME openvpn --version

The output included:

Code: Select all

[DCO]

I confirmed the FreeBSD module existed:

Code: Select all

ls -l /boot/kernel/if_ovpn.ko

I backed up the VPN profile:

Code: Select all

cp -p \
  /usr/local/jails/JAIL_NAME/usr/local/etc/openvpn/client.conf \
  /usr/local/jails/JAIL_NAME/usr/local/etc/openvpn/client.conf.pre-dco

I checked for common DCO blockers:

Code: Select all

jexec JAIL_NAME grep -n -E \
'^[[:space:]]*(compress|comp-lzo|allow-compression|fragment|disable-dco|user|group)([[:space:]]|$)' \
/usr/local/etc/openvpn/client.conf

No output was returned.

The profile already used UDP:

Code: Select all

dev tun
proto udp4

I loaded the DCO module on the host:

Code: Select all

kldload if_ovpn
kldstat | grep if_ovpn

Then restarted OpenVPN:

Code: Select all

jexec JAIL_NAME service openvpn restart
sleep 5

The log confirmed DCO:

Code: Select all

DCO device tun0 opened
ovpn-dco device [tun0] opened
Initialization Sequence Completed

FreeBSD retained the interface name

Code: Select all

tun0

, but it now showed:

Code: Select all

groups: openvpn

The OpenVPN process also stopped holding

Code: Select all

/dev/tun0

directly:

Code: Select all

procstat -f "$(pgrep -x openvpn)"

Results:

• OpenVPN userspace CPU dropped from roughly 70 percent to nearly zero.
• VPN throughput improved.
• qBittorrent remained bound to

  Code: Select all

  tun0

  .
• Routing and the VPN public IP remained correct.

After testing, I made the module load at boot:

Code: Select all

sysrc -f /boot/loader.conf if_ovpn_load=YES

Verify after reboot:

Code: Select all

kldstat | grep if_ovpn
jexec JAIL_NAME ifconfig tun0

The VPN interface should show:

Code: Select all

groups: openvpn

Important: Test your firewall kill switch after enabling DCO. Stopping OpenVPN must not allow the jail to use the normal Internet connection.

[ccb056]

Load Balance Two OpenVPN DCO Tunnels in a FreeBSD VNET Jail

I run OpenVPN and qBittorrent inside a FreeBSD 15.x VNET jail. One DCO tunnel worked well, but I wanted qBittorrent's many TCP and UDP connections distributed across two tunnels.

First, I confirmed DCO and multipath support on the host:

Code: Select all

kldload if_ovpn
kldstat | grep if_ovpn
sysctl net.route.multipath

Inside the jail, OpenVPN reported DCO support:

Code: Select all

jexec JAIL_NAME openvpn --version

The output included:

Code: Select all

[DCO]

I created two OpenVPN profiles:

Code: Select all

/usr/local/etc/openvpn/vpn0.conf
/usr/local/etc/openvpn/vpn1.conf

The important settings were:

Code: Select all

# vpn0.conf
dev tun0
proto udp4
remote VPN_SERVER_HOSTNAME 1194
route-nopull
pull-filter ignore "redirect-gateway"
script-security 2
route-delay 3
route-up /usr/local/sbin/update-vpn-pf
route VPN_SERVER_HOSTNAME 255.255.255.255 net_gateway

Code: Select all

# vpn1.conf
dev tun1
proto udp4
remote VPN_SERVER_HOSTNAME 1194
route-nopull
pull-filter ignore "redirect-gateway"
script-security 2
route-delay 3
route-up /usr/local/sbin/update-vpn-pf
route VPN_SERVER_HOSTNAME 255.255.255.255 net_gateway

I created two OpenVPN service instances inside the jail:

Code: Select all

cd /usr/local/etc/rc.d
ln -s openvpn openvpn_vpn0
ln -s openvpn openvpn_vpn1

The corresponding jail settings were:

Code: Select all

openvpn_enable="NO"

openvpn_vpn0_enable="YES"
openvpn_vpn0_configfile="/usr/local/etc/openvpn/vpn0.conf"
openvpn_vpn0_dir="/usr/local/etc/openvpn"

openvpn_vpn1_enable="YES"
openvpn_vpn1_configfile="/usr/local/etc/openvpn/vpn1.conf"
openvpn_vpn1_dir="/usr/local/etc/openvpn"

pf_enable="YES"
pf_rules="/etc/pf.conf"

I used a small bootstrap PF configuration:

Code: Select all

# /etc/pf.conf
#
# The dynamic two-tunnel ruleset is generated as:
# /var/run/pf.vpn.conf
#
# The host-side PF rules remain the external kill switch.

set skip on lo0
pass all

A route-up script dynamically reads the jail address, connected LAN network, current VPN server address, and both tunnel IP and peer addresses.

It then generates and validates a runtime PF ruleset before loading it.

The relevant generated PF rules look like:

Code: Select all

pass out quick on $jail_if route-to (tun0 $tun0_gw) \
    inet proto tcp from $jail_ip to any probability 50% \
    nat-to $tun0_ip keep state

pass out quick on $jail_if route-to (tun1 $tun1_gw) \
    inet proto tcp from $jail_ip to any \
    nat-to $tun1_ip keep state

pass out quick on $jail_if route-to (tun0 $tun0_gw) \
    inet proto udp from $jail_ip to any probability 50% \
    nat-to $tun0_ip keep state

pass out quick on $jail_if route-to (tun1 $tun1_gw) \
    inet proto udp from $jail_ip to any \
    nat-to $tun1_ip keep state

LAN, DNS, management, and OpenVPN transport traffic are excluded from the balancing rules.

The generator discovers the current values instead of permanently storing VPN addresses in the main PF configuration.

I started both tunnels:

Code: Select all

jexec JAIL_NAME service openvpn_vpn0 start
jexec JAIL_NAME service openvpn_vpn1 start

Both services were running:

Code: Select all

jexec JAIL_NAME service openvpn_vpn0 status
jexec JAIL_NAME service openvpn_vpn1 status

Both interfaces confirmed DCO:

Code: Select all

jexec JAIL_NAME ifconfig tun0
jexec JAIL_NAME ifconfig tun1

Each interface showed:

Code: Select all

groups: openvpn

The generated PF ruleset was loaded from:

Code: Select all

/var/run/pf.vpn.conf

I verified PF and the dynamic VPN server table:

Code: Select all

jexec JAIL_NAME pfctl -s info
jexec JAIL_NAME pfctl -sr
jexec JAIL_NAME pfctl -sn
jexec JAIL_NAME pfctl -t vpn_servers -T show

PF confirmed that TCP and UDP states were using both tunnels:

Code: Select all

jexec JAIL_NAME pfctl -vvsr
jexec JAIL_NAME pfctl -ss
jexec JAIL_NAME netstat -I tun0
jexec JAIL_NAME netstat -I tun1

With qBittorrent running, translated states appeared on both tunnel addresses.

Results:

• Both OpenVPN connections use DCO.
• TCP and UDP connections are distributed approximately 50/50.
• Each connection remains on one tunnel because PF keeps state.
• Tunnel IP addresses and peer gateways are discovered dynamically.
• The current VPN server address is resolved dynamically.
• qBittorrent uses both tunnel paths.
• LAN, DNS, and management traffic remain on the normal jail interface.
• The host-side PF kill switch remains the final leak-prevention layer.

This is per-connection load balancing, not packet-level bonding.

A single TCP or UDP flow remains on one tunnel. Applications such as qBittorrent benefit because they create many independent peer connections that can be distributed across both tunnels.

Important: Test the firewall kill switch after making these changes. If both OpenVPN processes stop, the jail must not gain direct Internet access through its normal interface.