Exposing Device Files to FreeBSD Jails

Sun 12 January 2014
By Kyle

This is one of a few posts that has been ported from an old blog on account of having a steady stream of hits from search engines.

While trying to run ARM (the Tor relay monitor) in a FreeBSD jail, I encountered errors with lsof, one of ARM's dependencies.

Installing lsof via pkg_add or pkg install works just fine, but it doesn't run...

~# lsof
lsof: kvm_open(execfile=/boot/kernel/kernel, corefile=/dev/mem): No such file or directory

Luckily, it's an intelligible error. lsof uses /dev/mem to query the system's physical memory, but the device file system is not normally visible inside jails. We can make device files visible as needed, but only at some cost to the isolation normally offered by jails.

Temporary Visibility

I only need lsof when I run ARM. If, similarly, you only need device files visible for occasional administrative tasks, consider a variant of the following approach. I use the devfs command to apply a new rule and voila, the specified jail can access the specified device:

devfs -m /path/to/jail/dev/ rule apply type <device> unhide


devfs -m /usr/jails/tor-exit.unsecu.re/dev/ rule apply type mem unhide

Clear as mud? man devfs and man devfs.rules will serve you well!

Be creative - the "unhide" and "hide" devfs commands could be incorporated into shell aliases or .login/.logout scripts so that you don't have to think about them.

Permanent Visibility

If you need the device file(s) to be visible in jail(s) all the time, say to serve requests from web apps or some monitoring utility, you'll need to add the line add type mem unhide to the jail subsection of /etc/defaults/devfs.rules. There will already be some entries there, so the final product will look similar to:

# Devices usually found in a jail.
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path zfs unhide
add type mem unhide

The above applies the rule to all jails (unless you've tampered with things in rc.conf or the ezjail config file). To apply the rule to a single jail in an ezjail environment, make a new subsection in /etc/defaults/devfs.rules instead of appending to the one shown above. Call it whatever you like, but a sensible convention might read [devfsrules_jailname_here]. Copy the contents of the default jail section as applicable, and append add type mem unhide to this new jail-specific section. Then change the variable in /usr/local/etc/ezjail/jailname_here to point to the sub subsection, e.g., export jail_jailname_here_devfs_ruleset="devfsrules_jailname_here" (adapting paths and names to fit, of course).

Keep in mind that every additional interface between the jail and the host reduces isolation and thus reduces security.

Many thanks to @freebsdhelp and @evilgjb for helping me figure this out. Per usual, hind sight is 20-20.