How to Configure FreeBSD Kernel?

The heart of the FreeBSD operating system is the kernel. It is responsible for memory management, security enforcement, networking, disk access, and much more. Even though the majority of FreeBSD is dynamically changeable, it is sometimes required to configure and create a custom kernel.

In this tutorial, we will cover the following topics:

• When to construct a kernel from scratch.
• How to take an inventory of hardware.
• How to alter a kernel configuration file.
• How to generate and build a new kernel using the kernel configuration file.
• Installing the new kernel.
• How to troubleshoot when issues arise.

Why Do You Need a Customized Kernel?

Historically, FreeBSD used a single kernel. The kernel was a single, huge program that supported a set list of devices. To modify the kernel's functionality, one had to build and reboot into a new kernel.

Currently, the majority of the FreeBSD kernel's functionality resides in modules that are dynamically loaded and unloaded as required. This enables the operating kernel to instantly adapt to new hardware and to include new capabilities. It is called a modular kernel.

It is sometimes essential to do static kernel settings. Occasionally, the required functionality is so tightly bound to the kernel that it cannot be dynamically loaded. Certain security contexts prohibit the loading and unloading of kernel modules and mandate that only the necessary functionality be statically built into the kernel.

Building a custom kernel is often a need for sophisticated BSD users. This time-consuming technique gives advantages to the FreeBSD system. In contrast to the GENERIC kernel, which must support a variety of hardware, a custom kernel is tailored to suit just the computer's hardware. Custom kernel offers the following advantages:

• Reduced boot time: Since the kernel just examines the system's hardware, the system's startup time is reduced.

• Reduced memory usage: By removing unnecessary functionality and device drivers, a custom kernel usually utilizes less RAM than the GENERIC kernel. This is significant because the kernel code always resides in physical memory, prohibiting programs from using that memory. For this reason, a modified kernel is advantageous for systems with little RAM.

• Additional hardware support: A custom kernel adds device support that is absent from the GENERIC kernel. Before constructing a custom kernel, evaluate the motivation for doing so. If specialized hardware support is required, it may already exist as a module.

Kernel modules reside in /boot/kernel and are dynamically loaded via kldload into the operating kernel. The majority of kernel drivers provide a loadable module and a documentation page. For example, the ath(4) wireless network driver's manual page has the following information:

ATH(4) FreeBSD Kernel Interfaces Manual ATH(4)

NAME
ath � Atheros IEEE 802.11 wireless network driver

SYNOPSIS
To compile this driver into the kernel, place the following lines in your kernel configuration file:

device ath
device ath_pci
device ath_hal
device ath_rate_sample
device wlan

Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5):

if_ath_load="YES"
if_ath_pci_load="YES"

As explained in ath(4) manual, adding if_ath_load="YES" in /boot/loader.conf will dynamically load this module at boot time.

In some instances, there is no corresponding module in /boot/kernel. This holds true for a majority of subsystems.

How to Identify System Hardware?

Before modifying the kernel configuration file, it is suggested to do a hardware inventory. On a FreeBSD operating system, you may use dmesg to ascertain the hardware discovered and listed by the boot probe as given below.

Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: Common KVM processor (3066.77-MHz K8-class CPU)
Origin="GenuineIntel" Id=0xf61 Family=0xf Model=0x6 Stepping=1
Features=0x783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2>
Features2=0x80202001<SSE3,CX16,x2APIC,HV>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1<LAHF>
Hypervisor: Origin = "KVMKVMKVM"
real memory = 8489271296 (8096 MB)
avail memory = 8172363776 (7793 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <BOCHS BXPCAPIC>
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-23
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
kvmclock0: <KVM paravirtual clock>
Timecounter "kvmclock" frequency 1000000000 Hz quality 975
kvmclock0: registered as a time-of-day clock, resolution 0.000001s
smbios0: <System Management BIOS> at iomem 0xf58a0-0xf58be
smbios0: Version: 2.8, BCD Revision: 2.8
aesni0: No AES or SHA support.
acpi0: <BOCHS BXPCRSDT>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe0e0-0xe0ef at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xe080-0xe09f irq 11 at device 1.2 on pci0
usbus0 on uhci0
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem 0xfd000000-0xfdffffff,0xfea90000-0xfea90fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: <VirtIO PCI (legacy) Balloon adapter> port 0xe000-0xe03f mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: <VirtIO Balloon Adapter> on virtio_pci0
virtio_pci1: <VirtIO PCI (legacy) SCSI adapter> port 0xe040-0xe07f mem 0xfea91000-0xfea91fff,0xfe404000-0xfe407fff irq 10 at device 5.0 on pci0
vtscsi0: <VirtIO SCSI Adapter> on virtio_pci1
virtio_pci2: <VirtIO PCI (legacy) Network adapter> port 0xe0a0-0xe0bf mem 0xfea92000-0xfea92fff,0xfe408000-0xfe40bfff irq 10 at device 18.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci2
vtnet0: Ethernet address: 26:be:03:c3:a5:cc
vtnet0: netmap queues/slots: TX 1/256, RX 1/128
000.000760 [ 450] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128
virtio_pci3: <VirtIO PCI (legacy) Network adapter> port 0xe0c0-0xe0df mem 0xfea93000-0xfea93fff,0xfe40c000-0xfe40ffff irq 11 at device 19.0 on pci0
vtnet1: <VirtIO Networking Adapter> on virtio_pci3
vtnet1: Ethernet address: 9e:95:a7:6f:72:93
vtnet1: netmap queues/slots: TX 1/256, RX 1/128
000.000761 [ 450] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128
pcib1: <ACPI PCI-PCI bridge> mem 0xfea94000-0xfea940ff irq 10 at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> mem 0xfea95000-0xfea950ff irq 11 at device 31.0 on pci0
pci2: <ACPI PCI bus> on pcib2
acpi_syscontainer0: <System Container> on acpi0
vmgenc0: <VM Generation Counter> on acpi0
acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: <System Container> port 0xae00-0xae13 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: <ISA Option ROM> at iomem 0xea800-0xeffff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
attimer0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14.
fdc0: No FDOUT register!
Timecounter "TSC-low" frequency 1533333043 Hz quality 800
Timecounters tick every 10.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
usbus0: 12Mbps Full Speed USB v1.0
ugen0.1: <Intel UHCI root HUB> at usbus0
uhub0 on usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
Trying to mount root from zfs:zroot/ROOT/default []...
da0 at vtscsi0 bus 0 scbus2 target 0 lun 0
da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 32768MB (67108864 512 byte sectors)
uhub0: 2 ports with 2 removable, self powered
ugen0.2: <QEMU QEMU USB Tablet> at usbus0
intsmb0: <Intel PIIX4 SMBUS Interface> irq 9 at device 1.3 on pci0
intsmb0: intr IRQ 9 enabled revision 0
smbus0: <System Management Bus> on intsmb0
lo0: link state changed to UP
vtnet0: link state changed to UP
vtnet1: link state changed to UP
uhid0 on uhub0
uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 2> on usbus0
pflog0: promiscuous mode enabled

The majority of FreeBSD device drivers provide a documentation page detailing the available hardware. For instance, the lines below show that the psm(4) driver detected a mouse:

psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4

This driver should not be deleted from a custom kernel configuration file since the hardware exists.

If the output of dmesg does not show the boot probe output, read the contents of /var/run/dmesg.boot.

pciconf is an alternative program for locating hardware that delivers more verbose output. For instance, run the next command:

pciconf -lv

You may see the following output showing that the ath driver located a wireless Ethernet device:

ath0@pci0:3:0:0: class=0x020000 card=0x058a1014 chip=0x1014168c rev=0x01 hdr=0x00
vendor = 'Atheros Communications Inc.'
device = 'AR5212 Atheros AR5212 802.11abg wireless'
class = network
subclass = ethernet

The man -k option is used to get relevant information. For instance, to view a list of manual pages including the brand or name of a certain device you may run the next command:

man -k Atheros

You may see the output similar to the given below:

ae, if_ae(4) - Attansic/Atheros L2 FastEthernet controller driver
age, if_age(4) - Attansic/Atheros L1 Gigabit Ethernet driver
alc, if_alc(4) - Atheros AR813x/AR815x/AR816x/AR817x Gigabit/Fast Ethernet driver
ale, if_ale(4) - Atheros AR8121/AR8113/AR8114 Gigabit/Fast Ethernet driver
ath, if_ath(4) - Atheros IEEE 802.11 wireless network driver
ath_ahb(4) - Atheros AHB device glue
ath_hal(4) - Atheros Hardware Access Layer (HAL)
ath_pci, if_ath_pci(4) - Atheros PCI device glue
otus, if_otus(4) - Atheros AR9170 USB IEEE 802.11a/b/g/n wireless network device
uath, if_uath(4) - Atheros USB IEEE 802.11a/b/g wireless network device
ath3kfw(8) - firmware download utility for Atheros AR3011/AR3012 chip based Bluetooth USB devices
uathload(8) - firmware loader for Atheros USB wireless driver

Once the hardware inventory list has been generated, it should be consulted to verify that drivers for installed hardware are not deleted while the kernel configuration is modified.

How to Edit Kernel Configuration File?

To generate a custom kernel configuration file and construct a custom kernel, it is necessary to install the whole FreeBSD source tree.

If the /usr/src directory does not exist or is empty, source was not installed. You may install source code using Git.

After installing source, examine the contents of /usr/src/sys. This directory comprises subdirectories representing the supported architectures amd64, i384, powerpc, and sparc64. Everything inside a single architecture's directory is specific to that architecture, whereas the remainder of the code is platform-independent machine-independent code. Each architecture supported includes a conf subfolder containing the kernel configuration file for that architecture.

Do not alter the GENERIC file. Instead, copy the file to a new name and make any necessary changes to the copy. The norm is to capitalize a name completely. When managing numerous FreeBSD computers with varied hardware, it is prudent to name them after the hostname of the machine. This example builds a clone of the GENERIC configuration file for the amd64 architecture, entitled MYKERNEL.

cd /usr/src/sys/amd64/conf
cp GENERIC MYKERNEL

MYKERNEL is now modifiable using any ASCII text editor. The default editor is vi, however ee, a beginner-friendly editor, is also available with FreeBSD.

The configuration file format for the kernel is straightforward.

• Each line comprises a device or subsystem-representing term, an argument, and a short explanation.

• Any content after a # is treated as a comment and disregarded.

• To remove kernel support for a device or subsystem, prefix the line describing the device or subsystem with a #. Do not add or delete a # for lines you cannot comprehend.

warning

It is simple to remove support for a device or option and break the kernel as a result. If the ata(4) driver is deleted from the kernel configuration file, for instance, a system employing ATA disk drivers may fail to start. When in doubt, support should be left in the kernel.

NOTES, which may be located in the same directory as GENERIC for that architecture, has further descriptions. Refer to /usr/src/sys/conf/NOTES for architecture-independent options.

tip

After altering the kernel configuration file, store a backup copy outside of the /usr/src directory.

Alternately, you may store the kernel configuration file elsewhere and make a symbolic link to it by running the next commands:

cd /usr/src/sys/amd64/conf
mkdir /root/kernels
cp GENERIC /root/kernels/MYKERNEL
ln -s /root/kernels/MYKERNEL

include directives are supported inside configuration files. This enables another configuration file to be included in the current one, making it easier to manage tiny modifications relative to an existing file. This enables a delta to be preserved relative to GENERIC if just a limited number of new options or drivers are necessary, as shown here:

include GENERIC
ident MYKERNEL

options IPFIREWALL
options DUMMYNET
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPDIVERT

The local configuration file represents local variations from a GENERIC kernel using this manner. As updates are conducted, new features introduced to GENERIC are likewise added to the local kernel unless nooptions or nodevice are used to prevent this.

tip

To build a file which contains all available options, run the following command as root for amd64 architecture:

cd /usr/src/sys/amd64/conf && make LINT

The following architectures are available:

• amd64

• Arm

• Arm64

• I386

• x86

How to Build and Install a Custom Kernel?

Once the modifications to the custom configuration file have been saved, the kernel source code is built using the steps below:

1. Go to /usr/src folder:

cd /usr/src

2. Compile the new kernel by supplying the custom configuration file's name:

make buildkernel KERNCONF=MYKERNEL

You should see the output similar to the given below:

.................................
.................................
awk -f /usr/src/sys/conf/kmod_syms.awk zlib.ko.full export_syms | xargs -J% objcopy % zlib.ko.full
objcopy --only-keep-debug zlib.ko.full zlib.ko.debug
objcopy --strip-debug --add-gnu-debuglink=zlib.ko.debug zlib.ko.full zlib.ko
--------------------------------------------------------------
>>> Kernel build for MYKERNEL completed on Fri Jan 6 12:57:20 +03 2023
--------------------------------------------------------------
>>> Kernel(s) MYKERNEL built in 2714 seconds, ncpu: 1
--------------------------------------------------------------

3. Install the new kernel associated with the configuration file given. This command will save the old kernel to /boot/kernel.old/kernel and transfer the new kernel to /boot/kernel/kernel:

make installkernel KERNCONF=MYKERNEL

At the end of the installation, you should see the output similar to the below:

===> zfs (install)
install -T release -o root -g wheel -m 555 zfs.ko /boot/kernel/
install -T dbg -o root -g wheel -m 555 zfs.ko.debug /usr/lib/debug/boot/kernel/
===> zlib (install)
install -T release -o root -g wheel -m 555 zlib.ko /boot/kernel/
install -T dbg -o root -g wheel -m 555 zlib.ko.debug /usr/lib/debug/boot/kernel/
kldxref /boot/kernel
--------------------------------------------------------------
>>> Installing kernel MYKERNEL completed on Fri Jan 6 13:13:00 +03 2023
--------------------------------------------------------------

4. Reboot the machine with the updated kernel installed by running the next command:

shutdown -r now

When a custom kernel is produced, all kernel modules are recompiled by default. To update a kernel more quickly or to produce solely custom modules, change /etc/make.conf prior to beginning the kernel compilation process.

The following variable, for instance, provides the list of modules to create, as opposed to the default of constructing all modules:

MODULES_OVERRIDE = linux acpi

Alternatively, the following variable specifies which modules should be omitted from the build process.

WITHOUT_MODULES = linux acpi sound

Additional variables are available.

How to Fix If Building Custom Kernel Fails?

There are four types of problems that might arise while constructing a custom kernel:

• config fails: If config fails, the erroneous line number will be shown. Verify, for instance, that line 45 of the following message is accurately written by comparing it to GENERIC or NOTES:

config: line 45: syntax error

• make fails: If make fails, it is often due to a minor issue in the kernel configuration file that config did not notice. Review the settings, and if the issue is not evident, send an email with the kernel configuration file to the FreeBSD general inquiries mailing group.

• kernel does not boot: FreeBSD offers an effective technique for recovering from kernel incompatibility. Simply choose the kernel from which to boot at the FreeBSD boot loader. This may be reached by choosing the "Escape to a loader prompt" option from the system boot menu. At the prompt, enter boot kernel.old or the name of any other known-to-boot-correctly kernel. After successfully booting with a decent kernel, review the configuration file and attempt to create the kernel again. /var/log/messages is a useful resource that stores the kernel messages from each successful boot. Additionally, dmesg displays the kernel messages from the current boot. When debugging a kernel, be careful to preserve a copy of a known-to-work kernel, such as GENERIC. This is crucial because whenever a new kernel is installed, kernel.old gets replaced with the previous kernel, which may or may not be bootable. Move the functioning kernel immediately by renaming the directory holding the good kernel:

mv /boot/kernel /boot/kernel.notworking
mv /boot/kernel.good /boot/kernel

• The kernel works, but system commands do not: Many system status commands, such as ps and vmstat, do not function if the kernel version does not match that of the system utilities. For example, if a kernel built from -CURRENT sources is put on a -RELEASE system, many system status commands will not function. Recompile and install a world produced with the same version of the source tree as the kernel to resolve this issue. It is never advisable to use a kernel version that differs from the rest of the operating system.