Notes on OpenBSD

David J. Raymond

1 Introduction
2 Installation
3 Setting up your working environment
4 Organizing a cluster of systems
5 Notes on hardware compatibility
- 5.1 Graphics
- 5.2 Laptops

1 Introduction

OpenBSD is a derivative of the Berkeley UNIX operating system, developed at the University of California at Berkeley in the 1970s and 80s, which in turn was derived from the original UNIX operating system developed at Bell Labs. BSD thus stands for “Berkeley Software Distribution”. BSD UNIX forms the basis for many commercial UNIX distributions, including Darwin, which is the basis for Apple’s MacOS. In addition, BSD UNIX lives on in a number of free software forks, including FreeBSD, NetBSD, DragonFlyBSD, as well as OpenBSD. OpenBSD was forked from NetBSD in 1995 by Theo de Raadt, who lives in Calgary, Canada and is still in charge of the project.

According to the OpenBSD website, “Our efforts emphasize portability, standardization, correctness, proactive security and integrated cryptography.” OpenBSD is a viable alternative to Linux, with a highly secure core and a large selection of packages.

2 Installation

2.1 Basics

The OpenBSD installation guide is a good place to start. Once the install file is loaded on a USB drive, the actual installation procedure is mostly self-explanatory, with default options generally working fine. Here are a few pointers:

Be sure your system clock is set to UTC time and date before starting the install.
Unless you are using a language other than English, the default keyboard is appropriate.
If your Internet is via wireless, you probably will have to do the install without an Internet connection, which means you should use the install file with “sets”.
If your local network has DHCP, network setup is trivial. If not, come prepared with a network name, a network domain, your computer’s IP address, as well as the IP addresses for your Internet gateway and DNS servers.
Starting sshd is recommended. Starting xenodm during install is not recommended if you are not sure that your graphics card is supported. Testing after install is safest.
There are pros and cons to allowing password access to root. If you expect to need remote access to root without passing through a user account, then you may need to allow remote password access. In any case, root should have a strong password.
Setting up disks can be tricky if you have more than one disk in your system. Be sure that you know which will be the system disk. More on disk setup below.
My experience suggests using whole disk MBA formatting rather than whole disk GPT, but your mileage may vary. Some experimentation may be needed here as different BIOSs operate differently.
The default partitioning of the system disk is generally sufficient. Often not all of the disk is used. One option is to extend the /home directory to the end of the disk – see below.
The “sets” contain elements of the core system. It is recommended that all of the sets be installed. It is easiest to do this from the Internet if that is available. Otherwise they can be installed from install USB drive that contains them. If you select this option, the install program will ask you if the sets are mounted. Say no and follow instructions.
After rebooting into your new system, run “syspatch” as root. This patches the system for updates. You may need to reboot afterwards. Follow the instructions. Also run “fw_update” to install firmware needed by the CPU that is not included in the install medium. Rebooting afterwards is also required for this to to take effect.

2.2 Adding users

As root, run “adduser” and follow instructions. Classify everyone as “staff” in the login class. Users are by default given membership in the group defined by their login name. A user can be added to another group by using the “usermod” program. For instance,

     usermod -G newgroup username

Note that

Users can only become superusers using “su” if they have been added to the “wheel” group. However, ordinary users who are members of the “staff” group can be given permission to execute a defined set of privileged commands using the “doas” program. Doas is controlled by the “/etc/doas.conf” file. In the following doas.conf file

permit persist keepenv raymond
permit persist keepenv sessions
permit :staff cmd reboot

the users “raymond” and “sessions” can execute any privileged command upon entering their user password. Subsequent commands can be entered without a password for a short period of time. On the other hand, any member of the “staff” group can run the “reboot” command.

Users are removed using the “rmuser username” command.

2.3 Local configuration

After installing the core system, one should run “ssh-keygen” from root if communication with other computers via ssh is needed at the root level. For password access, just hit return for each question. This can also be useful at the user level.
On Linux /usr/local is available for locally generated material. On OpenBSD this is used for the installation of non-core packages. One alternative is to create install directories bin, etc, include, lib, man, and share owned by root in /home/root.

Resource limits are set to very conservative values on OpenBSD. These can be too small. I set the “staff” entry in “/etc/login.conf” to the following:

#
# Staff have fewer restrictions and can login even when nologins are set.
#
staff:\
 :datasize-cur=3072M:\
 :datasize-max=infinity:\
 :maxproc-max=512:\
 :maxproc-cur=256:\
 :stacksize-cur=32M:\
 :ignorenologin:\
 :requirehome@:\
 :tc=default:

This increases “datasize-cur” and “stacksize-cur” compared to the default.

Native printing on OpenBSD is Berkeley lpr. Non-postscript and non-ascii printing is difficult, so it is best for work with postscript-capable printers. Cups may be a simpler alternative and can be installed as a package.
Non-system disks can be initialized and mounted as demonstrated below.

2.4 Disk formatting

Disk formatting done during the above install process is done automatically. However this can be carried out after install as well. Setting up a disk on OpenBSD is a four step process:

The first step is to create hardware-level partitions using fdisk. For a disk named “sd1”, run “fdisk -g sd1”. This creates an OpenBSD partition on the disk. (“fdisk -i sd1” also works, but only for 2 TB and smaller disks.) To determine the state of the disk, just type “fdisk sd1”. For more information, type “fdisk -v sd1”. A variety of other options are available on the fdisk man page.
The next step is to create logical partitions on the physical OpenBSD partition. Run “disklabel -E sd1”, which starts a simple command line editor. Typing “?” to the prompt lists possible instructions. Logical partitions are indicated by a single letter in the range “a-p”. The “c” partition covers the full physical partition and is created by fdisk. It cannot be changed by disklabel. On a boot disk, “a” is root and “b” is swap. The default root disk partitioning uses partitions “a-k”. For a non-boot disk, one is free to partition it any desired way with the exception that “c” is untouchable. To investigate the state of logical partitions, on sd1, just type “disklabel sd1”. The various logical partitions are referred to as sd1a, sd1b, etc.
Step 3 is to create file systems on the new logical partitions. (This assumes that none will be used for swap space.) To create a Fast File System 2 (OpenBSD standard) on sd1a, run “newfs sd1a”. Be careful; if you do this unintentially to a partition already in use, it will ruin your whole day! (Technically, there are two different versions of the Fast File System, v1 and v2. The former has a maximum size of 2 TB while the latter maximum size is effectively infinite. Newfs uses v2 by default.)
Mount your partition on some directory that you have created, e.g., “mount sd1a /mydirectory”. You will want to add this mount instruction to your “/etc/fstab” file, as discussed below.
A safer way to mount disks in fstab is via a “DUID”, which is a random number that is assocated with a particular disk by the system. The DUID for a disk can be found in a number of ways. Perhaps the easiest is to rerun disklabel, e.g., “disklabel sd1” and cut and paste the DUID into your fstab file. The existing system disk entry serves as a pattern as to how to do this.
The mount point you created earlier should be the second field in the fstab entry for the disk in question.

2.5 Package management

OpenBSD has a simple but elegant package management system for installing your favorite software.

Run “pkg_info -Q name” to find out if a package containing that name exists. For instance, “pkg_info -Q emacs” will list all the possible emacs packages available on the system.
Run “pkg_add pkgx” to install pkgx. (Must be done as root.)
If you don’t like package pkgy, you can uninstall it with “pkg_delete pkgy”. (Also root.)
Deleting a package may leave behind other packages that are needed by the package your are deleting. To get rid of those as well, run “pkg_delete -a”. If one of these dependent packages is needed for another package that you have explicitly installed, it will not be deleted.
Updating packages should be done periodically with “pkg_add -u”. This takes a while as all installed packages have to be examined for an updated version.
If you would like to pass along a list of your installed packages to a friend, run “pkg_info -mz > mypackagelist” and send the list to your friend. He/she can install these by running “pkg_add -l mypackagelist”.

2.6 Networking

Sometimes you may need to change your default networking setup that was established during the initial install. The program “ifconfig” is your friend here. Just typing “ifconfig” gives a list of available network interfaces on your system and their status. It has many options to control networking.

At a higher level, the shell script “/etc/netstart” is normally used to start networking. It depends on the existence of files named “/etc/hostname.xxx” where “xxx” is the name of the desired network interface. For DHCP connections, the format of this file is simple. For example, a file named “/etc/hostname.iwm0”

-joinlist
join mylan wpakey mypasswd
dhcp
inet6 autoconf
up

sets up a wireless DHCP connection on the “iwm0” interface with SSID “mylan” WPA password “mypasswd”. If the network is open, the “wpakey” element is omitted. Multiple “join” lines can be added to this file and the system will try them in order until a successful connection is made. The networking is started by typing (as root) “sh /etc/netstart iwm0”.

For a manual connection on a wired network with interface “ure0”, the file “/etc/hostname.ure0” is simply

inet aaa.bbb.ccc.ddd 255.255.255.0

where the second element is the internet address assigned to you and the third is the netmask. “255.255.255.0” is the most common netmask you will encounter, but this might occasionally differ. For a manual connection, you will also need a “/etc/resolv.conv” file such as

nameserver aaa.bbb.ccc.ddd

which specifies the the internet address of the name server. Multiple servers may be listed on separate lines if they exist. You will also need to create the file “/etc/mygate” which contains a single line specifying the internet gateway. The network is started the same way as above, i.e., “sh /etc/netstart ure0”.

In order to avoid having to become root user to start networking, it is best to use the “doas” command. Once set up, networking restarts automatically after reboot. Read the “doas” webpage. Root access is needed to set up “doas”.

2.7 Upgrading to a new version of OpenBSD

Run “sysupgrade” and follow instructions.
Run “pkg_add -u” to get the packages that match the new OS version.

3 Organizing a cluster of systems

Our old Linux cluster used Network File System (NFS) to provide a central repository for home and other directories. This has two disadvantages: (1) NFS is slow in comparison to local disk access. (2) Dependence on an NFS server causes reliability issues, since everybody is down if the server has a problem. This makes users mad and system admins grouchy.

As disk is now very cheap, an alternative is to equip each computer with its own home directory. This also has disadvantages: (1) Unless backup provisions are made, failure of a disk can result in loss of one’s home directory. (2) Temporary use of another computer is difficult. The solution to both issues is to use a central server to back up one’s home directory periodically. Backup then becomes the user’s responsibility, which makes system admins less grouchy. If you lose your data due to a disk crash, guess who’s fault it is?!

For this to work, backups have to be easy and fast. The answer is “rsync”. This utility backs up your home directory to the server efficiently, copying over only those files that are modified. Rsync can also be used to add your home directory to a new system temporarily, though if your home directory is large, this can take a while.

Having your home directory on multiple computers (in addition to the central server) causes another problem. If incompatible changes are made on the local systems, backup by rsync to the central server becomes problematic, as syncing one system can step on changes synced earlier from the other system. Solution: Before moving from computer A to computer B, sync changes on A to the server before syncing from the server to B. Again, this is the user’s responsibility, which makes system admins happy. “You have been warned!”

3.1 Sysadmin tools

The control center for system administration is “/home/root/etc” on gryphon. A variety of shell scripts are available:

“clonepass” copies password and group information to other OpenBSD computers in the network. By this mechanism, everyone with an account on gryphon receives an account on the selected computer. However, home directories on that computer are not created. That is the function of...
“cloneuser”, which creates a home directory on the selected computer for the specified user only.
“cloneroot” copies “/home/root” to the selected computer.
“updateall.sh” runs clonepass, cloneroot, and syspatch on OpenBSD computers on our network (except gryphon). In addition, certain configuration files are distributed. Systems with “/home” mounted using NFS have cloneroot omitted, since this is already available from NFS. The script also reports if one of these systems is dead. Optional actions are commented out.
“ssh-build” enables passwordless access at the root level for all machines. This is needed for coordination at the root level to work.

In addition, /home/root/etc contains a few useful files:

“pkglist” is the current list of packages installed on our systems.
“skel” is a directory containing our version of files helpful to a new user.
“hosts” is a copy of the “/etc/hosts” file for our system.
“login.conf” is our modified version of “/etc/login.conf”, with increased resource limits.

3.2 Network file system (NFS)

An NFS server is set up on loro, following these instructions. The file systems “/home” and “/data.gy” are made available to other computers that need them. Currently these are our Beowulf cluster, the gateway computer “merlin”, and the loro backup, “gryphon”.

3.3 Beowulf setup

Machines “gy01” through “gy08” are the computational machines for our Beowulf cluster. Currently they consist of 8-core AMD Ryzen processors, giving a total of 64 cores. They are set up the same as other OpenBSD machines except that network file system (NFS) is used to make “/home” and “/data.gy” available to all machines. This means that users don’t have to configure anything on these machines, only on “loro”, which acts as the NFS server.

We use “openmpi” to provide the message passing interface for multi-processor programming. See the OpenMPI documentation. We use “openmpi-4.0.2” patched for use on OpenBSD. Currently, the official openmpi package doesn’t work properly. The patched source resides in “/home/src”. In order for openmpi to function on OpenBSD, you must set an environment variable: “export PMIX_MCA_gds=hash”.

3.4 Gateway

As noted above, merlin acts as a connection from our computer cluster to the Internet. This machine has two Ethernet connections, one to the Internet, the other to our main switch. It serves two functions, as a network address translation (NAT) device and as a local domain name server (DNS) inside our network.

NAT allows Internet packets to pass from our cluster to the Internet and changes their return address to appear that they are coming from merlin. Packets from the Internet can only reach internal machines in response to outgoing packets. All others are rejected. Thus, the Internet cannot “see” machines behind the gateway. OpenBSD’s packet filtering software on merlin does the trick. This is implemented by the file “/etc/pf.conf”:
```
# $OpenBSD: pf.conf,v 1.55 2017/12/03 20:40:04 sthen Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf

set skip on lo

block return # block stateless traffic
pass  # establish keep-state

# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010

# Port build user does not need network
block return out log proto {tcp udp} user _pbuild

# NAT for internal network
ext=re0
int=re1
pass out on $ext inet from $int:network to any nat-to ($ext)
```
The last 3 lines implement the NAT. The rest is default content. The line
```
net.inet.ip.forwarding=1
```
must be added to the “/etc/sysctl.conf” file as well.
DNS is provided by the "dnsmasq" package. The configuration file, "/etc/dnsmasq" has a lot of options, all of which are initially commented out. All that is really needed in this file is
```
interface=re1  # our internally facing Ethernet device
no-dhcp-interface=re1   # only provide DNS service
```

To enable and start the DNS service, which provides IP addresses both inside and outside the gateway, run (as root) on merlin “rcctl enable dnsmasq” and “rcctl start dnsmasq”. In order for this to work, A fake user with the name “_dnsmasq” must be created on merlin. Specify “daemon” user type and “nologin” for shell type.

Dnsmasq gets its information from the file “/etc/hosts”. This needs to be populated with a table connecting internet numbers with computer names. A sample host file is given in “/home/root/etc/hosts”. “hosts” files are distributed to all machines in our group, which makes internal ssh connections possible inside our cluster even if merlin is down.

Merlin (known as “me” inside our local network) has “loro:/home” mounted via NFS. Thus, one’s home directory on gryphon can be accessed directly on merlin from the outside network.

3.5 Powering up computers in the cluster

The order with which computers are powered up is important. First to be turned on is loro, since this provides NFS service to some of our computers. Second, merlin should be booted. This provides DNS and gateway services. The rest of the computers can be booted in any order. However, computers not served by NFS can be booted at any time, and do not have to be rebooted if loro or merlin are restarted.

4 Notes on hardware compatibility

The AMD64 version of OpenBSD generally works on all modern Intel/AMD hardware. However, there are specific issues with particular I/O hardware, graphics cards especially.

4.1 Graphics

Intel graphics is advertised as being generally supported. The same is said for Radeon graphics, though I have found some newer Radeon graphics cards to not work. Nvidia cards sometimes work, though support is weaker due to the unwillingness of Nvidia to release information on their cards.

Here is my personal experience:

Lenovo X1 Carbon Gens 1, 4, 5: Work out of the box.
Random cheap Radeon cards: Mostly work.
AMD Vega graphics on a Ryzen 4-core 2200g: Generally works, though there are some cursor and 3-D DRM issues. Turning on the software cursor and retreating to 2-D DRM mostly solves these issues (OpenBSD 6.6 stable and later). To do this install the following file with name “10-amdgpu.conf”
```
    Section "Device"
        Identifier "AMDgpu"
        Driver "amdgpu"
        Option "SWcursor" "true"
        Option "DRI" "2"
    EndSection
```
in “/etc/X11/xorg.conf.d”.

4.2 Laptops

This commentary is helpful for laptops. Of particular importance is power management. Running

rcctl enable apmd
rcctl set apmd flags -A
rcctl start apmd

as root sets this up.

Intel-based Lenovo laptops are reputed to generally work well with OpenBSD. AMD-based laptops should probably be avoided at this point. My experience with a Lenovo E-485 was particularly difficult even under Linux.

Fans running full speed even with an idle system is an occasional complaint with Lenovo laptops. Sometimes this can be alleviated by updating the BIOS. This should never be done lightly, as a mistake can brick your system! So, if you need to do this, do your Internet research first. A helpful blog for Thinkpads is here.

The laptop microphone is turned off by default for security. It may be turned on with the “sysctl kern.audio.record=1” command (as root). To make this persistent, add the line “kern.audio.record=1” to the “/etc/sysctl.conf” file. (You may have to create this file.) As a test, record your voice to a .wav file using “aucat -o testfile.wav” and listening to the recording with “aucat -i testfile.wav”. See the OpenBSD FAQ for details.

The laptop webcam can be accessed with the “/dev/video” device. By default, this has root-only permissions. Changing permissions as root as follows “chmod go+rw /dev/video” allows a ordinary user to access the webcam. Also, run “sysctl kern.video.record=1” or add to the “sysctl.conf” file as indicated above.

At least some applications (such as Google Hangouts and Zoom run on a web browser) know how to access the webcam and the microphone once proper permissions are set up. To run Zoom on chromium or firefox, set “ENABLE_WASM=1” before starting the browser. When responding to a Zoom invitation, select “run in browser”.

This document was translated from L^AT_EX by H^EV^EA.