Notes on OpenBSD

Contents

• 1  Introduction
• 2  Installation
  • 2.1  Basics
  • 2.2  Adding users
  • 2.3  Local configuration
  • 2.4  Disk formatting
  • 2.5  Package management
  • 2.6  Networking
  • 2.7  Upgrading to a new version of OpenBSD
• 3  Setting up your working environment
• 4  Organizing a cluster of systems
  • 4.1  Sysadmin tools
  • 4.2  Network file system (NFS)
  • 4.3  Beowulf setup
  • 4.4  Gateway
  • 4.5  Powering up computers in the cluster
• 5  Notes on hardware compatibility
  • 5.1  Graphics
  • 5.2  Laptops

1  Introduction

OpenBSD is a derivative of the Berkeley UNIX operating system, developed at the University of California at Berkeley in the 1970s and 80s, which in turn was derived from the original UNIX operating system developed at Bell Labs. BSD thus stands for “Berkeley Software Distribution”. BSD UNIX forms the basis for many commercial UNIX distributions, including Darwin, which is the basis for Apple’s MacOS. In addition, BSD UNIX lives on in a number of free software forks, including FreeBSD, NetBSD, DragonFlyBSD, as well as OpenBSD. OpenBSD was forked from NetBSD in 1995 by Theo de Raadt, who lives in Calgary, Canada and is still in charge of the project.

According to the OpenBSD website, “Our efforts emphasize portability, standardization, correctness, proactive security and integrated cryptography.” OpenBSD is a viable alternative to Linux, with a highly secure core and a large selection of packages.

2  Installation

2.1  Basics

The OpenBSD installation guide is a good place to start. Once the install file is loaded on a USB drive, the actual installation procedure is mostly self-explanatory, with default options generally working fine. Here are a few pointers:

• Be sure your system clock is set to UTC time and date before starting the install.
• Unless you are using a language other than English, the default keyboard is appropriate.
• If your Internet is via wireless, you probably will have to do the install without an Internet connection, which means you should use the install file with “sets”.
• If your local network has DHCP, network setup is trivial. If not, come prepared with a network name, a network domain, your computer’s IP address, as well as the IP addresses for your Internet gateway and DNS servers.
• Starting sshd is recommended. Starting xenodm during install is not recommended if you are not sure that your graphics card is supported. Testing after install is safest.
• There are pros and cons to allowing password access to root. If you expect to need remote access to root without passing through a user account, then you may need to allow remote password access. In any case, root should have a strong password.
• Setting up disks can be tricky if you have more than one disk in your system. Be sure that you know which will be the system disk. More on disk setup below.
• My experience suggests using whole disk MBA formatting rather than whole disk GPT, but your mileage may vary. Some experimentation may be needed here as different BIOSs operate differently.
• The default partitioning of the system disk is generally sufficient. Often not all of the disk is used. One option is to extend the /home directory to the end of the disk – see below.
• The “sets” contain elements of the core system. It is recommended that all of the sets be installed. It is easiest to do this from the Internet if that is available. Otherwise they can be installed from install USB drive that contains them. If you select this option, the install program will ask you if the sets are mounted. Say no and follow instructions.
• After rebooting into your new system, run “syspatch” as root. This patches the system for updates. You may need to reboot afterwards. Follow the instructions. Also run “fw_update” to install firmware needed by the CPU that is not included in the install medium. Rebooting afterwards is also required for this to to take effect.

2.2  Adding users

As root, run “adduser” and follow instructions. Classify everyone as “staff” in the login class. Users are by default given membership in the group defined by their login name. A user can be added to another group by using the “usermod” program. For instance,

     usermod -G newgroup username

Note that

Users can only become superusers using “su” if they have been added to the “wheel” group. However, ordinary users who are members of the “staff” group can be given permission to execute a defined set of privileged commands using the “doas” program. Doas is controlled by the “/etc/doas.conf” file. In the following doas.conf file

permit persist keepenv raymond
permit persist keepenv sessions
permit :staff cmd reboot

the users “raymond” and “sessions” can execute any privileged command upon entering their user password. Subsequent commands can be entered without a password for a short period of time. On the other hand, any member of the “staff” group can run the “reboot” command.

Users are removed using the “rmuser username” command.

2.3  Local configuration

• After installing the core system, one should run “ssh-keygen” from root if communication with other computers via ssh is needed at the root level. For password access, just hit return for each question. This can also be useful at the user level.
• On Linux /usr/local is available for locally generated material. On OpenBSD this is used for the installation of non-core packages. One alternative is to create install directories bin, etc, include, lib, man, and share owned by root in /home/root.
• Resource limits are set to very conservative values on OpenBSD. These can be too small. I set the “staff” entry in “/etc/login.conf” to the following:

  #
  # Staff have fewer restrictions and can login even when nologins are set.
  #
  staff:\
   :datasize-cur=3072M:\
   :datasize-max=infinity:\
   :maxproc-max=512:\
   :maxproc-cur=256:\
   :stacksize-cur=32M:\
   :ignorenologin:\
   :requirehome@:\
   :tc=default:
  

  This increases “datasize-cur” and “stacksize-cur” compared to the default.
• You may want to add the mount option “softdep” to the entries in your /etc/fstab file. This speeds write operations and reduces the number of disk accesses. See the “mount” man page.
• Native printing on OpenBSD is Berkeley lpr. Cups can be installed as a package. To avoid conflict between cups and lpr commands, put “/usr/local/bin” before “/usr/bin” in your PATH environment variable. A sample “printers.conf” file is normally kept in “/home/root/etc”. After installing this, enable and start cupsd using rcctl.
• Non-system disks can now be initialized and mounted as demonstrated below.

2.4  Disk formatting

Disk formatting done during the above install process is done automatically. However this can be carried out after install as well. Setting up a disk on OpenBSD is a four step process:

• The first step is to create hardware-level partitions using fdisk. For a disk named “sd1”, run “fdisk -i sd1”. This creates an OpenBSD partition on the disk. To determine the state of the disk, just type “fdisk sd1”. For more information, type “fdisk -v sd1”. A variety of other options are available on the fdisk man page.
• The next step is to create logical partitions on the physical OpenBSD partition. Run “disklabel -E sd1”, which starts a simple command line editor. Typing “?” to the prompt lists possible instructions. Logical partitions are indicated by a single letter in the range “a-p”. The “c” partition covers the full physical partition and is created by fdisk. It cannot be changed by disklabel. On a boot disk, “a” is root and “b” is swap. The default root disk partitioning uses partitions “a-k”. For a non-boot disk, one is free to partition it any desired way with the exception that “c” is untouchable. To investigate the state of logical partitions, on sd1, just type “disklabel sd1”. The various logical partitions are referred to as sd1a, sd1b, etc.
• Step 3 is to create file systems on the new logical partitions. (This assumes that none will be used for swap space.) To create a Fast File System (OpenBSD standard) on sd1a, run “newfs sd1a”. Be careful; if you do this unintentially to a partition already in use, it will ruin your whole day!
• Mount your partition on some directory that you have created, e.g., “mount sd1a /mydirectory”. You will want to add this mount instruction to your “/etc/fstab” file, as discussed below.
• A safer way to mount disks in fstab is via a “DUID”, which is a random number that is assocated with a particular disk by the system. The DUID for a disk can be found in a number of ways. Perhaps the easiest is to rerun disklabel, e.g., “disklabel sd1” and cut and paste the DUID into your fstab file. The existing system disk entry serves as a pattern as to how to do this.
• The mount point you created earlier should be the second field in the fstab entry for the disk in question.

2.5  Package management

OpenBSD has a simple but elegant package management system for installing your favorite software.

• Run “pkg_info -Q name” to find out if a package containing that name exists. For instance, “pkg_info -Q emacs” will list all the possible emacs packages available on the system.
• Run “pkg_add pkgx” to install pkgx. (Must be done as root.)
• If you don’t like package pkgy, you can uninstall it with “pkg_delete pkgy”. (Also root.)
• Deleting a package may leave behind other packages that are needed by the package your are deleting. To get rid of those as well, run “pkg_delete -a”. If one of these dependent packages is needed for another package that you have explicitly installed, it will not be deleted.
• Updating packages should be done periodically with “pkg_add -u”. This takes a while as all installed packages have to be examined for an updated version.
• If you would like to pass along a list of your installed packages to a friend, run “pkg_info -mz > mypackagelist” and send the list to your friend. He/she can install these by running “pkg_add -l mypackagelist”.

2.6  Networking

Sometimes you may need to change your default networking setup that was established during the initial install. The program “ifconfig” is your friend here. Just typing “ifconfig” gives a list of available network interfaces on your system and their status. It has many options to control networking.

At a higher level, the shell script “/etc/netstart” is normally used to start networking. It depends on the existence of files named “/etc/hostname.xxx” where “xxx” is the name of the desired network interface. For DHCP connections, the format of this file is simple. For example, a file named “/etc/hostname.iwm0”

-joinlist
join mylan wpakey mypasswd
dhcp
inet6 autoconf
up

sets up a wireless DHCP connection on the “iwm0” interface with SSID “mylan” WPA password “mypasswd”. If the network is open, the “wpakey” element is omitted. Multiple “join” lines can be added to this file and the system will try them in order until a successful connection is made. The networking is started by typing (as root) “sh /etc/netstart iwm0”.

For a manual connection on a wired network with interface “ure0”, the file “/etc/hostname.ure0” is simply

inet aaa.bbb.ccc.ddd 255.255.255.0

where the second element is the internet address assigned to you and the third is the netmask. “255.255.255.0” is the most common netmask you will encounter, but this might occasionally differ. For a manual connection, you will also need a “/etc/resolv.conv” file such as

nameserver aaa.bbb.ccc.ddd

which specifies the the internet address of the name server. Multiple servers may be listed on separate lines if they exist. You will also need to create the file “/etc/mygate” which contains a single line specifying the internet gateway. The network is started the same way as above, i.e., “sh /etc/netstart ure0”.

In order to avoid having to become root user to start networking, it is best to use the “doas” command. Once set up, networking restarts automatically after reboot.

2.7  Upgrading to a new version of OpenBSD

• Run “sysupgrade” and follow instructions.
• Run “pkg_add -u” to get the packages that match the new OS version.

3  Setting up your working environment

If you have an account on gryphon, you can login to any machine in our cluster. However, unless gryphon:/home is NFS mounted (gy0*, merlin), you will have to download your home directory from gryphon, as described below.

When you first login to a new account (under X11) on a machine, the ice window manager desktop will be started. The toolbar allows you to begin your work, as it contains buttons for a file manager, a terminal emulator, and a web browser.

Probably your first action should be to retrieve the contents of your home directory from loro, the old Linux server. Note that this may have been done for you by your friendly local sysadmin. If not, login to gryphon using ssh and run “getloro”. This will copy your home directory from loro to gryphon. Note that most “dot” files (those with filename beginning with a period) will not be copied, since they are often dependent on the operating system. If needed they can later be copied by hand using scp or rsync directly. Also, compiled programs will have to be recompiled, as stuff compiled under Linux will not run under OpenBSD and vice versa. The “getloro” shell script looks like this:

#!/bin/sh
#
# getloro -- Synchronize my home directory on loro to local machine
# from server This excludes most "dot" files, since these are likely
# to cause trouble on openbsd.
#
server=loro
cd

# grab some important OpenBSD files
rsync -av gryphon:/home/root/etc/skel/ .

# transfer mostly non-dot files from loro
# nice to have .bashrc for reverse transfers even though not used on OpenBSD
rsync -avxl ${server}:/home/${USER}/* ${server}:/home/${USER}/.bashrc \
      /home/${USER}

You can also send your OpenBSD files (minus dot files) to loro:

#!/bin/sh
#
# putloro -- Synchronize my home directory on gryphon to loro.  This
# excludes "dot" files, since these are likely to cause trouble on
# linux.  Delete is used to keep loro directories from growing.
#
server=loro
cd

# transfer non-dot files to loro
rsync -avxl --delete --exclude='.*' /home/${USER}/ ${server}:/home/${USER}

Non-dot files that don’t exist on OpenBSD are deleted. Dot files are preserved.

Next, on your local machine, run “gethome” (see below) to copy your home directory from gryphon. If your home directory is big, this will take a while. You will get a running commentary on files being loaded.

The toolbar elements in icewm may be modified by editing “.icewm/toolbar”. The format is obvious. The “preferences” file in this directory can also be edited for many options. Changes to the desktop come into effect when the ”Icewm” button in the main menu on the left is pressed. Alternatively, logout and login again.

You can also change the desktop to your tastes by editing the “.xsession” file in your main directory. The desktop options are listed at the bottom of this file with the currently activated option uncommented. To change, uncomment your choice and comment out the currently active entry. Then logout and login again. Current choices are

• I3: A tiling window manager that some people love, but is an acquired taste.
• Openbox: On startup, you are presented with a gloriously blank screen. The right mouse button lists a menu of available programs. The middle mouse button shows windows available in each desktop. The menu may be modified by editing “.config/openbox/menu.xml.
• Icewm: The default choice. Easy to use, modifications require a bit of editing skill.
• Mate: A full-featured desktop that is easy to configure and use.

The other entries in .xsession set up your working environment. The “.kshrc” file determines the behavior of ksh, which is the default shell for OpenBSD.

Two shell scripts make syncing home directories easy. “gethome” synchronizes the home directory on your workstation with that on the server:

#!/bin/sh
#
# gethome -- Synchronize my home directory to local machine from server
#
server=gryphon
cd
rsync -avxl --delete \
      --exclude=".ssh" \
      --exclude=".cache" \
      ${server}:/home/${USER}/ /home/${USER}

The reverse synchonization from the workstation to the server is called “puthome”:

#!/bin/sh
#
# puthome -- Synchronize my home directory to server from local machine
#
server=gryphon
cd
rsync -avxl --delete \
      --exclude=".ssh" \
      --exclude=".cache" \
      /home/${USER}/ ${server}:/home/${USER}

Files in “.ssh” are not transferred, as these can be specific to a machine. Neither is “.cache”, as this is generally very large with lots of web browser junk which gets regenerated automatically.

Sometimes it is useful to be able to log into your account on another computer without having to type a password. The script “ssh_trust” does this:

#! /bin/sh
#
# ssh_trust trusted_host -- Grab the identity.pub file on a host we trust
#                       and put it in our authorized_keys file so that
#                       passwordless login can occur using ssh from that
#                       host.
#
if test $# != 1
then
    echo ssh_trust trusted_host
else
    cd
    cd .ssh
    scp $1:.ssh/id_rsa.pub junk
    cat junk >> authorized_keys
    rm junk
fi

To set up passwordless login from machine A to machine B, login to B (using your password) and type “ssh_trust A”. You will possibly need your password on A for this to work. This can be automated for all machines using the “ssh-userbuild” script. This gives you passwordless access as a user to all machines. Run it and follow instructions. Depending on your preferences, you may or may not want to do this.

All of these scripts are available in the standard path.

4  Organizing a cluster of systems

Our old Linux cluster used Network File System (NFS) to provide a central repository for home and other directories. This has two disadvantages: (1) NFS is slow in comparison to local disk access. (2) Dependence on an NFS server causes reliability issues, since everybody is down if the server has a problem. This makes users mad and system admins grouchy.

As disk is now very cheap, an alternative is to equip each computer with its own home directory. This also has disadvantages: (1) Unless backup provisions are made, failure of a disk can result in loss of one’s home directory. (2) Temporary use of another computer is difficult. The solution to both issues is to use a central server to back up one’s home directory periodically. Backup then becomes the user’s responsibility, which makes system admins less grouchy. If you lose your data due to a disk crash, guess who’s fault it is?!

For this to work, backups have to be easy and fast. The answer is “rsync”. This utility backs up your home directory to the server efficiently, copying over only those files that are modified. Rsync can also be used to add your home directory to a new system temporarily, though if your home directory is large, this can take a while.

Having your home directory on multiple computers (in addition to the central server) causes another problem. If incompatible changes are made on the local systems, backup by rsync to the central server becomes problematic, as syncing one system can step on changes synced earlier from the other system. Solution: Before moving from computer A to computer B, sync changes on A to the server before syncing from the server to B. Again, this is the user’s responsibility, which makes system admins happy. “You have been warned!”

4.1  Sysadmin tools

The control center for system administration is “/home/root/etc” on gryphon. A variety of shell scripts are available:

• “clonepass” copies password and group information to other OpenBSD computers in the network. By this mechanism, everyone with an account on gryphon receives an account on the selected computer. However, home directories on that computer are not created. That is the function of...
• “cloneuser”, which creates a home directory on the selected computer for the specified user only.
• “cloneroot” copies “/home/root” to the selected computer.
• “updateall.sh” runs clonepass, cloneroot, and syspatch on OpenBSD computers on our network (except gryphon). In addition, certain configuration files are distributed. Systems with “/home” mounted using NFS have cloneroot omitted, since this is already available from NFS. The script also reports if one of these systems is dead. Optional actions are commented out.
• “ssh-build” enables passwordless access at the root level for all machines. This is needed for coordination at the root level to work.

In addition, /home/root/etc contains a few useful files:

• “pkglist” is the current list of packages installed on our systems.
• “printers.conf” contains printer information for use by the cups printing system. It should be copied to “/etc/cups” when cupsd is not running.
• “skel” is a directory containing our version of files helpful to a new user.
• “hosts” is a copy of the “/etc/hosts” file for our system.
• “login.conf” is our modified version of “/etc/login.conf”, with increased resource limits.

4.2  Network file system (NFS)

An NFS server is set up on gryphon, following these instructions. The file systems “/home” and “/data.gy” are made available to other computers that need them. Currently these are our Beowulf cluster and the gateway computer “merlin”.

4.3  Beowulf setup

Machines “gy01” through “gy08” are the computational machines for our Beowulf cluster. Currently they consist of 8-core AMD Ryzen processors, giving a total of 64 cores. They are set up the same as other OpenBSD machines except that network file system (NFS) is used to make “/home” and “/data.gy” available to all machines. This means that users don’t have to configure anything on these machines, only on “gryphon”, which acts as the NFS server.

We use “openmpi” to provide the message passing interface for multi-processor programming. See the OpenMPI documentation. We use “openmpi-4.0.2” patched for use on OpenBSD. Currently, the official openmpi package doesn’t work properly. The patched source resides in “/home/src”. In order for openmpi to function on OpenBSD, you must set an environment variable: “export PMIX_MCA_gds=hash”.

4.4  Gateway

As noted above, merlin acts as a connection from our computer cluster to the Internet. This machine has two Ethernet connections, one to the Internet, the other to our main switch. It serves two functions, as a network address translation (NAT) device and as a local domain name server (DNS) inside our network.

• NAT allows Internet packets to pass from our cluster to the Internet and changes their return address to appear that they are coming from merlin. Packets from the Internet can only reach internal machines in response to outgoing packets. All others are rejected. Thus, the Internet cannot “see” machines behind the gateway. OpenBSD’s packet filtering software on merlin does the trick. This is implemented by the file “/etc/pf.conf”:

  # $OpenBSD: pf.conf,v 1.55 2017/12/03 20:40:04 sthen Exp $
  #
  # See pf.conf(5) and /etc/examples/pf.conf
  
  set skip on lo
  
  block return # block stateless traffic
  pass  # establish keep-state
  
  # By default, do not permit remote connections to X11
  block return in on ! lo0 proto tcp to port 6000:6010
  
  # Port build user does not need network
  block return out log proto {tcp udp} user _pbuild
  
  # NAT for internal network
  ext=re0
  int=re1
  pass out on $ext inet from $int:network to any nat-to ($ext)
  

  The last 3 lines implement the NAT. The rest is default content. The line

  net.inet.ip.forwarding=1
  

  must be added to the “/etc/sysctl.conf” file as well.
• DNS is provided by the "dnsmasq" package. The configuration file, "/etc/dnsmasq" has a lot of options, all of which are initially commented out. All that is really needed in this file is

  interface=re1  # our internally facing Ethernet device
  no-dhcp-interface=re1   # only provide DNS service
  

To enable and start the DNS service, which provides IP addresses both inside and outside the gateway, run (as root) on merlin “rcctl enable dnsmasq” and “rcctl start dnsmasq”. In order for this to work, A fake user with the name “_dnsmasq” must be created on merlin. Specify “daemon” user type and “nologin” for shell type.

Dnsmasq gets its information from the file “/etc/hosts”. This needs to be populated with a table connecting internet numbers with computer names. A sample host file is given in “/home/root/etc/hosts”. “hosts” files are distributed to all machines in our group, which makes internal ssh connections possible inside our cluster even if merlin is down.

Merlin (known as “me” inside our local network) has “gryphon:/home” mounted via NFS. Thus, one’s home directory on gryphon can be accessed directly on merlin from the outside network.

4.5  Powering up computers in the cluster

The order with which computers are powered up is important. First to be turned on is gryphon, since this provides NFS service to some of our computers. Second, merlin should be booted. This provides DNS and gateway services. The rest of the computers can be booted in any order. However, computers not served by NFS can be booted at any time, and do not have to be rebooted if gryphon or merlin are restarted.

5  Notes on hardware compatibility

The AMD64 version of OpenBSD generally works on all modern Intel/AMD hardware. However, there are specific issues with particular I/O hardware, graphics cards especially.

5.1  Graphics

Intel graphics is advertised as being generally supported. The same is said for Radeon graphics, though I have found some newer Radeon graphics cards to not work. Nvidia cards sometimes work, though support is weaker due to the unwillingness of Nvidia to release information on their cards.

Here is my personal experience:

• Lenovo X1 Carbon Gen-4: Works out of the box.
• Random cheap Radeon cards: Mostly work.
• AMD Vega graphics on a Ryzen 4-core 2200g: Generally works, though there are some cursor and 3-D DRM issues. Turning on the software cursor and retreating to 2-D DRM mostly solves these issues (OpenBSD 6.6 stable). To do this install the following file with name “10-amdgpu.conf”

      Section "Device"
          Identifier "AMDgpu"
          Driver "amdgpu"
          Option "SWcursor" "true"
          Option "DRI" "2"
      EndSection
  

  in “/etc/X11/xorg.conf.d”.

5.2  Laptops

This commentary is helpful for laptops. Of particular importance is power management. Running

rcctl enable apmd
rcctl set apmd flags -A
rcctl start apmd

as root sets this up.

Intel-based Lenovo laptops are reputed to generally work well with OpenBSD. AMD-based laptops should probably be avoided at this point. My experience with a Lenovo E-485 was particularly difficult even under Linux.

Fans running full speed even with an idle system is an occasional complaint with Lenovo laptops. Sometimes this can be alleviated by updating the BIOS. This should never be done lightly, as a mistake can brick your system! So, if you need to do this, do your Internet research first. A helpful blog for Thinkpads is here.

The laptop microphone is turned off by default for security. It may be turned on with the “sysctl kern.audio.record=1” command (as root). To make this persistent, add the line “kern.audio.record=1” to the “/etc/sysctl” file. (You may have to create this file.) As a test, record your voice to a .wav file using “aucat -o testfile.wav” and listening to the recording with “aucat -i testfile.wav”. See the OpenBSD FAQ for details.

The laptop webcam can be accessed with the “/dev/video” device. By default, this has root-only permissions. Changing permissions or using “doas” allows a ordinary user to access the webcam. At least some applications (such as Google Hangouts run on a web browser) know how to access the webcam and the microphone once proper permissions are set up.

This document was translated from L^AT_EX by H^EV^EA.