While the need for a realtime kernel or kernel-rt hasn’t been very high recently, there are these special cases where you really need one. A “standard” linux kernel is fast but also balanced in order to treat all workload fair and give each kind of task an equal share of the system resources. A realtime kernel allows you to squeeze that extra bit of performance out of the kernel. It basically allows you to tune it in a more aggressive way in regards to timing and priorities. Since I couldn’t find any clear instructions on how to get a realtime kernel for CentOS 7 and RHEL 7, I decided to write them myself.

The main difference between a “standard” linux kernel and a realtime kernel is actually quite simple. The “standard” linux kernel will not stop processing a running or queued task when a task with a higher priority comes up. A relatime kernel will be able to do that in order to reach certain deadlines and performance. This behaviour is called being preemtive. The rest of the differences can be found in tuning possibilities regarding the preemtive behaviour and specific priorities or timings.

More information about realtime Linux kernels can be found here: https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions

In most cases, you don’t really need a realtime kernel. Even if you suffer from some performance issues for very specific cases, in 99% of those cases, a workaround is a better idea and is available. Most programming languages allow you to overcome performance issues and allow you to tune specific delays in a very granular way.

Sometimes, you can’t influence a specific program or it’s behavior or the delay has already been reduced to it’s minimum and isn’t satisfying yet. In those cases, you can consider using a realtime kernel. Most standard distributions do not provide a realtime kernel and it most certainly won’t be supported. This is also the case for Redhat Linux or CentOS.

The process of getting a realtime isn’t very hard. Basically you need to follow this flow:

• Download the kernel source (preferably of a version which is close to the one from your distribution and which has a realtime-patch available)
• Download a realtime patch for the source you’ve downloaded
• Extract the kernel source and patch it
• Copy or create a .config for the downloaded kernel
• Compile and install the kernel
• Boot the realtime kernel and exploit it

Lists as I just gave are easy to write but let’s look how it’s done in practice.

Prerequisites

Before we can start compiling the custom kernel, we need to make sure that we’ve got all necessary tools installed on our system.

[jensd@cen ~]$ sudo yum groupinstall "Development Tools" -y
...
Complete !
[jensd@cen ~]$ sudo yum install hmaccalc zlib-devel binutils-devel elfutils-libelf-devel ncurses-devel bc wget -y
...
Complete !

Besides installing the necessary tools, make sure that you have enough free space on the file system where you plan to compile the kernel. I recommend a minimum of 8GB free space to comfortably compile the kernel.

Download the kernel source and realtime patch

The easiest way to get a realtime kernel on a specific system is to search for the existence of a kernel patch for a kernel version as close to a version which is already installed. Using a different version usually shouldn’t be a problem but the further you go from an installed version, the more questions you will get about the contents of .config.

The .config file contains all parameters for the kernel-compilation, for example which drivers to include or which file systems to support. Every new kernel version introduces (and removes) some of these settings.

First, let’s check which kernel we’re running a this time:

[jensd@cen ~]$ uname -r
3.10.0-229.el7.x86_64

As you can see, the system used for this post is running on kernel version 3.10.0. The .config file which was used to compile that kernel should be available in /boot:

[jensd@cen ~]$ ls /boot/config-*
/boot/config-3.10.0-229.el7.x86_64

We’ll use this file as a base for our new .config-file.

On https://www.kernel.org/pub/linux/kernel/projects/rt, you can see the list of realtime patches available for specific kernel versions. I can see that for kernel version 3.10, the following was available at the time of writing:

So I chose to use the patch for version 3.10.75 and downloaded the file patch-3.10.75-rt80.patch.gz:

[jensd@cen ~]$ wget https://www.kernel.org/pub/linux/kernel/projects/rt/3.10/patch-3.10.75-rt80.patch.gz
...
2015-06-01 18:09:16 (51.2 KB/s) - ‘patch-3.10.75-rt80.patch.gz’ saved [203541/203541]

Since we downloaded the patch for kernel 3.10.75, we’ll also need to download the kernel source for this version from: https://www.kernel.org/pub/linux/kernel/v3.x/.

[jensd@cen ~]$ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.75.tar.gz
--2015-06-01 12:11:47--  https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.75.tar.gz
Resolving www.kernel.org (www.kernel.org)... 149.20.4.69, 199.204.44.194, 198.145.20.140, ...
Connecting to www.kernel.org (www.kernel.org)|149.20.4.69|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 110473449 (105M) [application/x-gzip]
Saving to: ‘linux-3.10.75.tar.gz’

100%[=================================================>] 110,473,449  318KB/s   in 8m 28s

2015-06-01 18:20:16 (212 KB/s) - ‘linux-3.10.75.tar.gz’ saved [110473449/110473449]

[jensd@cen ~]$ ls -l
total 108088
-rw-rw-r--. 1 jensd jensd 110473449 Apr 19 10:20 linux-3.10.75.tar.gz
-rw-rw-r--. 1 jensd jensd    203541 Apr 22 16:24 patch-3.10.75-rt80.patch.gz

Extract and patch the kernel source

Now that we downloaded the source of the kernel and it’s matching patch to allow realtime, it’s time to extract the source and patch the kernel source.

[jensd@cen ~]$ tar -xvzf linux-3.10.75.tar.gz
...
[jensd@cen ~]$ cd linux-3.10.75
[jensd@cen linux-3.10.75]$ zcat ../patch-3.10.75-rt80.patch.gz | patch -p1

Configure the kernel in .config

The last step before we can start compiling the kernel is to create a .config-file valid for the kernel source which we downloaded. This is the point where it saves you some time if you found a kernel and patch which is close to an already installed kernel version. The easiest is to copy the .config-file of that installed kernel and re-use it as a base for the new kernel which we downloaded. The closer the versions, the less questions you’ll need to answer.

Copy the existing .config to the directory of the kernel source:

[jensd@cen linux-3.10.75]$ cp /boot/config-3.10.0-229.el7.x86_64 .config

Run make oldconfig to ask questions about changed parameters between the two kernel versions:

[jensd@cen linux-3.10.75]$ make oldconfig
...
Preemption Model
  1. No Forced Preemption (Server) (PREEMPT_NONE)
> 2. Voluntary Kernel Preemption (Desktop) (PREEMPT_VOLUNTARY)
  3. Preemptible Kernel (Low-Latency Desktop) (PREEMPT__LL) (NEW)
  4. Preemptible Kernel (Basic RT) (PREEMPT_RTB) (NEW)
  5. Fully Preemptible Kernel (RT) (PREEMPT_RT_FULL) (NEW)
choice[1-5]: 5
...
Debug preemptible kernel (DEBUG_PREEMPT) [Y/n/?] (NEW) n
...
  Preemption-off Latency Tracer (PREEMPT_TRACER) [N/y/?] (NEW)
  Scheduling Latency Tracer (SCHED_TRACER) [Y/n/?] y
    Scheduling Latency Histogram (WAKEUP_LATENCY_HIST) [N/y/?] (NEW)
  Missed Timer Offsets Histogram (MISSED_TIMER_OFFSETS_HIST) [N/y/?] (NEW)
...
Enable RCU priority boosting (RCU_BOOST) [Y/n/?] (NEW)
  Real-time priority to boost RCU readers to (RCU_BOOST_PRIO) [1] (NEW)
  Milliseconds to delay boosting after RCU grace-period start (RCU_BOOST_DELAY) [500] (NEW)
...
#
# configuration written to .config
#

In the above output, I only show the questions which were introduced because of the realtime patch. make will ask for more questions, mainly related to new functions or drivers but those aren’t related to realtime. For most other questions you can just choose the default (press enter) unless you’re sure about what you’re choosing.

At the end of this step, you should get a .config file in the patched kernel’s source tree which is suitable for the downloaded kernel source and patch.

If you want to configure and change all values, not only the values which have been changed between the installed and new kernel version, you can run make menuconfig (or make xconfig) to get a graphical overview of all possible parameters:

Compile and install the realtime kernel

Now that we’ve got a .config containing all possible answers on questions during compilation, it’s time to actually start compiling the kernel:

[jensd@cen linux-3.10.75]$ make

The compilation process might take a while to complete. To give an indication, it took me around 20min in a VM with one CPU core of an i7 4770 without using an SSD.

To use more, or a specific number of, cores during the compilation, you can specify them with -j<number> as a parameter of the make command. While this improves the required time for compilation, it can be hard to find error messages.

After compiling the kernel itself, we also need to compile the modules which were marked in .config and install them in the required locations:

[jensd@cen linux-3.10.75]$ sudo make modules_install install

This step takes less time than the previous one. On the same system mentioned above, this took me around 3 min.

Update the boot configuration for Grub

Now that the realtime kernel and related modules are compiled and installed, we need to update the GRUB-configuration in order to include the newly installted realtime kernel in the list of installed kernels.

[jensd@cen linux-3.10.75]$ sudo /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.75-rt80
Found initrd image: /boot/initramfs-3.10.75-rt80.img
Found linux image: /boot/vmlinuz-3.10.0-229.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-229.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-b7053b44ce9741f2aaf894be25d6a3dc
Found initrd image: /boot/initramfs-0-rescue-b7053b44ce9741f2aaf894be25d6a3dc.img
done

As you can see in the above output, the new kernel was found and added to the list of available kernels.

Reboot and start using the realtime kernel

The last thing which we need to do, is to reboot our system and select the new realtime kernel from the list in GRUB:

If all goes well, your system should boot with a realtime kernel and you can start exploiting it:

[jensd@cen ~]$ uname -r
3.10.75-rt80

Should you not manage to boot your system with the newly compiled kerel, you can just reboot and select the old kernel. When the system comes back up, you can find out what went wrong.