SAP Note

3108302 - SAP HANA DB: Recommended OS Settings for RHEL 9

Component: HAN-DB (SAP HANA > SAP HANA Database), Version: 8, Released On: 11.12.2023

Symptom
You want to configure optimal settings for running SAP HANA 2.0 on RHEL 9 with RHEL for SAP Solutions

Other Terms
HANA, DB, BW on HANA, Suite on HANA, S/4HANA, BW/4HANA, RedHat, Linux

Reason and Prerequisites

You are planning to run SAP HANA on Red Hat Enterprise Linux 9 (RHEL 9) for SAP Solutions.

Solution
To optimize the use of SAP HANA with RHEL 9, apply the following settings. There might be architectural limitations in
virtual and cloud environments, so that some recommendations might not apply or deviate. For example, CPU power
management might be unavailable. Please refer to the virtualization or cloud provider for any such limitations.
You must have a valid Red Hat Enterprise Linux for SAP Solutions subscription for all RHEL 9 servers running SAP HANA. To
ensure that these systems remain on the minor releases which are supported by SAP for SAP HANA, all RHEL 9 servers
running SAP HANA must be subscribed to the "RHEL for SAP HANA - Extended Update Support" ("eus") channel or to the
"RHEL for SAP HANA - Update Services for SAP Solutions" ("e4s") channel. These two channels are included in the Red Hat
Enterprise Linux for SAP Solutions subscription.
Update Services for SAP Solutions is available/planned for RHEL 9.0 (see the RHEL Product Life Cycle web page).
Please refer to the following Red Hat knowledge base article for instructions on how to configure your system accordingly (Red
Hat customer portal login required): How to subscribe to Update Services for SAP Solutions on RHEL 8 and RHEL 9.
Supported Kernel versions and patches
You can use official RHEL 9 Linux Kernels and packages shipped by Red Hat for the RHEL 9 minor releases supported by SAP
HANA, including security and bug fixes provided via "Extended Update Support" for the supported RHEL 9 minor releases.
Please see below for release-specific requirements. Using updated Linux Kernels and packages from RHEL 9 minor releases
not listed in SAP Note 2235581 - SAP HANA: Supported Operating Systems - is not supported and will break the certification.
RHEL 9 minor release specific requirements
• RHEL 9.2
◦ The minimum supported Linux kernel version is 5.14.0-284.25.1.el9_2 or newer.
• RHEL 9.0
◦ Please see SAP Note 3318049 for additional remarks.
◦ Intel: The minimum supported Linux kernel version is 5.14.0-70.22.1.el9_0.x86_64 or newer.
◦ Power: The minimum supported Linux kernel version is 5.14.0-70.43.1.el9_0.ppc64le or newer and glibc
version 2.34-28.el9_0.3.ppc64le.
Additional 3rd-party kernel modules
If you are using additional 3rd-party binary kernel modules, the following SAP Note applies: SAP note 784391 - SAP support
terms and 3rd-party Linux kernel drivers.
SELinux configuration
Red Hat Enterprise Linux uses SELinux technology which is enabled by default. Since there is no SELinux policy available for
SAP HANA, leaving SELinux enabled can lead to problems when running SAP HANA on RHEL.
It is therefore generally recommended to disable SELinux.
Although SAP does not deliver or support any SELinux policies, SELinux can allow SAP HANA to run within "unconfined"
mode.
This configuration was tested to work with a standard workload on
• RHEL 9.2 with HANA 2.0 SPS05 revision 59.09, HANA 2.0 SPS06 revision 67.02 and HANA 2.0 SPS07 revision 72
• RHEL 9.0 with HANA 2.0 SPS05 revision 59.08, HANA 2.0 SPS06 revision 67.01 and HANA 2.0 SPS07 revision 70
(Intel only)
Nevertheless, SAP, when analyzing problems, reserves the right to have SELinux disabled. If a problem cannot be reproduced
with SELinux disabled, Red Hat needs to analyze and solve the cause of the problem.
For checking the SELinux mode, use the getenforce command, as in the following example:
# getenforce
Enforcing
Changing the SELinux mode from "Enforcing" to "Disabled" immediately on a running system is not possible. For setting the
SELinux mode to "Disabled" permanently, use the following procedure:
Modify the RHEL kernel command lines as follows (the command will update all kernel command lines but by choosing a
different argument to option --update-kernel, you can update just one of the kernel command lines):
# grubby --args="selinux=0" --update-kernel=ALL
The change will only become effective after a system reboot. Please refer to the following Red Hat knowledge base article for
instructions on changing SELinux states and modes.
Configure tuned to use profile "sap-hana"
The tuned profile "sap-hana", which is provided by Red Hat as part of the RHEL for SAP Solutions subscription, contains
many of the settings mentioned below and configures some additional settings. Therefore the "sap-hana" tuned profile must
be activated on all systems running SAP HANA:
Use the following commands to install and activate the tuned profile "sap-hana" and check if it is active:
# yum install tuned-profiles-sap-hana
# systemctl start tuned
# systemctl enable tuned
# tuned-adm profile sap-hana
# tuned-adm active
The output of "tuned-adm active" should look like:
Current active profile: sap-hana
There is no minimum version requirement for package tuned-profiles-sap-hana.
Disable kdump
The kernel crash dump facility (kdump) can be disabled with the following commands:
# systemctl stop kdump
# systemctl disable kdump
In case you want to leave kdump enabled, have a look at the following Red Hat Customer Portal articles (Red Hat Customer
Portal login required):
How to troubleshoot kernel crashes, hangs, or reboots with kdump on Red Hat Enterprise Linux
How to reduce the size of a vmcore file?
Turn off auto-numa balancing
In RHEL 9, auto-numa balancing is switched off by default. To verify, use the following command:
# sysctl kernel.numa_balancing
kernel.numa_balancing = 0
If package numad is installed on your server, please make sure that it is switched off and disabled:
# systemctl stop numad
# systemctl disable numad
Disable Transparent Hugepages (THP)
With RHEL 9, the usage of transparent hugepages (THP) is generally activated in the Linux kernel. THP allow the handling of
multiple pages as hugepages, reducing the translation lookaside buffer footprint (TLB) in situations where it might be useful.
Due to the special manner of SAP HANA's memory management, the usage of THP may lead to hanging situations and
performance degradations.
If you use the "sap-hana" tuned profile as described above, THP will be disabled via the profile and you don’t need to change
this setting manually. If you do not use the "sap-hana" tuned profile, to disable the usage of transparent hugepages during
runtime, set the kernel settings at runtime with the command:
# echo never > /sys/kernel/mm/transparent_hugepage/enabled
There is no need to shut down the database to apply this setting. For making this change to persist across system reboots, it is
necessary to edit the OS bootloader configuration as follows:
#grubby --update-kernel=ALL --args="transparent_hugepage=never"
In a scale-out environment, those changes have to be done on every server of the landscape.
See Chapter 4 Setting kernel command line parameters in the RHEL 9 Managing, monitoring and updating the kernel guide
for more information on setting kernel parameters at boot time.
To verify that THP have been disabled, use the following command:
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
Cluster-On-Die (COD) / sub-NUMA clustering technology (Intel 64 platform only)
SAP HANA neither supports use of Intel Cluster-On-Die (COD) technology nor sub-NUMA clustering technology.
Configure C-States for lower latency in Linux (Intel platform only)
The Linux kernel shipped with RHEL 9 on the x86_64 platform includes a cpuidle driver for Intel CPUs: intel_idle. This
driver leads to a different behavior in C-States switching.
C-States range from C0 to Cn. C0 indicates an active state. All other C-States (C1-Cn) represent idle sleep states where the
processor clock is inactive (cannot execute instructions) and different parts of the processor are powered down. As the C-
States get deeper, the exit latency duration becomes longer (the time to transition to C0) and the power savings become
greater.
Therefore it is generally recommended to limit the C-States to C0 and C1 by setting the following parameter:
processor.max_cstate=1

Additionally, you can also configure the intel_idle kernel module to allow C-State C1:

intel_idle.max_cstate=1

To change these parameters, run the following command:

#grubby --update-kernel=ALL --args="processor.max_cstate=1 intel_idle.max_cstate=1"

However, depending on the type of workload and the processor model, setting a deeper c-state can be beneficial for power
savings as well as reaching peak frequencies (depending on the Thermal Design Power of a CPU board which usually means
that some CPUs can achieve peak frequency only if other CPUs are at lower frequencies).
To allow more flexibility the C-State is set in the sap-hana tuned profile as cstate.id:3

If sap-hana tuned profile is already installed and active in your system you will not need to take any action if c-state id 3 is the
best choice for your workload.

*Please note that addressing the C-States via their ID will result in different idle states on different architectures, which also
have different wakeup latencies.

In a scale-out environment, those changes have to be done on each server of the landscape.
See Chapter 4. Setting kernel command line parameters in the RHEL 9 Managing, monitoring and updating the kernel guide
for more information on setting kernel parameters at boot time.
For more information on setting C-States, see the following Red Hat Knowledge Base article: What are CPU "C-states" and
how to disable them if needed? (Red Hat Customer Portal login required).
Configure CPU Governor for performance (Intel 64 platform only)
Linux is using a technology for power saving called CPU governors to control CPU frequency and power consumption. By
default, Linux uses the governor "ondemand", which will dynamically scale frequency and voltage up and down depending on
CPU load. If you want to maximize CPU performance at the cost of increased energy consumption, SAP recommends to use
the governor "performance". If you use the "sap-hana" tuned profile as described above, the governor "performance" is
configured via the profile and you don't need to change this setting manually. If not using "sap-hana" tuned profile, insert the
following code in a system startup script such as /etc/rc.d/rc.local :
cpupower frequency-set -g performance
The setting will be applied during system boot. To take effect immediately, the script or the command must be executed.
Configure Energy Performance Bias (EPB, Intel 64 platform only)

The Linux kernel obeys the EPB bit in the CPU flags. Consequently, it configures the CPUs according to the global "energy
performance bias" setting in the BIOS.

We suggest to set the EPB using either one of the following options:
1. Set Energy Performance Bias or EPB in the BIOS to "Maximum performance"
2. Activate the 'sap-hana' tuned profile as described above which will set EPB to 'performance
3. Add the following command to a script executed on system boot, such as /etc/init.d/boot.local:
# cpupower set -b 0
(see 'cpupower help set' for more information)

To verify that EPB has been set correctly, you can run the following command:
# cpupower info
If the command reports 'perf-bias: 0', EPB has been set to the correct value.
Enable TSX (Intel Platform only)
Because Intel TSX is disabled by default on RHEL 8.3 and later on x86_64 systems (see Transactional Synchronization
Extensions (TSX) Asynchronous Abort for more information), add kernel parameter "tsx=on" to enable the TSX feature again:
# grubby --args="tsx=on" --update-kernel=ALL
In order for this change to become effective, the system has to be rebooted. Note that if further changes to the kernel
command line are required, the system should only be rebooted after performing all changes.
Disable Kernel samepage merging (KSM)

The KSM feature helps reduce physical memory overhead by detecting memory pages with identical content. The feature is
useful for VMs, but the space-time tradeoff does not pay off for SAP HANA instances not running in VMs.

If KSM is enabled, we recommend to disable it by adding the following command to a script executed on system boot, such as
/etc/init.d/boot.local:

# echo 0 > /sys/kernel/mm/ksm/run

Execute the command to turn off KSM immediately.

The following command can be used to show the status of KSM:

# cat /sys/kernel/mm/ksm/run

The result of "0" confirms that KSM is deactivated.

Further system configuration settings
Please see SAP note 2382421 - Optimizing the Network Configuration on HANA- and OS-Level - for network related
configuration parameter settings which may apply.
Please note: Unless noted otherwise, all changes mentioned above require root access on operating system level. It is
recommended to reboot the system after the changes have been applied. Future implementations might lead to changes of
these recommendations.

Attributes
Key Value

Other Components Basis Components > Operating System Platforms > Linux > Red Hat Linux (BC-OP-LNX-RH)

This document refers to

SAP Note/KBA Component Title

3108316 BC-OP-LNX-RH Red Hat Enterprise Linux 9.x: Installation and Configuration

This document is referenced by

SAP Note/KBA Component Title

3062226 SAP HANA: RHEL Settings for Fujitsu Servers

3318049 HAN-DB Indexserver Crashes at __pthread_kill_implementation When Running on RHEL 9.x

3108316 BC-OP-LNX-RH Red Hat Enterprise Linux 9.x: Installation and Configuration