How to Secure IB Fabric Management
Securing IB Fabric Management Overview
There are several means by which IB fabric management may be protected, part of them do exist in the
IB specifications being standard part of the SM and can be configured on the opensm.conf file .
Other tools are Specifically developed by Mellanox to strengthen Security capabilities .
Main Security Tools
1. Mellanox Secured Host mechanism, combines multiple security mechanisms, and implemented on
Mellanox HCA cards
2. IB specifications – SM Management Key, known as the M_KEY
3. IB specifications – SM Authentication Key, known as the SM_KEY
The following slide will provide brief description of those features
Mellanox Secured HOST Main features
1. Mellanox Secured Host feature
a. A server installing that feature, cannot be “hijacked“ by malicious elements in order to function as an SM
This function avoids the capability of this HCA, to send SM MAD packets, therefore, it will be useless as an
SM, and cannot be used to control the fabric. Any query commands sent to SM cannot work as well.
b. Blocking access of untrusted entities to the device configuration registers
QP 0 QP 0
SM MADS
RX TX
SM
Mellanox HCA
“Secured Host” TX RX
Outgoing SM MADS
are restricted
Mellanox Secured HOST
supporting HCA versions
1. Currently Supported by Mellanox HCAs (01/2017)
a. Connectx_3 pro
b. Connectx_3 vpi
Avoid Fabric Discovery by Unauthorized SM
Using IB Management Key (M_Key)
1. Subnet manager will distribute the
M_KEY to all HCA ports, and all
managed switches port 0.
SM get DISCOVERY MADS
2. M_KEY protected Node, ignores any
M_KEY
MAD arrives with the wrong key. Management Key Right m_key Right m_key
SM
3. M_KEY will avoid Fabric discovery by Right m_key Right m_key
malicious SM, since it does not use a
Valid M_KEY. SM Discovery MADS Response
SM get DISCOVERY MADS
M_KEY NEW /
Wrong m_key Wrong m_key MALICIOUS
Management Key
SM ENTITY
Inter-SM Authentication for Handover Control
Using OPENSM.CONF SM_KEY SM INFO PACKETS
1. SM_KEY – INTER SMs Authentication
a. SM info protocol will allow SM Right m_key Right m_key New SM Entity
Handover to a Higher priority Priority 11 Priority 13
SM, only in case the new SM
wishing to be Master, is Right m_key Right m_key
SM INFO PACKETS
authenticated with the right
SM_key, distributed by the
Master SM.
New SM Entity
SM
Priority 11 Priority 13
b. Newcomer SM trying to use New Master
Higher priority for Handover, will
not be able to have a dialog with
current SM, in case it does not
have exact same key.
SM INFO PACKETS
Wrong m_key Wrong m_key New SM Entity
SM
Priority 11 Priority 13
No return packets
How to Implement Mellanox “Secure host“ (A)
1. Verify that your HCAs are Mellanox ones, 1. Currently on 01/2017 the feature is
and do support “secure host feature“ supported by :
2. Find out the PSID of that card, and locate a) Connectx_3 pro
its relevant .mlx and .ini files , b) Connectx_3 pro
3. Modify the relevant .ini file, by adding the 2. Trace the following:
“secure host“ capability using a text 1. Check the relevant PSID of your card
editor. (IBV_DEVINFO)
2. Ask Mellanox Support to provide you the
4. Create a new .bin file, composed by the relevant .mlx file, and .ini file for the
original .mlx file ,and the modified .ini file, relevant PSID
using the “mlxburn” command
3. Add the following line, under [HCA]:
cr_protection_en=true
mlxburn -fw ./fw-4103-rel.mlx -conf
./secure_host.ini –wrimage
newname_secure.secure.bin
How to implement Mellanox “Secure host “ (B)
5. Burn the new bin file (image) to you HCA , 5. flint -d /dev/mst/mt4103_pci_cr0 -i
using the flint command newname_secure.bin b
6. Reboot the driver
6. service openibd restart
7. Set a Secure Host key is in order to enable or
disable the features , using flint 7. flint -d /dev/mst/mt4099_pci_cr0 set_key
<xcsdfsdad334>
8. Reboot the driver to complete feature
activation
8. service openibd restart
9. You may Verify that the feature is activated ,
using sminfo command should be responded 9. Sminfo
with errors ibwarn: [14488] mad_rpc: _do_madrpc
failed; dport (Lid x)
sminfo: iberror: failed: query
How to Implement M_KEY Protection Mode
Allowing Authentication Process Between SM and Fabric Nodes
Start editing opensm.conf file opensm.conf
1. Set the M_KEY parameter, and enter 1. m_key 0x2b6jkkscomso81a5
your secured m_key
2. Set the protection mode as “2”, in 2. m_key_protection_level <2>
order to activate the feature
# M_Key value sent to all ports qualifying all Set(PortInfo)
m_key 0x0000000000000xxx
# The protection level used for the M_Key on this subnet
m_key_protection_level 2
Operational Commands
when M_KEY is activated
Operational command must be preceded with mkey. ▪ <command > -y MKEY
Examples :
▪ sminfo -y MKEY
▪ sminfo
▪ ibnetdiscover -y MKEY
▪ Ibnediscover
▪ iblinkinfo -y MKEY
▪ Iblinkinfo
▪ ibdiagnet –mkey (lowercase)
▪ Ibdiagnet
▪ Note the following command will be supported only
from OFED 4 and later:
ibhosts , ibswitches , ibnodes , ibaddr
[root@mtlacad06 ~]# sminfo -y MKEY
M_Key: xxxxxxxxxxx
