Onyx for HPE M-Series

Ethernet Switch Release Notes

Software Version 3.9.21xx

Part Number: 882262-001

Published: January 10, 2021

Edition: 1
Notices
© Copyright 2020 Hewlett Packard Enterprise Development LP

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard
Enterprise products and services are set forth in the express warranty statements accompanying such products
and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard
Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise
has no control over and is not responsible for information outside the Hewlett Packard Enterprise website.
Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use or
copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software
Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's
standard commercial license.

Mellanox®, Mellanox logo, ConnectX®, MLNX-OS®, SwitchX®, Virtual Protocol Interconnect®, are registered
trademarks of Mellanox Technologies, Ltd.

All other trademarks are property of their respective owners.

For the most updated list of Mellanox trademarks, visit http://www.mellanox.com/page/trademarks

2
Table of Contents
Document Update History .................................................................................................................... 4
1 Introduction ..................................................................................................................................... 5
2 Supported Platforms, Firmware, Cables and Licenses .............................................................. 5
2.1 Supported Switch Systems..................................................................................................... 5
2.2 Embedded Onyx Firmware ..................................................................................................... 5
2.3 Software/Firmware Interoperability ......................................................................................... 5
2.4 Supported BIOS Version ........................................................................................................ 6
2.5 Supported Software Licenses................................................................................................. 6
2.6 Upgrade From Previous Releases ......................................................................................... 6
3 Changes and New Features ........................................................................................................... 7
4 Known Issues ................................................................................................................................ 24
4.1 General Known Issues ......................................................................................................... 24
4.2 Ethernet Known Issues......................................................................................................... 27
4.3 IP Routing Known Issues ..................................................................................................... 38
5 Bug Fixes ....................................................................................................................................... 43
Support and Other Resources ........................................................................................................... 63
Customer Self Repair .......................................................................................................................... 64
Remote Support ................................................................................................................................... 64
Websites ............................................................................................................................................... 66

3
 Document Update History
Table 1: Document Update History
Date Description

January 10, 2021 First release of this Onyx version

4
1 Introduction
This document is the Onyx Release Notes for HPE M-Series Ethernet Switch.
Onyx is a comprehensive management software solution that provides optimal
performance for cluster computing, enterprise data centers, and cloud computing
over Spectrum® family switch IC. The fabric management capabilities ensure the
highest fabric performance while the chassis management ensures the longest
switch up time.
The Onyx documentation package includes the following documents:
• User Manual provides general information about the scope, organization, and
command line interface of Onyx as well as basic configuration examples
• Release Notes provide information on the supported platforms, changes and
new features, and reports on software known issues as well as bug fixes

2 Supported Platforms, Firmware, Cables and

Licenses
2.1 Supported Switch Systems
Table 2: Supported Platforms
Model Number Description

SN4600cM HPE SN4600cM 100GbE 64QSFP28 switch

SN3700M HPE SN3700M 200GbE 32QSFP28 switch
SN3700cM HPE SN3700cM 100GbE 32QSFP28 switch
SN2100M HPE SN2100M 100GbE 16QSFP28 half width switch
SN2010M HPE SN2010M 25GbE 18SFP+ 4QSFP28 switch
SN2410bM HPE SN2410bM 10GbE 48SFP+ 8QSFP28 switch
SN2410M HPE SN2410M 25GbE 48SFP28 8QSFP28 switch
SN2700M HPE SN2700M 100GbE 32QSFP28 switch

2.2 Tested and Supported Firmware

• Spectrum® firmware version 13.2008.22xx
• Spectrum-2 firmware version 29.2008.22xx
• Spectrum-3 firmware version 30.2008.22xx

2.3 Software/Firmware Interoperability

• ConnectX®-3 firmware version 2.42.5000
• ConnectX-4 Lx firmware version 14.28.4000
• ConnectX-5 firmware 16.28.4000

5
 • ConnectX-6 firmware 20.28.4000
• Mellanox NEO® version 2.6
• ONIE version 5.2.0020

2.4 Supported BIOS Version

• x86 ATOM 5.6.5 – 0ACIF007_02.02.x02

2.5 Supported Software Licenses

For the software licenses supported with Onyx software please refer to the
“Licenses” section of the “Getting Started” chapter of the Onyx for HPE M-Series
Ethernet Switch User Manual.

NOTE: When installing an HP license in the following format:

#Switch nx
aaaa bbbb cccc .... yyyy zzzz
"Switch nx"
The system will report the following error:
"Malformed License: aaaa bbbb cccc ... yyyy".
To avoid this issue, please insert the license without the underlined hash-
marked description, above, and make sure the date set on your system matches
the date embedded in the supplied HP license.

2.6 Upgrade from Previous Releases

Older versions of Onyx may require upgrading to one or more intermediate versions
prior to upgrading to the latest. Missing an intermediate step may lead to errors.
Please refer to the table below to identify the correct upgrade order.
Table 3: Supported Software Upgrades
Target Version Verified Versions From Which to Upgrade

3.9.2xxx 3.9.1xxx, 3.9.0xxx

3.9.1xxx 3.9.0xxx, 3.8.2xxx

3.9.0xxx 3.8.2xxx, 3.8.1xxx

3.8.2xxx 3.8.1xxx, 3.7.1xxx

3.8.1xxx 3.8.1xxx, 3.7.1xxx

3.7.1xxx 3.7.0xxx, 3.6.8xxx, 3.6.6xxx

3.6.8xxx 3.6.6xxx, 3.6.5xxx

3.6.6xxx 3.6.5xxx, 3.6.4xxx

6
 Target Version Verified Versions From Which to Upgrade

3.6.5xxx 3.6.4xxx, 3.6.3xxx

3.6.4xxx 3.6.3xxx, 3.6.2xxx

3.6.3xxx 3.6.2xxx, 3.6.1xxx

3.6.2xxx 3.6.1xxx, 3.5.1xxx

3.6.1xxx 3.5.1xxx, 3.5.0xxx, 3.4.3xxx

3.5.1xxx 3.5.0xxx, 3.4.3xxx

3.5.0xxx 3.4.3xxx, 3.4.2xxx

3.4.3xxx 3.4.2xxx, 3.4.1xxx

3.4.2xxx 3.4.1xxx,3.4.0xxx

3.4.1xxx 3.4.0xxx, 3.3.5006

3.4.0xxx 3.3.5006

For upgrade instructions refer to the section “Upgrading Onyx Software” in the Onyx
for HPE M-Series Ethernet Switch User Manual.

3 Changes and New Features

Table 4: HPE M-Series Ethernet Switch Changes and New Features
Release # Category Description

3.9.21xx IGMP Snooping Added support for IGMP snooping report

profile filtering and max-group limit. For more
information, see "IGMP Snooping
Commands" and "Debugging" sections in the
user manual.

3.9.21xx General Bug fixes

3.9.2006 Auto-Negotiation Added ability to disallow speed auto-

negotiation on an interface. For more
information see the command "speed" in
"Ethernet Interface Commands" section in
the user manual.

3.9.2006 Cables Added support for passive copper splitter

cables 200Gbe to 4x50GbE on SN3700
systems.

3.9.2006 CRC Added beta-level support for replacing CRC

field with PTP timestamp on Spectrum-2 and
Spectrum-3 systems. For more information
see "System Synchronozation" section in the
user manual.

7
 Release # Category Description

3.9.2006 Management VRF Added support for Management VRF. For

more information, see "Management
Interfaces" section in the user manual

3.9.2006 Modules Added support for 40GbE SWDM4 modules

on SN2700 system.

3.9.2006 OpenFlow Added ability to add router ports into

OpenFlow bridge.

3.9.2006 Password Added support for password hardening. For

more information see "Getting Started"
section in the user manual.

3.9.2006 PBR Added support for Policy Based Routing

(PBR). For more information, see "IP Routing
Overview" and "General IP Routing
Commands" section in the user manual.

3.9.2006 PIM BiDir Added GA-level support for Bidirectional PIM.

For more information, see "IGMP and PIM"
section in the user manual.

3.9.2006 PTP Added GA-level support for PTP on

Spectrum-2 systems.

3.9.2006 PTP Added beta-level support for PTP on

Spectrum-3 systems.

3.9.2006 PTP TTL Added ability to set the TTL value of the PTP
messages. For more information, see "PTP
Commands" section in the user manual.

3.9.2006 SSH, Docker Trust Added support for SSH trust relation between
a running container and its host. For more
information see the commands "start" and
"show docker containers" in the
"Virtualization" section of the user manual.

3.9.2006 WJH Added support for WJH event suppression by

the severity for each drop group. For more
information, see "Configure What Just
Happened (WJH) Using CLI" in the user
manual.

3.9.2006 WJH SNMP Added support for what-just-happened

SNMP. For more information, see "Configure
What Just Happened (WJH) Using CLI"
section in the user manual.

3.9.2006 WJH, Buffer Added Buffer Occupancy filed in WJH buffer

event. For more information, see "Configure
What Just Happened (WJH) Using CLI"
section in the user manual.

8
Release # Category Description

3.9.1900 Bidirectional PIM Added beta-level support for Bidirectional

PIM. For more information see "Multicast"
section in the user manual.

3.9.1900 General Bug fixes

3.9.1900 PTP Added beta-level support for PTP on

Spectrum-2 systems.

3.9.1604 Docker Login Added ability to securely login and logout

through the CLI to various server repositories
and pull docker images from these servers.
See the commands "docker login", "docker
logout", and "show docker login" in the
Virtualization section of the user manual.

3.9.1604 General Bug fixes

3.9.1604 OpenFlow Added support for using a maximum number

of ports per ASIC in an OpenFlow group.

3.9.1304 EVPN Added GA-level EVPN centralized gateway

functionality.

3.9.1304 General Bug fixes

3.9.1304 Systems Added support for SN4600cM system.

3.9.1014 10GBASE-T Added support for 10GBASE-T modules on

modules SN3700 and SN3700C systems.

3.9.1014 Counters Added the ability to use a range of ports in

"show interfaces ethernet counters"
commands. For more information see the
following sections in the user manual:
"Ethernet Interfaces Commands," "QoS
Commands," "Link Aggregation Group
(LAG)," and "MLAG Commands."

3.9.1014 CRC Added support for replacing CRC field with

timestamp on Spectrum-2 and Spectrum-3
systems. For more information see "System
Synchronozation" section in the user manual.

3.9.1014 Ethernet Interfaces Improved the performance of "show running

config" & "show configuration running"
commands. When running the above
commands for the first time after configuring
the switch, the configuration is saved in
internal cache and the execution time of
these commands will be as in previous
releases. Each following execution of these
commands will retrieve the running
configuration from internal cache and will
display the output immediately. In case
configuration on switch is changed again, the
internal cache will be re-build and the output

9
 Release # Category Description

of these commands will be as in previous

releases.

3.9.1014 Ports, Reset Port Added support for resetting an interface to its
default settings. For information see "default
interface ethernet" command under "Ethernet
Interface Commands" section in the user
manual.

3.9.1014 Systems Added GA-level support for SN3700 system.

3.9.1014 VRRP Added support for IPv6 in VRRP and MAGP.

3.9.1014 VXLAN, IPv6 Added support for IPv6 Neighbor

Suppression. For more information see "nve
neigh-suppression" in the VXLAN
Commands section of the user manual.

3.9.1014 Web UI Added support for BGP feature in the Web

UI.

3.9.1014 Web UI Added support for VRF feature in the Web

UI.

3.9.1014 WJH, ACL Extended What-Just-Happened ACL display

to show the user ACL rule information of
dropped packets, including ACL rules added
using OpenFlow. For more information, see
"show what-just-happened" command in the
user manual.

3.9.0920 General Bug fixes

3.9.0914 General Bug fixes

3.9.0914 IP Address Added command "switchmode exceptions

sip-equals-dip" to disable the packet drops
for packets with a source IP address that
equals to the destination IP address. For
more information, see "Configure What Just
Happened (WJH) Using CLI" section in the
user manual.

3.9.0914 Link, Auto- Added GA-level support for link auto-

Negotiation negotiation on Spectrum-2 systems.

3.9.0914 MAC Added support for MAC masking in log

messages. For more information, see
"Logging" section in the user manual.

3.9.0914 PTP Added ES-level support for PTP backward

compatibility on Spectrum-2 and Spectrum-3
systems.

3.9.0914 Speed Added beta-level support for 200GbE on

Spectrum-2 systems with 3rd party support.

10
Release # Category Description

3.9.0914 WJH Added support for WJH auto-export file

generation. For more information, see
"Logging" section in the user manual.

3.9.0914 WJH Extended What-Just-Happened ACL display

to show the user ACL rule information of
dropped packets. For more information, see
"show what-just-happened" command in the
user manual. *Not including ACL rules added
using OpenFlow. This will be supported in
future releases.

3.9.0612 General Bug fixes

3.9.0608 BGP Added GA-level support for BGP

unnumbered interfaces.

3.9.0608 Configuration Added support for automated configuration

Management file backup.

3.9.0608 EVPN Added beta-level EVPN centralized gateway

functionality.

3.9.0608 General Bug fixes

3.9.0608 Recirculation Port Added GA-level support for port recirculation

feature for Spectrum-based systems, which
supports display of buffer dropped packets
by circulating them into the CPU using a
reserved port. For more information see
"Ethernet Interfaces" section in the user
manual.

3.9.0608 Telemetry, Added beta-level support for Telemetry

Spectrum-2 Threshold on Spectrum-2-based systems.

3.9.0608 Telemetry, Added support for Telemetry Sampling on

Spectrum-2 Spectrum-2-based systems.

3.9.0608 WJH Added support for WJH concurrent operation

of raw and aggregated channels on the same
level.

3.9.0608 WJH Added support for Forwarding and ACL drop

groups in WJH aggregation.

3.9.0608 WJH Added support for WJH Layer-1 raw and

aggregation data.

3.9.0608 WJH Added support for WJH Buffer Drops' raw

and aggregation data.

3.9.0300 BGP Added beta-level support for BGP

unnumbered interfaces.

3.9.0300 Configuration Added a new "reboot" option to the

Management "configuration text file * apply" command that

11
 Release # Category Description

writes the configuration and reboots after

successful execution.

3.9.0300 General Bug fixes

3.9.0300 LAG Added ability to support to up 32 ports per

LAG group in LAG and MLAG.

3.9.0300 Ports; Speed Added beta-level support for 200Gb/s

MLNX↔MLNX Force.

3.9.0300 Recirculation Port Added beta-level support for port

recirculation feature for Spectrum-based
systems, which supports display of buffer
dropped packets by circulating them into the
CPU using a reserved port. For more
information see "Ethernet Interfaces" section
in the user manual.

3.9.0300 RoCE Enhanced the “show roce” command to

display a list of exceptions from the default
profile on RoCE.

3.9.0300 Security SSH Login Notification now displays the

following information after authentication:
-Last successful and unsuccessful login
date/time
-Number of unsuccessful logins since last
successful login
-Changes to user's account since last login
(password, capability)
-Location of last successful and unsuccessful
login (terminal or IP)
-Number of total successful logins since last
X days

3.9.0300 Security Upgraded OpenSSH version to 8.0p1.

3.9.0300 SN3800 Added 40GbE support for SN3800 system.

3.9.0300 WJH Added support for Layer-1 aggregation data.

3.8.2306 General Bug fixes

3.8.2204 BGP Added new show commands to display

received/advertised BGP routes for a specific
address-family per neighbor.
See "show ip bgp neighbors
received/advertised address-family"
command in the user manual.

3.8.2204 BGP Added display of enhanced route-refresh

capability to the existing “show ip bgp
neighbor *” command.

3.8.2204 BGP Added new show command to display

12
Release # Category Description

3.8.2204 EVPN Added MAC mobility sequence number

information to the “show ip bgp evpn detail”
command.

3.8.2204 EVPN Added a new show command to display

current VXLAN tunnels when running in NVE
BGP controller mode.

3.8.2204 EVPN Replaced auto-completion of “show ip bgp

evpn route-type *” command with string
keywords instead on numbers.

3.8.2204 EVPN Added VLAN to VNI auto-mapping

capabilities. For more information, see
"interface nve auto-vlan-map" command in
the user manual.

3.8.2204 General Bug fixes

3.8.2110 General Bug fixes

3.8.2102 BGP Added support for BGP IP prefix-list filtering

and updated prefix-list in context mode. See
"BGP Commands" section in user manual,
for new CLI commands.
Note: “export” direction is at GA-level while
"import" direction in "route-table prefix-list"
command is at alpha-level testing.

3.8.2102 BPDU Guard Added reflection of BPDU guard errors in the

interface status show command. When an
interface receives a BPDU packet while
BPDU guard is enabled, the status will show
"down (suspended)."
Added log the BPDU Guard event occurred.
See "BPDU Guard" section in user manual.

3.8.2102 EVPN Added EVPN MAC mobility warnings

messages to assist in network and
administration management. See EVPN
Logging example in the user manual.

3.8.2102 General Bug fixes

3.8.2102 NTP Server Added new CLI commands that allow the
user the block the switch's ability to function
as an NTP server. For more information, see
NTP commands in "NTP and Clock" section
in the user manual.

3.8.2102 PFC Added the ability to disable global PFC mode

regardless of the default.

3.8.2102 PTP The feature allows for tracking of state

changes of the PTP stack. The new show
command allows the following:
-Displays the last 100 entries for Offset from

13
 Release # Category Description

Master and Mean Path Delay.

-Enables logging of the mean path delay
value if it exceeds the specified threshold.
-Displays PTP time-property parameters
(time source, current utc offset etc).
-Displays all PTP foreign masters per each
PTP port.
-Enables logging of the offset from master
value if it exceeds the specified threshold
For more info see "PTP Commands" section
in the user manual for new CLI commands.

3.8.2102 RoCE, Port Modified default LAG HASH to support

Channel, LAG TCP/UDP ports.

3.8.2102 Syslog UDP/TCP The "crypto certificate system-self-signed

regenerate" command was added with the
option to specify the certificate CA basic
constraints flag. For more information see "
Cryptographic (X.509, IPSec) and
Encryption" section in the user manual.

3.8.2102 WJH Added the following What Just Happened

CLI enhancements and changes:
-Changed display of dropped packets
captured by WJH into a single table.
-Removed "Size" columns and added
"Severity" and "Proposed Action" columns.
-Added support for LAG/MLAG port.

3.8.2008 General Bug fixes

3.8.2004 ARP Added new "show ip arp timeout" CLI

command to display the value of ARP
timeout.

3.8.2004 General Removed “StoreFabric" wording from the

HPE switch software in both the CLI and the
web UI.
Note: In case of an upgrade from versions
3.8.1000 or older to 3.8.2000 or newer, clear
configuration with reset-factory to completely
remove "StoreFabric" wording.

3.8.2004 General Added support to prevent disk from running

out of space due to sysdump files.

3.8.2004 General Bug fixes

3.8.2004 IGMP Snooping Added support for IGMP snooping querier-

guard.

3.8.2004 Link Down Added support for Link Down Reasoning.

Reasoning

14
Release # Category Description

3.8.2004 Link Speed Added engineering sample support for

200GbE link speed for SN3700 Spectrum-2
switch systems (currently works only
between Mellanox switches with optic cables
up to 5m and copper cables up to 2m only).

3.8.2004 Logging Added support to send syslog messages

based only on a given regex.
For more information, see “Logging
Commands” section in the user manual.

3.8.2004 PTP Added PTP support for IPv6.

3.8.2004 RoCE Added the ability to clear all counters that are
shown under "show interface ethernet
counters roce" including max-usage
counters. For more information see "RoCE
Commands" section in the user manual.

3.8.2004 RoCE Automation Added autoconfiguration of shared buffers,

ETS, trust mode, and ECN in RoCE scenario.
For more information, see “RoCE” section in
the user manual.

3.8.2004 Running-config Added version information to the show

running-config.

3.8.2004 Switch System Added software-level support for SN3700cM

Spectrum-2 switch systems.

3.8.2004 WJH Added ACL support for What Just Happened.

3.8.1302 Configuration Added the ability to delete shared storage for

Management the docker during “reset factory.”
See “reset factory” command in the user
manual.

3.8.1302 General Bug fixes

3.8.1302 Global Flow Added support for Global Flow Control

Control Watchdog.
For further information, see section “PFC
Watchdog” in the User Manual.

3.8.1302 QoS Enabled PCP and/or DSCP fields rewrite on

a port configuration, with any port trust mode
configured (L2/L3/both).

3.8.1208 General Bug fixes

3.8.1000 Break-out Cables Confirmation for split-port has changed to an

all-caps “YES”.

3.8.1000 Cables Added Spectrum Finisar SFP 10GbE-ZR

module PN: FTLX1871D3BxL up to 80km.

15
 Release # Category Description

3.8.1000 CLI Added Ctrl-w key shortcut support to the CLI.

See section “CLI Shortcuts” in the user
manual.

3.8.1000 CLI Enhanced “show running-config” command

to display information for a specific interface.
Example: “show running-config interface
ethernet 1/1”

3.8.1000 CLI Enhanced “show running-config” command

to show differences between saved
configurations.
Example: “show running-config diff”

3.8.1000 Docker Added the option to mount USB to a docker

container.

3.8.1000 Docker Enhanced dockers performance with

Overlay2 driver.

3.8.1000 ECMP Added GA level support for consistent

hashing function.

3.8.1000 EVPN Added support for EVPN (L2).

Supported scale: Up to 512VNIs and 10K
MAC addresses

3.8.1000 IP Routing Added the option to configure IPv4 Link Local

Range on an IP interface.

3.8.1000 IPv6 Added IPv6 support for RADIUS and Syslog.

3.8.1000 JSON Added support for running a single HTTP

request for both authentication and JSON
request.

3.8.1000 LAG Added support for configuring symmetric

LAG hash.

3.8.1000 PIM Added support for PIM with MLAG

configuration.

3.8.1000 Security Added support for EAP/PEAP cipher on

RADIUS client.

3.8.1000 Shared Buffers Added command to clear multicast shared

buffers counters.
See the command “clear buffers pool mc-
buffers max-usage” in the user manual.

3.8.1000 Telemetry Added drop reason group granularity for

displaying and clearing dropped packets in
What Just Happened (WJH) capability.

3.8.1000 Telemetry Improved What Just Happened (WJH)

response time.

16
Release # Category Description

3.8.1000 Telemetry Added support for deleting telemetry

statistics files.

3.8.1000 Telemetry Added ability to create PCAP files from

dropped packets when using the What Just
Happened (WJH) capability at GA level.

3.8.1000 Telemetry Added a Wireshark dissector in order to

analyze WJH PCAP files. The analyzed file
displays the packets’ added metadata at GA
level.

3.8.1000 Telemetry Upon packet loss, or a critical system failure,

the system will auto-generate a .pcap file
when using the What Just Happened
capability at GA level.

3.8.1000 Telemetry What Just Happened: Added support for

Router packet discards.

3.8.1000 WebUI Removed a few insecure parameters of the

HTTP header.

3.8.1000 WebUI Added support for What Just Happened

(WJH).

17
 4 Customer Affecting Changes
Table 5: HPE M-Series Ethernet Switch Customer Affecting Changes
Release # Category Description

3.9.2000 MTU MTU default value of L2 and L3 interfaces was

changed from 1500 to 9216.
The new behavior depends on how the versions
are installed.
1. Switches that perform an upgrade to the new
version will experience the following:
-MTU value of interfaces included in MLAG IPL
will get the default value (9216).
-MTU value of all existing interfaces will not be
changed (will be 1500 or any other value
configured by user).
-Show running config will now show the
configuration.
-For newly created L2/L3 interfaces, MTU value
will be 9216.
2. Switches that will perform a manufacture
restart to the new version will experience the
following:
-All L2 interfaces will be created with MTU value
of 9216.
-Newly created L2/L3 interfaces will have MTU
value of 9216.

3.9.2000 Spectrum-2, On Spectrum-2 and Spectrum-3 systems,

Spectrum-3, entPhysicalIndex calculation for port module and
SNMP port module sensor will change and a new value
will be displayed in the following SNMP tables:
Entity MIB—entPhysicalTable and Entity Sensor
MIB—entPhySensorTable.

3.9.2000 SSH SSH connections will no longer support the diffie-

hellman-group14-sha1 cipher under key
exchange algorithms and will be moved to non-
secure ciphers.

3.9.1900 Static IP In cases where a management interface has both

static IP and DHCP configured, DHCP will not tun
and static IP will be the assigned IP.

3.9.1304 JSON Changed the JSON output of command "show ip

bgp neighbors".

3.9.1304 Logging Monitor The behavior of "logging monitor event" was

Event changed and the prompt line will now be printed
for each event that arrives. As a result, multiple
events can no longer be seen printed as a group;
each event is treated separately.

3.9.1304 WJH The following changes have been made when

another application takes control of WJH. show

18
Release # Category Description

what-just-happened status:Before change: WJH

groups appeared with "Operational status" as
Enabled. After change: If another application (e.g.
NEO) takes control of WJH, the "Operational
status" will be Disabled. show what-just-
happened <drop-reason-group> will no longer
try to take control of WJH. Running the show
command while another application controls WJH
will now provide no output.If the OS needs to take
control of WJH, the command must be disabled
the flow must be enabled again.

3.9.1014 Certificate If a certificate is provided without the CA flag, the

Configuration switch will parse this certificate as though the flag
was set to false. All CA certificates should have
the CA flag set to true.

3.9.1014 Default Port The default port speed for SN3700 system has
Speed, Spectrum- been changed from 100GbE to 200GbE. As such,
2, SN3700 while splitting to 2, the default speed is now
100GbE and, which splitting to 4, the default
speed is 50GbE.
The speeds that have been configured, will
remain unchanged.
Note: This change does not relate to the
SN3700C system.

3.9.1014 Ethernet Due to new counter display enhancement, the

Interfaces, JSON output has been modified.
Counters

3.9.1014 QoS, Congestion- show interface congestion-control output was

Control modified to increase readability and fix JSON
form.

3.9.1014 RoCE The behavior in the following scenario has been

changed: The system starts in non-RoCE mode.
Parameter 'v1' has value 'x' which is the default in
non-RoCE mode. The user manually configures
parameter 'v1' to 'y'. The user switches to RoCE
mode. Parameter 'v1' still has the value 'y'. 'y' also
happens to be the default in RoCE mode. The
user switches back to non-RoCE mode. We now
check the value of parameter 'v1'. The old
behavior: Parameter 'v1' now has value 'y'. The
new behavior: Parameter 'v1' now has value 'x'.

3.9.1014 VRRP The ping response to VIP is now with the

interface VLAN MAC of the Master and not with
the VMAC.

3.9.1014 Deprecation The following previously-unsupported commands

are officially deprecated as of 3.9.1000:
ipv6 multicast-routing
ipv6 multicast-routing vrf *

19
 Release # Category Description

no ipv6 multicast-routing
no ipv6 multicast-routing vrf *
show ipv6 mld interface
show ipv6 mld interface * brief
show ipv6 mld * groups interface *
show ipv6 mld * groups interface * detail
clear ipv6 mld * groups interface *
protocol mld
no protocol mld
[no] interface <type> <id> { ipv6 mld | ipv6 mld
immediate-leave |
ipv6 mld
robustness-variable |
ipv6 mld
robustness-variable * |
ipv6 mld query-
max-response-time |
ipv6 mld query-
max-response-time * |
ipv6 mld query-
interval |
ipv6 mld query-
interval * |
ipv6 mld last-
member-query-response-time |
ipv6 mld last-
member-query-response-time * |
ipv6 mld version |
ipv6 mld version *
|
ipv6 mld join-
group |
ipv6 mld static-oif
|
ipv6 mld static-oif
*|
ipv6 mld static-oif
* source-ip |
ipv6 mld static-oif
* source-ip * |
ipv6 mld last-
member-query-count |
ipv6 mld last-
member-query-count * |
ipv6 mld startup-
query-interval }

3.9.0914 MAC From this release on, the first two bytes of the
MAC address in the log will be marked with
asterisks for security purposes. This may be
disabled using the no form of the “logging mac
masking” command. See “Logging” section in the
user manual for more information.

20
Release # Category Description

3.9.0914 PAM4 Link The following are the minimal software/firmware

Speeds when versions that support PAM4 link speeds when
Using connected using M-Series-to-M-Series and
400GbE/200GbE Mellanox-to-3rd party devices:
Mellanox Spectrum-3: 30.2007.1142
Mellanox Spectrum-2: 29.2007.1142
Mellanox Onyx: 3.9.0830
SONiC/SAI: 201911-
fffee7e33a1e9ccf564407966003ac45305fe64e
ConnectX-6/ConnectX-6 Dx: 20/22.27.2008*
*Note: NICs with this firmware version, and
above, support M-Series-to-M-Series connectivity
with PAM4 link speeds.

3.9.0914 Split For split commands, the syntax “module-type

qsfp” was replaced with “no module-type”. For
more information, see “Ethernet Interfaces”
section in the user manual.

3.9.0914 WJH, Logging For the logging commands, the “interfaces”

option/syntax was replaced with “what-just-
happened-packets”. For more information, see
“Logging” section in the user manual.

3.9.0608 Autonegotiation The "no auto-neg" command is no longer

available. Autonegotiation configuration can be
achieved by the 'auto' option.

3.9.0608 WJH no-pcap' attribute will be removed, and no pcap

file will be generated with execution of the basic
flavor of the command. To create pcap files on
command execution, use the following form:
switch (config) # show what-just-happened *
[export [<file-name>]] [no-metadata]

3.9.0300 WJH WJH pcap file names beginning with a dot are
now prohibited, since such files cannot be
uploaded or deleted.

3.9.0300 WJH Modified “show what-just-happened” table output.

3.9.0300 BGP [deprecation] BGP command "no-route-map" is deprecated and

been replaced with the disable form of the BGP
route-map command. For more information see
"BGP Commands" section in the User Manual.

3.9.0300 Counters Currently, not all discarded packets are counted

[deprecation] in "show interface counters" command 'discard
packets' counter. The name of the counter will be
changed to 'no buffer discards' in upcoming
releases.

3.9.0300 VRF [deprecation] “show interfaces <IP Interface>” (for example,

router-port or VLAN interface) will be deprecated
in a future release. Use “show ip interface . . .”

21
 Release # Category Description

command that has VRF support, for example

“show ip interface vrf all brief”.

3.8.2204 EVPN Existing configuration CLI was changed from

“router bgp 100 address-family l2vpn-evpn auto-
create” to “router bgp 100 address-family l2vpn-
evpn vni auto-create”
When enabling address-family l2vpn-evpn vni
auto-create mode then RT value will be
generated as following: (a) In case of 2bytes AS,
AN is VNI (b) In case of 4bytes AS, AN is VLAN.
There is no interoperability with old EVPN
releases when running VNI auto-create VS. VNI
auto-create mode. A workaround to interoperate
with old EVPN VNI auto-create is to switch to
manual VNI mode by using the following
commands: (1) Disable VNI auto-create mode:
(config) # no router bgp * address-family l2vpn-
evpn vni auto-create (2) Configure manual VNI:
(config) # router bgp * address-family l2vpn-evpn
vni * (3) Configure VNI RD: (config) # router bgp *
address-family l2vpn-evpn vni * rd * (4) Configure
manual VNI RT: (config) # router bgp * address-
family l2vpn-evpn vni * route-target both *

3.8.2102 RoCE, Port The default value of port-channel load-balance

Channel, LAG has been changed from "source-destination-mac"
to "source-destination-mac, source-destination-ip,
source-destination-port, l3-protocol, l2-protocol,
flow-label". This occurs only upon fresh
installations or after "reset factory". Upgrading
users will retain the old load balancing value and
show running-config will indicate this.

3.8.2008 Security As of version 3.8.20xx, California law SB-327 will

Enhancements be enforced so that Admin and Monitor
passwords must be typed upon initial
configuration. Default passwords will no longer be
created automatically. Nevertheless, the user will
be allowed to manually write in the default user
name and password (admin/admin or
monitor/monitor). The changes are implemented
in a manner that minimizes the impact on the
automation processes so that Zero Touch
Provisioning (ZTP) will continue to work as usual
and will not be affected by the new regulation.

3.8.2008 XPL API As of the September release of software version

Deprecation 3.8.2000, the XML user accounts will no longer
be supported and the XML gateway will be
closed. Access through XML will no longer be
available. Interfaces will only be available through
SNMP and JSON.

22
Release # Category Description

3.8.2008 Web UI Re-designed interface to enhance user

experience.

3.8.2008 IGMP The default IGMP version was changed from v2

to v3. This occurs only upon fresh installations or
after "reset factory". Upgrading users that are
running with IGMP v2 will not be affected and
show running configure will indicate this.

3.8.2008 LLDP LLDP is now enabled by default. This occurs only

upon fresh installations or after “reset factory”.
Upgrading users that have LLDP disabled will
leave LLDP disabled and show running configure
will indication this.

23
 5 Known Issues
5.1 General Known Issues
Table 6: General Known Issues
Index Category Description W/A

1 Breakout Cables The non-functional error N/A

“Invalid eeprom memory data”
appears in the “ethernet speed
and type” row when running
the command “show interface
ethernet transceiver” on a split
port.

2 CLI Command output filtering does N/A

not support the commands
"show log", "show puppet-
agent log", "show ib sm log",
and "show configuration text
files <file>".

3 CPU Temperature Switch reports high CPU

temperature in cold
environment (<10C).

4 Cables 40GbE passive copper cable Replace copper

of 5m length might experience cable with fiber
link rise issues when cable.
connected to some 3rd party
40GbE interfaces.

5 Chassis Duplicate notification is sent N/A

Management upon high temperature alerts,
both specific module alerts and
general “too high” alerts.

6 Configuration Merging two binary Use

Management configuration files using the “configuration
command “configuration text file apply”
merge” is not supported. instead.

7 Configuration Binary configuration files Save

Management saved from releases older than configurations as
3.6.1000 cannot be fetched to text files before
the switch and fail with a upgrading to this
decryption error. It is version.
recommended to create a new
copy of the bin configuration
file after upgrade in case
configuration backup is
needed.

8 Configuration Applying a configuration file of N/A

Management one system profile to another

24
Index Category Description W/A

is not supported from one

system type to another.

9 Docker If two docker images are Delete older

installed, both from same latest image or
distribution and both chosen download while
as “latest”, the command specifying a
“show docker images” may version.
display image name and
image version as “none”.

10 Ethernet Interfaces The “no” negation prefix is not Use the “no”
supported prefix in the CLI keyword as an
commands "interface ethernet infix (e.g.
<s/p> ipv4 port access-group "interface port-
nodhcprelay" and "interface channel <num>
port-channel <num> switchport no switchport").
mode".

11 Ethernet Interfaces, Auto-negotiation is not N/A

Speed, Auto- supported for 3rd party
Negotiation vendors which do not support
auto-negotiation for 40GbE
and 100GbE speeds.

12 JSON Sending JSON commands that Divide the 50K

exceed 50K lines, the JSON lines into
job gets stuck in the multiple JSON
background or gets timed out. requests.

13 JSON The command "show whoami" Run the

is not supported through JSON command
HTTP request. through the CLI.

14 Linux Dockers, When running “configuration Run the init

Configuration text apply” (with “docker no configuration
Management shutdown”), a container that is from the CLI
configured as init, may run session.
immediately (instead of waiting
for next boot).

15 Logging The warning “[pm.WARNING]: N/A

snapshots and sysdumps are
on separate partitions; space
constraints not thoroughly
enforced on sysdumps” may
appear if operating with an
encrypted file system. This
warning can be safely ignored.

16 Management It is not possible to create a N/A

Interface username called “ns”.

25
 Index Category Description W/A

17 Management Speed of mgmt0 interface is N/A

Interfaces shown as “UNKNOWN” when
working with VM.

18 Management Consecutive hostname Wait 25 seconds

Interfaces modification is not supported. before
reattempting to
modify the
hostname.

19 Management Switch systems may have an Generate a new

Interfaces expired HTTPS certification. certificate by
changing the
hostname.

20 Modules When using 5m splitter cable N/A

(P/N MC2609125-005), wrong
data is displayed under
transceiver information.

21 SNMP Request timeout should be set N/A

to at least 20 seconds since
initial table calculation requires
time.

22 SNMP The tables dot1dTpFdbTable N/A

and dot1qTpFdbTable support
up to 10,000 entries.

23 System If a faulty cable is used, rise N/A

Management time of other interfaces in the
system may be delayed.

24 Telemetry Splitting a port with telemetry Telemetry

configuration makes telemetry histograms may
non-functional and may cause be applied after
errors in log. a port is split
(unsplitting may
only be done if
no telemetry
configuration is
present).

25 Upgrade/Downgrade Downgrading back to 3.8.2204 N/A

after upgrading to 3.8.2306 will
save configuration
modifications from 3.8.2306.

26 Virtual Machine Virtualization connection might Connect through

fail when trying to connect text.
through graphics.

27 Virtual Machine For volume fetch, using a USB Use EXT3 USB
drive formatted with VFAT format.
causes errors in the log and
may require additional reboot

26
 Index Category Description W/A

for the USB to be registered

for virtual machine volume
usage.

28 WebUI Interactive CLI commands N/A

cannot be executed via
WebUI.

29 WebUI The following commands Run these

cannot be executed from commands
“Execute CLI commands” through the CLI
section on WebUI. instead.

– interface ethernet
<interface> module-type

– dcb priority-flow-control
enable

30 WebUI Enabling/disabling HTTPS Refresh the

while connected via HTTP to page or navigate
the WebUI may result in back using the
temporary loss of connection browser’s back
to the webpage. button.

31 WebUI Reversing the time clock can Clear the graphs

result in WebUI graphs’ data after setting
corrupted data. the clock.

32 WebUI Importing a configuration text Import the

file with commands that only configuration text
get enabled after running other file through the
commands is not possible CLI.
through the WebUI. For
example: the command “mlag-
vip” is only available after
running “protocol mlag”.

5.2 Ethernet Known Issues

Table 7: Ethernet Known Issues
Index Category Description W/A

1 802.1x PAE may send unneeded EAP- N/A

START messages when
authentication starts.

2 ACL The Maximum supported ACL N/A

scale numbers are 5k for mac-acl,
5k for ipv4-acl, and 4k for ipv6-acl.
Using existing configurations with a
higher scale, will prevent login via
SSH.

27
 Index Category Description W/A

3 ACL Configuring 6k ACL role on the N/A

switch leads to out-of-memory
errors when performing ‘reset-
factory’ action, during which the
switch is inaccessible.

4 ACLs A single ACL table is limited to up N/A

to 2K rules.

5 ACLs Packets dropped by the switch due N/A

to congestion or ACL rules are
added to the bad type counter of
the matching VLAN interface.

6 BGP When there are multiple BGP IPv4 If there are

or IPv6 sessions in different VRFs multiple BGP
running on switches and the sessions in
sessions have overlapped IPv4 or multiple VRFs
IPv6 peer addresses, only one with overlapped
password will be taken if different IPv4 or IPv6
passwords are configured. That is, peer addresses,
only one of the sessions will be use the same
negotiated with its correct password or do
password; the others may not not use a
reach the established state. password for the
sessions.

7 BGP When there are parallel multiple Use the same

Unnumbered BGP unnumbered sessions password on
running between two Onyx parallel multiple
switches, only one password will BGP
be taken if different passwords are unnumbered
configured. That is, only one of the sessions.
sessions will be negotiated with its
correct password, the others may
not reach the established state.
The limitation also applies to
sessions belonging to one or more
peer groups with passwords.

8 BGP, VRF When there are multiple BGP N/A

sessions in more than 3 VRFs
(including default VRF), it may take
around 2 minutes for a BGP
session to be established in a VRF
that is not the default.

9 CRC Replacing CRC field with N/A

timestamp is currently not
supported on Spectrum-based
systems.

28
Index Category Description W/A

10 DCBX DCBX auto-select type DCBX type

(IEEE/CEE) is not supported. should be
selected
manually.

11 DCBX When moving from DCBX CEE to After mode

DCBX IEEE, TLVs may be sent change, the
improperly. TLVs to be sent
must be reset.

12 DCBX Enabling LLDP triggers a faulty N/A

notation of PFC oper-state. This
status may be ignored.

13 ECN Configuration of minimum-absolute N/A

value smaller than 3K might cause
constant ECN tagging.

14 EVPN EVPN Centralized Gateway is not N/A

Centralized supported on Spectrum-based A0
Gateway systems. Use “show inventory”
command to see ASIC revision.

15 EVPN, IPv6 When Neighbor Suppression (NS) N/A

Neighbor is enabled, all incoming IPv6 NS
packets are handled by the switch
and are not transmitted via the
Tunnel. Even though these
packets are not entering the
Tunnel, they are still counted as
encapsulated on the tunnel and will
be shown when using "show
interface NVE" or specific vni
counters.

16 EVPN, IPv6 When Neighbor Suppression (NS) N/A

Neighbor is enabled, all incoming IPv6 NS
suppression packets are handled by the switch
and are not transmitted via the
Tunnel. Even though these
packets are not entering the
Tunnel, they are still counted as
encapsulated on the tunnel and will
be shown when using "show
interface NVE" or specific vni
counters.

17 Ethernet Not all discarded packets are N/A

Interface visible in "show interface ethernet
1/17 counters" command.

29
 Index Category Description W/A

18 Ethernet Interface indexes are wider than N/A

Interfaces 24-bit and do not fit into the data-
source ID field.

19 Ethernet The command "show interface" N/A

Interfaces, does not display 1GbE modules
Modules correctly.

20 HLL Changing the HLL duration No action is

configuration while the port is in needed because
stall state takes effect after a few once it exits stall
seconds of random delay. The next state, expected
time stall state is entered, it may system behavior
take a little less time (next HLL) or resumes.
up to 60 seconds.

21 Historical JSON output for the command N/A

Statistics “show stats sample interface-port-
channel data max-samples”
supports up to 800 max-samples.

22 IGMP A large amount of 'IGMP general Fix topology loop

queries' on L2 interface cause the for burst packets
message queue to overload that get into the
because of a topology loop. As a switch.
result, some control packets may
not be handled.

23 IGMP Snooping Fast leave always works on v3 N/A

reports even when “fast-leave” is
disabled.

24 IGMP Snooping IGMP snooping is limited to a N/A

single MC IP per MC MAC.

25 IGMP Snooping If the IP address of the IGMP Configure IGMP

querier is not specified explicitly querier IP
and no IP address is configured on address explicitly
the VLAN interface, the default or setup an IP
IGMP querier address is 0.0.0.0. address on the
required VLAN.

26 IGMP Snooping There is no possibility to have After enabling

more than one MRouter port on a IGMP snooping
single VLAN if IGMP snooping is on that VLAN,
disabled for that VLAN. the user needs
to re-add the
MRouter ports
on that VLAN.

27 IGMP IGMP Snooping over VXLAN is not N/A

Snooping, supported on Spectrum-based A0
VXLAN systems. Use “show inventory”
command to see ASIC revision.

30
Index Category Description W/A

28 IGMP snooping, IGMP Snooping does not work on Run the

MSTP, STP a VLAN which is added to an "spanning-tree"
MSTP instance if STP is disabled command via the
and is then changed to MSTP CLI in config
mode. mode.

29 IPv6, PTP PTP IPv6 has an incorrect path N/A

delay for 50GbE interface speed.

30 LAG On rare occasion, adding a port to N/A

a LAG may fail after setting no ip l3
force.

31 LAG Spanning tree configuration Configure the

performed on a LAG is not retained same values on
by ports when they are removed the port before
from the LAG. Adding back ports to insertion to the
a LAG that has spanning tree LAG.
configuration will fail due to
mismatch.

32 LLDP The command “show lldp statistics” N/A

displays incremented “TLVs
Unrecognized” inaccurately.

33 LLDP LLDP ifIndexes do not match the Use the ifDescr

ones published in the ifTable in and fields.
IEEE 802.1.

34 MLAG During manual reconfiguration

MLAG Port-channel (disabling and
then enabling ports in the MLAG
port-channel), small traffic loss
may be observed.

35 MLAG MLAG does not work when Work with IPv4

configuring IPL with IPv6 address. source address.

36 MLAG Redundant links between the Remove the

MLAG peers do not converge redundant links
properly. between peers
(just like a
standard
configuration of
an MLAG
cluster) to only
enable the IPL
between peers.

37 MLAG Changing MLAG port configuration N/A

on master MLAG switch (higher IP
over the IPL) before software
upgrade is finished might cause
traffic flood (BC and MC).

31
 Index Category Description W/A

38 MLAG Changing STP mode affects traffic User should

and may cause MLAG to stop disable MLAG
working for an interval of 21 prior to changing
seconds until the new STP the STP mode.
topology is built.

39 MLAG MLAG VIP is limited to 2 switches. N/A

If more than 2 switches connected
to the same MLAG VIP, MLAG
behavior is not anticipated.

40 MLAG MLAG configuration (including N/A

mlag-port-channel configuration)
must be identical in an MLAG pair.

41 MLAG Fast LACP cannot be used on the Cannot use fast

link partner when working with over LACP on the link
40 MLAG port-channels and over partner when
650 VLANs. working with
over 40 MLAG
port-channels
and over 650
VLANs.

42 MLAG EVPN In specific cases, traffic loss occurs When using

on MLAG peers when using MLAG MLAG EVPN
EVPN configuration. configuration,
enable boot
delay of 240 on
each on the
MPO member
ports on both
MLAG peers in
order to
eliminate traffic
loss upon reload.

43 MLAG, EVPN IPL flapping on MLAG cluster Perform "clear

(Which also has EVPN mac-address-
configuration) while traffic is table dynamic"
running may lead to a situation to clean out all
where the Slave switch has one or dynamic entries
more non-ageable entries in FDB from FDB table
that do not appear on the master and re-learn.
switch.

44 MLAG, STP Spanning tree domains do not Disable the

converge correctly on the rare relevant ports
occasion that an MLAG peer before adding
running MSTP is connected to a VLANs to MST
different region and needs to act instances.
as a border switch for no MPOs
ports.

32
Index Category Description W/A

45 Monitor Session Monitor session commands Allow one

entered in rapid succession (i.e., second or more
pasting a script at once), may between monitor
cause errors. session
commands.

46 Monitor Session Changing the port type from local Reapply the
analyzer to remote analyzer or deleted
regular port (not a monitor session configuration
destination) deletes some user after the port
configuration such as admin state type change.
and speed.

47 OpenFlow Immediately configuring Wait a few

flows/group after enabling "protocol seconds after
openflow" will cause errors in the running "protocol
log to appear and configuration will openflow" before
not be applied. applying
flows/groups.

48 OpenFlow When setting flows with the same N/A

match keys, only the last flow will
be considered. If the last flow is
deleted, other flows configured
with the same match key will also
be deleted.

49 OpenFlow For OpenFlow to be enabled, at N/A

least one LAG port should be
available.

50 OpenFlow When clearing the maximum Reboot the

number of flows added by the switch.
controller, the switch may get
stuck.

51 OpenFlow "no protocol openflow" fails if all Use no more

supported tables are used. than 40 tables
for OpenFlow.

52 OpenFlow 1.3 OpenFlow must be disabled before N/A

running the command “no ip l3
force”.

53 OpenFlow 1.3 Using “TRAP” in FDB floods the N/A

packets to all ports.

54 OpenFlow, OVS Configuring an OVS server port Change OVS

which is already in use, causes server port
errors in the log. configuration to
an unused port.

55 OpenFlow, OvS The count of rules is based on the N/A

key size. Minimum key size is 18
on Spectrum and 9 on Spectrum-2
and Spectrum-3. The algorithm

33
 Index Category Description W/A

always uses 18 as a minimum key

size, therefore on Spectrum-2 and
Spectrum-3 systems, the
maximum number of rules will not
match the real value for a 1 small
key.

56 PAM4, Auto- When connecting a Mellanox-to- N/A

Negotiation Mellanox in PAM4 using DAC
cables, auto-negotiation should
always be enabled.

57 PTP PTP over port-channel is not N/A

supported on Spectrum-2 and
Spectrum-3 systems.

58 PTP On very rare occasion, link down N/A

when PTP is enabled may lead to
the following error message in the
log:
"[ptp4l.ERR]: [.....] port 1 []: bad
message".

This error can be safely ignored.

59 PTP On occasion, when disabling N/A

protocol PTP the following error
message may appear in the log:
"[phc2sys.ERR]: [...] poll failed".
This error can be safely ignored.

60 PTP, Port Adding L2 interface to port channel Enable PTP on

Channel after PTP was enabled for this L2 L2 interface only
interface will lead to fault on PTP. after adding the
L2 interface to
port channel.

61 Port Mirroring When port mirroring congestion is N/A

configured to “drop-excessive-
frames” (best effort), while using
the same analyzer port for more
than a single mirroring session,
that port could drops packets from
all flows not according to their
configured priorities.

62 Port Mirroring Using “force” in the command N/A

“monitor session 1 destination
interface ethernet 1/4 force” should
be done after configuring all other
related configuration for that
monitor session or wait 1 second

34
Index Category Description W/A

before issuing further configuration

on that monitor session.

63 Port, Split, If a port is split into 2 or 4 and one To monitor

Monitor Session of the split ports other than the first destination, use
is used as a monitor session the first sub-port
destination and the port is later in the split. If a
removed from the session and different port
unsplit, then errors are logged after was already
save and reload. used, do not
save the
configuration
unless the split is
already set on
this port.

64 Puppet Agent Description parameter is not N/A

supported on L2 interfaces.

65 Puppet Agent Speed parameter is not supported N/A

for LAG interfaces.

66 QinQ QinQ is not supported over N/A

VXLAN.

67 SNMP LLDP MIB OID lldpRemPortDesc N/A

is not supported.

68 SNMP, VTEP On power cycle, part of the N/A

Certification hardware gateway test from the
VTEP certification test suit, the
system appears up, but does not
respond to SNMP uptime requests.

69 Spanning Tree Setting spanning tree port type to N/A

“network” (interfaces ethernet <if>
spanning-tree port type network)
does not work.

70 Spanning Tree Moving from one STP mode to N/A

another while the system is
functional may lead to error in the
log which can be ignore.

71 Spanning Tree BPDUs are sent from both the N/A

superior and the inferior switches.

72 Spanning Tree, RPVST is not interoperable with N/A

RPVST RPVST+ on Cisco’s Nexus system
when using TAG VLAN.

73 Spectrum-2, HPE Spectrum-2 based M-Series N/A

10G BaseT switches (SN3700cM) support only
Modules the HPE R0R41A/B 10G BaseT
transceiver with the QSA28 HPE
845970-B21.

35
 Index Category Description W/A

74 Spectrum-2, At 200GbE speed, traffic drop N/A

Cables, Speed occurs between SN3700 and IXIA
K400.

75 Spectrum-2, Currently, raising the link at 10GbE N/A

Cables, Speed link speed is not supported when
using 200GbE optical cables.

76 Spectrum-2, Tunnel encapsulated counters is N/A

EVPN not increased when ARP packets
are forwarded to tunnel.

77 Spectrum-2, COLORZ module link up time N/A

Modules, increased to 85 seconds.
COLORZ

78 Spectrum-2, Testing SN3700 200GbE with Ixia N/A

SN3700, AresONE requires additional steps.
Cables, Contact our technical support for
Speeds, specific settings.
Testing, Ixia

79 Spectrum-2, On SN3700 and SN3700C

SN3700, Port systems, when splitting ports, the
Split following error may appear:
"[issd.ERR]:
getIssindexFromIfindex can’t find
in port_indexes_ht the entry for
ifindex 134238448"
This error can be safely ignored.

80 Spectrum-2, On SN3700C systems, the N/A

SN3700C maximum allowed power class for
QSFP ports is 4.5W for ports 1-2
and 21-31 and 3.5W for all other
ports.

81 Spectrum-3 The following capabilities are N/A

currently not supported on
Spectrum-3-based systems:

-MPLS

-Fast boot

-Split to 8

-PSU hot plug/unplug

82 Spectrum-3, In Spectrum-3 while using Optic N/A

PAM4, Optic cable MFS1S50-H003E in Split
Cables 4x1 mode in PAM4, when one port
is toggled, all other 3 ports go
down.

36
Index Category Description W/A

83 Spectrum-3, For egress mirroring on a 400GbE N/A

Port Mirroring port the monitor session
congestion mode should be set to
“drop-excessive-frames” as the
“pause-excessive-frames” mode
does not work in this case.

84 Spectrum-3, In SN4600C systems, while using N/A

SN4600C, 100GbE auto-negotiation mode
Auto- with copper cables, the link up time
Negotiation may takes up to 15 seconds.

85 Spectrum-3, In Spectrum-3 devices, boot can N/A

SN4700 take up to 75 seconds.

86 Spectrum-3, On Spectrum-3 systems, links are N/A

Split not stable on split interfaces that
were split using qsfp-split-4 or
qsfp-split-2 command.

87 Speeds, Cables When connecting Spectrum-2- Use Force FEC

based systems with Spectrum-3
based systems using 25GbE
speed with LR module and auto-
negotiation, there is no link up due
to incorrect FEC selection.

88 Traffic, Port While there is traffic to a port, N/A

when either running "no shutdown"
or reloading a port that has
incoming traffic, the counters of the
discard packets will be counted for
3 seconds.

89 Unsupported The following are currently N/A

Protocols unsupported:

• SSH server

• TCACAS

90 VRF, IPsec IPsec may not work properly if Remove mgmt

mgmt VRF is created. VRF if IPsec
feature in use.

91 VRF, Upgrade In version 3.9.2006, "mgmt" VRF is Remove "mgmt"

considered a special VRF for VRF from the
switch management purposes. If, configuration
before the upgrade, "vrf definition before the
mgmt" is defined in the running- upgrade.
config and then an upgrade to
3.9.2006 is performed with "no
boot next fallback-reboot enable",
the boot will be in an inconsistent
state.

37
 Index Category Description W/A

92 WJH WJH "t trap_probability" is the N/A

probability to extract a packet from
the stream of packets that were
dropped by HW. If the packet drop
rate is low (small number of
packets) then WJH events are not
triggered.

93 WebUI, IGMP When inputting a wrong value in N/A

Snooping the querier IP address, no error
message appears.

94 sFlow There is an invalid agent-ip in first N/A

exported sFlow packet. All other
packets are sent properly.

95 sFlow Errors are seen when defining N/A

sFlow session from two WebUI or
CLI sessions simultaneously.

96 sFlow Discard counters are not reported N/A

to the collector.

97 sFlow The discarded packets counter in N/A

sFlow samples may not be
accurate and may expose a skew
of one second.

5.3 IP Routing Known Issues

Table 8: IP Routing Known Issues
Index Category Description W/A

1 BFD User cannot clear BFD counters. N/A

2 BFD A BFD session is not signaled as N/A

removed on a static route which is
set to admin down.

3 BGP If there are multiple IP addresses Do not use

configured on an IP interface, “peer-group”,
when update-resource is enable “update-
configured on a peer group, there source” per
is no guarantee that the primary individual peer.
IP address gets used as the
source IP address to establish
peer relationship.

4 BGP IGP metric does not influence N/A

route selection where routes are
learned from different IGPs.

38
Index Category Description W/A

5 BGP A peer does not inherit the Run “shutdown”

update-source parameter from its then “no
group configuration. shutdown” on the
peer.

6 BGP The command “no neighbor <ip- Delete the

address> local-as” is not neighbor with “no
functional. neighbor {<ip-
address>}
remote-as” and
restore the
neighbor without
the command
“neighbor local-
as”.

7 BGP When running both BGP and N/A

OSPF, changing the configuration
of the command “router bgp <AS>
<external distance> <internal
distance> <local distance>”
requires running “clear ip bgp all”
afterwards.

8 BGP AS-path request/set are limited to N/A

128. When more are received,
TCP FIN is sent and connection to
the peer is lost.

9 BGP The command “bgp aggregate- N/A

address summary-only” does not
silence aggregate-route re-
advertisements.

10 BGP Using the command “clear ip bgp If route-refresh is

<ip> soft in” mandates enabling not supported on
route-fresh on the peer. the peer device,
use the
command “clear
ip bgp <ip> in”
instead.

11 BGP Aggregate overlapping is not N/A

supported.

12 BGP Route aggregation configuration Delete route

cannot be modified. aggregation and
create a new
configuration.

13 BGP When 1 VRF loopback interface N/A

Unnumbered alias is set, it will copy addresses
from the target loopback, which is
a duplicated address. The
following warning log will appear:

39
 Index Category Description W/A

Jan 8 13:38:46 mtbc-onyx-01-

2410 snmpd8543:
[snmpd.WARNING]: Duplicate
IPv4 address detected, some
interfaces may not be visible in IP-
MIB

14 BGP, IPv6 When enabling or disabling IPv6 N/A

Routing routing, BGP sessions of the
related VRF will be restarted.

15 Bidirectional If there is more than one PIM Set all the

PIM mode used in the same domain, switches in PIM
the switch will also learn the PIM domain to work
routes of the PIM modes different is the same
than it. mode (Bidir/SM).

16 Bidirectional The switch forwards all multicast N/A

PIM traffic for a group even though
only a particular source is
requested via IGMPv3. The "show
ip mroute" command will show
that only a particular source was
requested.

17 Bidirectional Changing Load Balance mode Use "clear ip

PIM, Load from LB to Non-LB will not clear mroute"
Balance the routes that are no longer command to
required. Those routes will remain clear the desired
as long as there is traffic on those routes.
routes.

18 Counters In IP Routing mode, multicast or N/A

broadcast packets (such as ARP)
may be counted as “RX discard”
in case only one interface is a
member of the VLAN. Such
packets reach the CPU for
processing. The progressing
counter can be ignored.

19 DHCP DHCP-client is not functional. Use static IP

instead.

20 IGMP Querier Running the command “show ip N/A

igmp groups” does not print
“source” and “expires” parameters
in its output.

21 IP Routing When TFTP source IP is enabled, N/A

ICMP redirects are not sent.

22 IP Routing IPv4 link local addresses N/A

(169.254.0.0/16) are not
supported. Addresses should be

40
Index Category Description W/A

changed before upgrade, if not

changed, they will be removed
from the interface.

23 IP Routing It is required for neighbor entries N/A

on an IP interface to be fewer
than 25K.

24 Interfaces ICMP echo packets are not ICMP packets

counted as part of the VLAN are counted in
interface counters. the interface L2
counters.

25 LLDP A port connected to a router-port N/A

receives from it an LLDP packet
containing IPv4 and IPv6
management addresses.
However, the IPv6 address is not
included in the LLDP MIB as a
remote management address.
Only the IPv4 remote
management address is
published.

26 Multicast Multicast packets sent over VLAN Disable either

with enabled EVPN/VNI and EVPN/VNI or
IGMP snooping at the same time IGMP snooping
are not sent over VXLAN network for that VLAN.
and are only sent to local
receivers.

27 Multicast, In Spectrum-based systems, N/A

VLAN, IGMP multicast packets sent over VLAN
with enabled EVPN/VNI and
IGMP snooping are flooded to all
VTEPs and flooded to VLAN after
VxLAN decap to all VLAN
members.

28 OSPF Reverse packet filtering is not N/A

supported. When misconfigured,
traffic may pass on an unintended
subnet.

29 OSPF Removing a static ID using the N/A

command “no router-id” requires
at list one VLAN interface to be
configured.

30 PIM PIM-SM MRoute is not created N/A

when incoming multicast data
traffic is not mapped to RP, or
when RP is not reachable.

31 PIM In PIM-SM, the non-DR router N/A

displays interfaces with received

41
 Index Category Description W/A

IGMP reports under “outgoing

interface list”. Traffic is forwarded
from DR router.

32 PIM Next-hop multi-path: Multicast N/A

traffic continues being forwarded
for 2-3 minutes after deleting PIM
configuration from an interface
which is part of port distribution.

33 PIM When a static IP route is Configure static

configured after an IP MRoute MRoute after
(with the same prefix), the static configuring static
route has higher priority. route.

34 PIM MRoute entries learned via PIM N/A

remains when the interface is
shutdown until the timer expires.

35 Router Port A trunk port which is configured Configure the

as router port does not receive port to access
traffic to the host interface. mode (CLI
command
“switchport mode
access”) before
making it a router
port.

36 Router Port Setting 802.1Q mode on a router N/A

port does not function properly.

37 Router Port ICMP redirect is not supported N/A

over router port.

38 VRF Any VRF can be monitored with N/A

sFlow, but a collector can be on
the management VRF only.

39 VRRP Using the same Virtual IP (VIP) as N/A

the VLAN interface is not
supported.

42
6 Bug Fixes
Table 9: General Bug Fixes
Index Category Description Fixed in
Version

1 BGP, PIM If a PIM DF is from a BGP path, the 3.9.21xx

metric value in "show ip pim interface
DF" is a very large value. This has no
functional impact to the system.

2 Counters Sent traffic in certain size is wrongly 3.9.21xx

increase to a smaller size counter,
because port extended counter has a
counter for sent traffic per packet-
size range.

3 Counters, Bulk On Spectrum systems, the allocation 3.9.21xx

Flow of a large bulk group of flow counters
may fail once the number of allocated
flow counters reaches 6K.

4 ECMP When operating in Async mode, if an 3.9.21xx

ECMP container is deleted
immediately after a large number of
routes referencing it is deleted, a
transient error can be seen where the
ECMP container fails to delete
correctly.

5 IGMP When receiving malformed IGMP 3.9.21xx

packets memory leak occurs. In turn,
this may affect the handling of IGMP
control packets.

6 IP Filter IP Filter does not display the In-intf 3.9.21xx

on "show ip filter."

7 NTP, Logging NTP tries to open sockets on all L3 3.9.21xx

interfaces with IP addresses, but fails
on bind, because NTPD runs in a
different context than the L3 interface
VRF context.

8 PTP, CRC PTP does not work properly on the 3.9.21xx

ports which have disabled CRC
check and recalculation.

9 SN3700, On occasion, link down is 3.9.21xx

Module, experienced with INPHI COLORZ
COLORZ PAM4 100G optic cables on SN3700
systems.

10 SSH SSH session cannot be established 3.9.21xx

with a configured listen interface for

43
 Index Category Description Fixed in
Version

the SSH server ("ssh server listen

interface").

11 Shared Configuring shared headroom on the 3.9.21xx

Headroom fly may, on occasion, cause the chip
to get stuck.

12 Spectrum, IPv6 On Spectrum systems, when running 3.9.21xx

IPv6 traffic on eight 100GbE ports at
full speed (where 50K IPv6 prefixes
are advertised), packet loss may
observed on several of the ports.

13 Spectrum-2, Failed to add more than 20 actions 3.9.21xx

OpenFlow per one Openflow rule.

14 Spectrum-3, In Spectrum-3 systems, intermittent 3.9.21xx

Cable link up is experienced with Molex
DAC cable (1110409084) after cable
reset.

15 Spectrum-3, In Spectrum-3 systems, there is a 3.9.21xx

Cable failure to link up Luxshare
(L7KQF003-SD-R) cable.

16 Upgrade, MTU After upgrading to version 3.9.2006, 3.9.21xx

ports with MTU 1500 cannot be
assigned to IPL because the default
MTU value changed to 9216.

17 WJH, Ports In the WJH library, local port 128 was 3.9.21xx
considered an invalid port.

18 Cables, Speeds, In Spectrum, after changing speeds 3.9.2000

Firmware the firmware may get stuck.

19 Driver, MTU When sending a control packet that is 3.9.2000

larger than the port MTU, instead of
control-sending queue will get stuck,
driver will return an error.

20 Initialization Occasionally errors are observed in 3.9.2000

CLI and portd during initialization,
after which the switch does not
function normally due to a rare timing
issue.

21 JSON Sending multiple JSON requests 3.9.2000

simultaneously can occasionally
cause one of the requests to hang
and return an error response even
though it was executed successfully.

22 JSON On rare occasion, when the switch 3.9.2000

receives an asynchronous JSON
request while the system is handling

44
Index Category Description Fixed in
Version

many additional operations, the

request can sometimes get lost and
will not be executed.

23 Link, Operational When link drops from Active to 3.9.2000

Speed Polling, operational speed will not be
cleared (i.e., it will continue showing
the last active state).

24 Management Error string displays an unclear error 3.9.2000

Interfaces message for RADIUS servers whose
ciphers are not supported in secure
mode.

25 OpenFlow In deinitialization phase, OpenFlow 3.9.2000

table was removed before table rules
were removed causing assert. As
result, a core file was generated and
reboot/boot took longer than usual.

26 OpenFlow, When the OpenFlow SDN controller 3.9.2000

VLAN is connected to the switch via router
port without VLAN, not all the ports
under the group will work.

27 PTP, Logging Configuring PTP logging level to 3.9.2000

debug starts a phc2sys process in
verbose mode that overloads the log.

28 SNMP "ConnectorPresent" at SNMP does 3.9.2000

not work well.

29 SNMP, VTEP BFD failover test from the VTEP 3.9.2000

Certification certification test suit fails, as ARP
request is duplicated by the hardware
VTEP.

31 mDNS mDNS announcements are being 3.9.2000

prevented from operating inside the
container on the switch.

32 ACL When traffic is captured on an ACL 3.9.1900

rule with log enabled, the rules
always appear in the syslog with
deny action (even when permit is
set).

33 MLAG In case MPO is set to LACP 3.9.1900

individual mode and a member in
MPO is active, a repetitive misleading
log appears in messages file.

34 MLAG In some scenario, when using Break 3.9.1900

before making an MLAG cluster
could lead to a state where the MPO

45
 Index Category Description Fixed in
Version

is set to flood block enabled even

though the port is up.

35 Monitor Session From release 3.9.1000 and up, 3.9.1900

deleting a monitor session that had
no destination port did not work.

36 OpenFlow When setting up a group where one 3.9.1900

of the ports contains a modifier and
others do not, all ports get the traffic
with the modification.

37 OpenFlow The internal error occurs when 3.9.1900

modifying the group ALL with output
ports which are in use by added
flows.

38 PIM, IGMP, Enabling recirculation port for 3.9.1900

Recirculation telemetry will produce IGMP fail log
Port messages when PIM is enabled on
VLAN interface 1.

39 RoCE, Multicast RoceV1 Multicast packets are being 3.9.1900

discarded in the ingress port.

40 Static IP, DHCP In cases where a management 3.9.1900

interface has both static IP and
DHCP configured, reloading the
switch will not assign an IP if no
DHCP received.

41 ZTP, BGP IP Exported text config files which 3.9.1900

Prefix List include BGP IP prefix list
configuration cannot be applied via
ZTP because the 'cli session'
command is not supported in the ZTP
scenario.

42 ACL Setting an ACL group to be exactly 3.9.1600

the same as it was before may
corrupt the group and cause traffic
misbehavior.

43 CLI, Port Having a pipe symbol ( | ) in the port 3.9.1600

Description description brakes the CLI output of
two commands: "show interfaces
status" and "show interfaces ethernet
description".

44 ECMP When empty ECMP containers (such 3.9.1600

as unresolved next hops) are
updated but still remain empty, an
error will occur (relevant only if an
ACL action points to this ECMP).

46
Index Category Description Fixed in
Version

45 Histograms In Spectrum-3 systems, Latency 3.9.1600

Histogram may have incorrect values
if a port is configured to an 8x width.

46 LAG MSTP state is not updated correctly 3.9.1600

for ports in certain VLAG scenarios.

47 MAC Address When using show mac-address-table 3.9.1600

with MAC address in the following
format "xxxx.xxxx.xxxx" it caused the
CLI session to crash.

48 MAGP Host in MLAG cannot resolve ARP of 3.9.1600

base VLAN interface of MAGP which
belongs to non-default VRF.

49 Modules In some cases, error may occur when 3.9.1600

using MCION query on a module with
types "sfp_rx_los_soft_en" or
"sfp_tx_fault_soft_en" that enable
rx_los/tx_fault read from the memory
map.

50 OpenFlow The OpenFlow rule does not work 3.9.1600

properly after a group is modified if
the rule contains action group and
additional actions.

51 PIM "show ip pim rp-hash "238.0.1.0" 3.9.1600

appears as an invalid group address.

52 SNMP, Listen SNMP process may cause flapping 3.9.1600

Interface, Static when mgmt0 with static IPv6 address
IPv6 (DHCP disabled on mgmt0) is
configured as listen interface.

53 Spectrum-2, Puppet agent is not supported on 3.9.1600

Spectrum-3, Spectrum-2 and Spectrum-3
Puppet Agent systems.

54 Upgrade, Split, After upgrading to 3.9.13xx, 3.9.1600

Speeds configuring split-ports to 100GbE or
50GbE causes the configuration to
break.

55 WJH When WJH lib is enabled and WJH 3.9.1600

ACL is enabled, in very rare cases,
undefined trap behavior may occur.

56 WebUI, Auto- Configuring auto-negotiation through 3.9.1600

Negotiation web UI may fail when choosing
unsupported speed option.

57 Spectrum-2, ECMP hash on Spectrum-2 results in 3.9.1304

ECMP multiple next-hops for a single ping
session.

47
 Index Category Description Fixed in
Version

For non-TCP/UDP packets, disable

hashing on L4 source port for correct
LAG/ECMP load balancing.

58 Spectrum-3, 10GbE link up time, when using 3.9.1304

SN4600C, SN4600C with ConnectX-4, may take
Cables, Speeds up to 2 minutes.

59 ACL Configuring the ACL table with bind- 3.9.1300

point RIF does not get configured as
expected after switch rebooting the
switch.

60 BGP BGP is unable to establish sessions 3.9.1300

configured in different VRFs (only
one VRF will establish BGP session).

61 BGP, Scale Spectrum-2 and Spectrum-3 based 3.9.1300

systems may run into a condition
where the SDK process consumes
very high memory when running BGP
with Max Scale.

62 DHCP Relay When configuring DHCP relay 3.9.1300

downstream interfaces and then
immediately configure many (etc
>100) IP interfaces, DHCP relay may
not work properly.

63 ECMP, CPU When editing a MAC that points to a 3.9.1300

Utilization new ECMP with a next-hop that
points to another ECMP, the SDK
operation does not end.

64 EVPN, IPv6, Enabling Neighbor Suppression 3.9.1300

Neighbor leads to IPv6 link local although IPv6
Suppression is disabled. As such, EVPN
advertises the Default Gateway.

65 JSON Using "show interface status" over a 3.9.1300

JSON request repetitively, may
consume high memory overtime.

66 Logging Monitor Because the "logging monitor" 3.9.1300

commands use "echo \rMESSAGE\n
> TTY" for every logged-in user, the
CLI prompt is overwritten by
MESSAGE, making it appear as if the
prompt was never sent.

67 Recirculation "show interfaces ethernet link- 3.9.1300

Port diagnostics" reports an error in JSON
API web UI if a recirculation port is
defined.

48
Index Category Description Fixed in
Version

68 Recirculation Attempting to perform invalid 3.9.1300

Port configuration on a recirculation port
(e.g., changing speed or MTU, had
no effect, but no error message was
displayed to user. Now, an error
message is displayed).

69 SSH Only deprecated cipher in SSH was 3.9.1300

supported. Added support for new
ciphers.

70 STP Failover STP Failover Latency on 1GBASE-T 3.9.1300

Latency, is longer than expected (up to 250
1GBASE-T msec) in Spectrum-based systems.

71 Shared Buffer In some cases, wrong reserved 3.9.1300

buffer and descriptor values were
configured for ports. This could cause
some traffic loss in cases of
momentary congestion that should
have been absorbed by those
buffers.

72 Spectrum-2, On Spectrum-2 systems and up, PTP 3.9.1300

PTP, WJH cannot work with WJH enabled. As
WJH is enabled by default, it must be
disabled before PTP is enabled.

73 Spectrum-2, Telemetry threshold events stop 3.9.1300

Telemetry being generated after the LAG port
Threshold member changes.

74 Speeds, Port, The “Operational FEC” value is “auto” 3.9.1300

Stress instead of the real FEC on some
speeds if the input speed does not
match the actual port speed.

75 Streaming, MTU, In Streaming Settings, the events 3.9.1300

TTL "TTL value is too small" and "Packet
size is larger than MTU" (under
Forwarding > L3) are always
streamed even if configured
otherwise.

76 VRF, BPF Cannot use BPF filters due to default 3.9.1300

Linux restrictions for maximum
amount of locked memory per
process. Affected utilities include
those which run in a VRF context
such as tcpdump, ping, etc.

77 VXLAN, Flood On Spectrum-based systems, on 3.9.1300

some occasions, traffic that is flooded
to the tunnel through an MC

49
 Index Category Description Fixed in
Version

BRIDGE_MC container may be

dropped.

78 WJH The following changes have been 3.9.1300

made when another application takes
control of WJH.

"show what-just-happened status":

Before change: WJH groups

appeared with "Operational status"
as Enabled.

After change: If another application

(e.g. NEO) takes control of WJH, the
"Operational status" will be Disabled

"show what-just-happened <drop-

reason-group> " will no longer try to
take control of WJH. Running the
show command while another
application controls WJH will now
provide no output.If the OS needs to
take control of WJH, the command
must be disabled the flow must be
enabled again.

79 WJH WJH port down reasons is 3.9.1300

inconsistent with the PDDR register.

80 WJH, Spectrum On Spectrum-based systems, when 3.9.1300

what-just-happened buffer drop is
enabled and while WJH Telemetry is
activated, executing the "show what-
just-happened" command ends with
the following error: "% Request
failed; sdkd subsystem is busy".

81 ACL, Binding Multi-groups linked together do not 3.9.1000

bind correctly when using multi-
binding.

82 BGP For a IPv6 route learned from a BGP 3.9.1000

Unnumbered, unnumbered peer, the prefix and
IPv6 nexthop fields of the output of "show
ipv6 bgp <prefix/mask> longer-
prefixes detail" are incorrectly
displayed. The nexthop should be the
IPv6 link local address of the BGP
unnumbered peer.

This is a display issue and will not

affect the forwarding table.

50
Index Category Description Fixed in
Version

83 BGP For an IPv6 route learned from a 3.9.1000

Unnumbered, BGP unnumbered peer, the nexthop
IPv6 field of "show ipv6 bgp <prefix/mask>
detail" is displayed as a IPv4 address
169.254.101.101 instead of the link
local address of the unnumbered
peer.

This is a display issue and will not

affect the forwarding table.

84 EVPN In some instances, when ARP 3.9.1000

suppression is enabled in an overlay
network that uses IPv6, ND packets
get duplicated.

85 Ethernet IPv4 subnet /31 is not functional. 3.9.1000

Interfaces

86 Ethernet Traffic does not pass through the 3.9.1000

Interfaces NVE interface because there are no
routes in "Local" Linux routing table.

87 IGMP Snooping Splitting a port which has a static 3.9.1000

IGMP Snooping group causes an
error.

88 Link While using 3rd party FPGA with 3.9.1000

Spectrum-based systems, 50GbE
RSFEC does not raise a link while
using auto-negotiation or force mode.

89 Local Analyzer, Configuration write immediately (less 3.9.1000

Monitor Session, than one seconds) after configuring
Destination Port destination port of monitor session,
could result in failure to remove the
destination port after reload.

90 Login Flooding Using telnet server on a switch in 3.9.1000

combination with login flooding may
lead to unexpected reboot.

91 MLAG MLAG process may crash in extreme 3.9.1000

scenarios.

92 MLAG When configuring load interval on 3.9.1000

MPO interface, running "show
running config" will display errors.

93 MLAG MLAG VIP is deleted when 3.9.1000

management interface is flapping.

94 MLAG, JSON When hostname is bigger than 20 3.9.1000

characters, "show mlag-vip" did not

51
 Index Category Description Fixed in
Version

display the output correctly when

using JSON print.

95 Monitor Session It takes a few seconds after the CLI 3.9.1000

prompt returns for a monitor session
configuration change to be updated
in the database.

96 OvS, OpenFlow Adding a large amount of flows which 3.9.1000

differ only in the "priority" field caused
OvS to crash.

97 OvS, OpenFlow The SDK errors occur the OvS in the 3.9.1000
process of modifying OpenFlow
group after removing flows which
contain the group's "action" field.

98 PBS When a high scale of PBS multicast 3.9.1000

entries are used (with a high scale of
flood vectors), some PBS multicast
packets may be flooded instead of
being sent to their intended ports.

99 PTP The message "clockcheck: clock 3.9.1000

jumped backward or running slower
than expected!" appears in the log
due to improper clock check
procedure.

100 PTP, Control When PTP shaper is enabled, traffic 3.9.1000

Packets rate of over 95% might cause control
packets loss.

101 PTP, Spectrum On Spectrum-based systems, on rare 3.9.1000

occasions, timestamp event of an
ingress packet arrives before the
packet itself and, eventually, the
process of matching between the
packet and timestamp takes place
only when the next packet arrives.

102 Port Mirroring Deleting a monitor session 3.9.1000

immediately (within a second) after
configuring destination port may
cause errors.

103 Recirculation If all configurations are not cleared 3.9.1000

Port before configuring the recirculation
port, the configurations cannot be
changed after configuration.

104 RoCE When configuring a default mapping 3.9.1000

of switch-priority to traffic-class,
configuration sticks when enabling
RoCE.

52
Index Category Description Fixed in
Version

105 RoCE, Traffic When changing the traffic pool type 3.9.1000
Pool of traffic pool 'roce-reserved' that is
dedicated to RoCE traffic, setting a
type that matches the current RoCE
mode default will not remain when
switching to a different RoCE mode.

106 RoCE, Trust When enabling RoCE, the Trust 3.9.1000

Mode Mode value becomes the current
default instead of being retained
when disabling RoCE.

107 SNMP, QoS, OS memory leak detected when 3.9.1000

MIB querying QoS MIB.

108 Security Regeneration of certificate failed if 3.9.1000

the switch was unable locate it.

109 Spectrum-2, Connecting to ConnectX-5 adapter 3.9.1000

BER, Cables card with copper splitter cable
MCP7H50-V001R30 in 100G
(2x50GbE NRZ) FC-FEC/NO-FEC
causes high BER.

110 Spectrum-2, Link flaps in 200GbE with AOM Optic 3.9.1000

Cables cable MMA1T00-VS.

111 Spectrum-2, Split mode with one PLL module 3.9.1000

SN3700C, Split (MFS1S50-Vxxx) is not functional in
Port SN3700C systems.

112 WEB UI, Speed, Configuring any speed from Web UI 3.9.1000
Auto-Negotiation automatically sets the auto-
negotiation flag to "yes".

113 WJH On very rare occasions, driver 3.9.1000

crashes while reading WJH packets
on Spectrum systems.

114 WJH Without any traffic or events on the 3.9.1000

idle system, the CPU load is
constantly above 4%.

115 WJH, Port Configuring 3 monitor sessions and a 3.9.1000

Mirroring, recirculation port on Spectrum
Monitor Session systems or configuring 8 monitor
sessions and enabling WJH buffer
drop on Spectrum-2 systems, caused
internal errors. The operations that
exceed system resources are now
blocked in the CLI.

116 Web UI, debug Multiple debug dumps may be issued 3.9.1000
dump simultaneously, causing the switch to
crash on some occasions.

53
 Index Category Description Fixed in
Version

117 WebUI Some monitor session commands 3.9.1000

may return "Operation in Progress"
error.

118 LAG, Multicast On Spectrum-2 and Spectrum-3 3.9.0920

systems, when configuring more than
16 LAGs, the Multicast
container/Flooding vectors may
overwrite each other's configuration.

119 Spectrum-3, At times, high effective BER is 3.9.0920

SN4600C, BER experienced when using
LUX42604CO module.

120 Spectrum-3, 1AT-3Q4M01XX-12A cable has long 3.9.0920

SN4600C, link-up time.
Cables

121 Spectrum-2, On rare occasions on Spectrum-2 3.9.0914

ACL, Routes systems, while deleting ACLs or
modifying routes, the firmware will
hang and will not accept additional
configuration.

122 BGP BGP unnumbered neighborship is not 3.9.0900

Unnumbered re-established after upgrade or
reboot and remains in IDLE state
when more than 15k unnumbered
routes are installed.

123 IGMP Snooping, When processing control packets 3.9.0900

Memory (IGMP/LLDP/LACP), memory
Management, allocation was unsuccessful.
Spanning-Tree

124 IGMPv3 IGMPv3 membership report with 3.9.0900

Immediate mixed "leave" and "join" MC groups
Leave fail to leave the IGMP MC groups.
This occurs only when IGMP
"immediate leave" is enabled.

125 IGMPv3, IGMP IGMPv3 membership report with 3.9.0900

Snooping mixed "leave" and "join" MC groups
removes all IGMP Snooping entries.

126 Logging Default configuration of log rotation 3.9.0900

criteria "size" appears in "show
running-config".

127 Running-Config, In show running-config output, BFD 3.9.0900

BFD interface commands appear before
interface IP commands.

As a result, when applying a

54
Index Category Description Fixed in
Version

configuration text file, BFD interface

commands do not work.

128 SYSDUMP In versions 3.8.2100 and up, on rare 3.9.0900

occasion, a system dump could
cause a switch halt.

129 Spectrum-2, On SN3800 and SN3700 systems 3.9.0900

PSU with DC-PSU, when removing or
inserting a single PSU during the run
time, the operating system does not
properly recognize the PSU state and
shows them as if they are in fail state.

130 Switch In version 3.9.06xx, when more than 3.9.0900

Management 1000 DNS resolutions occur via
DHCP, the switch may get stuck.

131 VLAN, JSON JSON VLAN name is incorrectly 3.9.0900

parsed when the name includes more
than 24 characters.

132 BGP BGP Route Table changes the 3.9.0600

displayed routes for local routes
injected.

133 CLI, MGMTD Running "switchport allowed-vlan 3.9.0600

auto-completion" takes a few minutes
when using high CPU usage for
MGMTD and CLI.

134 ECMP Using API “sx_api_router_ecmp_set” 3.9.0600

API with the command
"sx_access_cmd_set” allows the user
to provide a list of next hops.

If the current state of the ECMP

container is that ALL of its next hops
are unresolved, and the user
provides a new list of next hops,
where at least one of the next hops is
already resolved, SDK will not update
the hardware accordingly and traffic
loss will be caused.

135 EVPN Remote Zero MAC leads to a crash. 3.9.0600

136 Ethernet In version 3.9.0300, an error 3.9.0600

Interfaces message, which is a false alarm, may
appear stating that "voltage is out of
range".

137 General When "SSH server login record- 3.9.0600

Management period" is set to 30 days and the
successful login count is higher than

55
 Index Category Description Fixed in
Version

20K, a delay may be experienced in

initializing SSH, the console, or web
sessions.

138 IGMP, IPv6 IPv6 ICMP6 does not work when 3.9.0600
IGMP MRouter is enabled.

139 Logging Logging to remote host in WELF 3.9.0600

format is not working.

140 Logging When Remote Logging is configured 3.9.0600

to use a specific source interface, a
remote syslog session will be sent
with default UDP port 514 regardless
of the user configured UDP port.

141 MLAG In MLAG, occasionally peer MPO 3.9.0600

moved to STP state discarding when
it was actually in forwarding mode.

142 Management The IPv6 address of management 3.9.0600

Interfaces, IPv6 interfaces disappears when link flips,
if it is a static IP.

143 Management Setting IPv6 address on 3.9.0600

Interfaces, management interfaces may cause
LLDP, IPv6 incorrect LLDP BPDU content. As a
result, BPDUs could be discarded by
LLDP tools installed on the host.

144 Multicast, OSPF packets are dropped when the 3.9.0600

Buffers egress multicast buffer is congested.

BGP and OSPF packets were

modified to work with high priority
(VLAN PCP 6) to address the issue.

145 NTP NTP protocol times out when 3.9.0600

configuring loopback as source-
interface—data is sent but cannot be
received on the loopback interface.

146 OpenFlow Execution of "no protocol openflow" 3.9.0600

leads to crash of OVS process due to
errors.

147 OpenFlow OpenFlow group select has incorrect 3.9.0600

load balancing.

148 PFC, RoCE When explicitly disabling PFC 3.9.0600

configuration and then enabling
RoCE lossless/semi-lossless mode,
PFC will remain disabled and will be
shown on running-configuration.

56
Index Category Description Fixed in
Version

149 SNMP, MIB On systems with fixed PSUs (where 3.9.0600

PSUs cannot be removed), when one
PSU is down, SNMP OID
1.3.6.1.2.1.99 fails.

150 Spectrum-2, Bidi module is currently not 3.9.0600

Modules supported.

151 Spectrum-2, SDQ gets stuck when completions 3.9.0600

SN3700C are not received.

152 WJH A large number of named what-just- 3.9.0600

happened files may cause new files
not to be written, with errors in log.

153 WJH What-just-happened pcap files 3.9.0600

cannot be read by tcpdump -r on
switch.

154 WJH Heavy duty generation of what-just- 3.9.0600

happened pcap files (e.g. typing
"show what-just-happened*watch")
might overflow the space allocated
for it and cause errors in the log.

155 WJH what-just-happened-named pcap files 3.9.0600

are not included in sysdump.

156 BGP BGP Peer-Group Route-Map is not 3.9.0300

configured when configuration is
reapplied after running "no protocol
bgp" or "no router bgp" commands.

157 BGP The commands “neighbor export- 3.9.0300

localpref” and “neighbor import-
localpref” do not work after running
the following BGP commands:
activate address-family, route-
reflector, route-map, update-source,
add to group.

158 EVPN JSON output for "show ip bgp evpn * 3.9.0300

detail" is incorrect.

159 EVPN Show IP BGP EVPN for a specific 3.9.0300

VNI was not showing IMET routes.

160 IGMP Receiving ‘IGMP Leave’ packets may 3.9.0300

lead to traffic loss with enabled
immediate-leave mode.

161 Interfaces On a switch that was running 3.9.0300

continuously for at least 7 weeks, the
output of "show interface <>" showed

57
 Index Category Description Fixed in
Version

incorrect values for "last change in

operational status".

162 LDAP Missing LDAP group support. 3.9.0300

163 Logging In some cases, ipstrc.log grows 3.9.0300

indefinitely.

164 Shared Buffer, When upgrading with Advanced 3.9.0300

Advanced Buffer Buffer Configuration, if configured
Configuration, pool size is bigger than current
Advanced Mode available resources, pool size
configuration fails with error logs.

Note that as a part of the solution, the

maximal allowed size will be set
instead.

165 WSMD In very rare cases, heavy web loads 3.9.0300

caused WSMD to crash.

166 rsyslog When rsyslog is flooded with log 3.9.0300

requests, restarting the rsyslog
during the deinitialization flow will
result in a crash.

167 OpenFlow OVS crashes when adding 128 ports 3.8.2300

into the OpenFlow.

168 OpenFlow The OpenFlow rule with action 3.8.2300

"Normal" floods the tagged ARP
packets through all ports on the
switch.

169 SN3800, Split Ports do not go up after rebooting 3.8.2300

with split configuration.

170 ARP MLAG EVPN forwards broadcast 3.8.2204

control packets causing traffic failure.
An ARP request packet may be
looped in setup when working with
MLAG on MLAG (leaf and spine)
setup with EVPN.

171 BGP BGP session gets stuck in connect 3.8.2204

state when adding peer to a peer
group when already configured
correct password.

172 BGP Export route-map was not configured 3.8.2204

when adding peer to peer-group with
import-export route-map
configuration.

58
Index Category Description Fixed in
Version

173 BGP BGP route-map set as-path prepend 3.8.2204

does not function after reboot.

174 BGP, SNMP At times, using snmpwalk with many 3.8.2204

BGP neighbors takes more than 10
seconds.

175 BID-PIM Switch crashes when PIM SM/SMM 3.8.2204

is configured and receives PIM BiDir
control packets.

176 Docker Docker daemon failed to start. 3.8.2204

177 EVPN, MLAG, ARP broadcast duplication. In some 3.8.2204

ARP cases, ARP broadcast packets
received from remote VxLAN
gateway can be recursively
encapsulated into VxLAN and sent
back to the network.

178 IGMPv3 IGMPv3 membership reports appear 3.8.2204

in the log with group IP of 0.0.0.X
instead of the real group IP.

179 IPv6 Ping to an IPv6 address does not 3.8.2204

work when there is MLD querier in
the network on a different VLAN.

180 Logging Redundant log message appears in 3.8.2204

serial output.

181 MLAG In case of a traffic loop inside a setup 3.8.2204

which consists an MLAG cluster, one
of the peers in the MLAG cluster
could learn its own router MAC. In
this scenario, the MLAG peer issued
a Notice Log endlessly.

182 MLAG Spanning MPO on slave side was transmitting 3.8.2204

Tree BPDUs packets when MLAG is UP
and MPO is globally up.

183 MLAG VIP If MLAG VIP ip is lower than MGMT 3.8.2204

IP address, it was chosen as the
advertised IP for LLDP after
performing switch reboot.

184 Management The CLI command “ip default- 3.8.2204

Interfaces gateway <inter-face>” sets the
gateway address to 0.0.0.0 and
prevents the user from adding other
gateways and the command cannot
be removed.

59
 Index Category Description Fixed in
Version

185 Management The CLI allowed for configuration of 3.8.2204

Interfaces an invalid IP and netmask, such as
0.0.0.0/0, on the management
interfaces.

186 PTP The CLI command "show ptp vrf *" 3.8.2204
does not show PTP information for
specific VRF.

187 QoS Changing the default mapping of the 3.8.2204

Switch Priority to Traffic Class will
cause traffic loss.

188 Wizard On manufactured systems, closing 3.8.2204

the terminal suddenly during the
wizard process will cause the wizard
to consume 100% of the CPU.

189 BGP BGP session resets when a peer 3.8.2100

does not support Enhanced Route
Refresh and it moves from one
update group to another.

190 CLI Changed wording from "soruce" to 3.8.2100

"source".

191 General In some rare cases, the system could 3.8.2100

fail to perform a reboot after a
software failure.

192 ICMP A ping to down-state interface VLAN 3.8.2100

will continue to work.

193 Interfaces "no interface ethernet <> 3.8.2100

encapsulation dot1q vlan force"
removes VLAN encapsulation.

194 Logging In case multiple identical log 3.8.2100

messages are sent in a short period
of time, the aggregation may not
work properly.

195 Logging If BPDU Guard is enabled on an 3.8.2100

interface and the interface receives a
BPDU packet, its status should
indicate "down" and a log should be
added to the system log files.

196 MLAG During rapid port flap events, the 3.8.2100

switch may fail to respond to port flap
events.

197 MLAG, L2 Learning new MAC entries at high 3.8.2100

SWITCHING rate could lead, in certain cases, to a
resource leak that later prevents the

60
Index Category Description Fixed in
Version

switch from learning new MAC

entries.

198 OVS OVS crashes after connection of the 3.8.2100

SDN controller.

199 OpenFlow The following CLI commands deleted 3.8.2100

configuration: "openflow del-flows",
"openflow del-group", "openflow del-
meter".

200 RoCE An incorrect port PFC state is 3.8.2100

displayed after disabling RoCE lossy
mode.

201 RoCE Global Flow Control configuration is 3.8.2100

not displayed in running-config after
applying RoCE.

202 SNMP SNMPd crashes when running 3.8.2100

snmpwalk or reading hrDeviceTable
in HOST-RESOURCES-MIB, when
loopback or LAG interface is
configured

203 Spanning Tree, When enabling BPDU filter on an 3.8.2100

MLAG MPO port and one side of the MPO
port is down (MPO is in a state of
partially up globally), it may send
BPDU packets through the port and
ignore the BPDU filter configuration.

204 Spectrum-2, In some cases, traffic is not received 3.8.2100

SN3700 by the switch-host interface in
SN3700 systems.

205 Spectrum-2, On SN3700 switch system, link may 3.8.2100

SN3700 not rise using 200GbE optical cables.

206 Spectrum-2, Configuring a split on Spectrum-2 3.8.2100

Split switch systems causes
misconfiguration.

207 Split Ports Show running-config may associate 3.8.2100

configuration with the wrong port
after the port was split and unsplit.

208 Static Route Default route is excluded from 3.8.2100

recursive next-hop resolution.

209 WJH ACL In some cases, the switch crashes 3.8.2100

when WJH ACL feature is enabled.

61
62
Support and Other Resources

Accessing Hewlett Packard Enterprise Support

• For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide
website:
http://www.hpe.com/assistance
• To access documentation and support services, go to the Hewlett Packard
Enterprise Support Center website:
http://www.hpe.com/support/hpesc

Information to Collect

• Technical support registration number (if applicable)

• Product name, model or version, and serial number
• Operating system name and version
• Firmware version

• Error messages
• Product-specific reports and logs
• Add-on products or components

• Third-party products or components

Accessing Updates
• Some software products provide a mechanism for accessing software updates
through the product interface. Review your product documentation to identify the
recommended software update method.
• To download product updates:
Hewlett Packard Enterprise Support Center
wwww.hpe.com/support/hpesc
Hewlett Packard Enterprise Support Center: Software downloads
wwww.hpe.com/support/downloads
Software Depot
wwww.hpe.com/support/softwaredepot
• To subscribe to eNewsletters and alerts:
wwww.hpe.com/support/e-updates

63
 • To view and update your entitlements, and to link your contracts and warranties
with your profile, go to the Hewlett Packard Enterprise Support Center More
Information on Access to Support Materials page:
www.hpe.com/support/AccessToSupportMaterials

Important: Access to some updates might require product entitlement when

accessed through the Hewlett Packard Enterprise Support Center. You must have
an HPE Passport set up with relevant entitlements.

Customer Self Repair

Hewlett Packard Enterprise customer self repair (CSR) programs allow you to repair
your product. If a CSR part needs to be replaced, it will be shipped directly to you so
that you can install it at your convenience. Some parts do not qualify for CSR. Your
Hewlett Packard Enterprise authorized service provider will determine whether a
repair can be accomplished by CSR.
For more information about CSR, contact your local service provider or go to the
CSR website:
http://www.hpe.com/support/selfrepair

Remote Support
Remote support is available with supported devices as part of your warranty or
contractual support agreement. It provides intelligent event diagnosis, and
automatic, secure submission of hardware event notifications to Hewlett Packard
Enterprise, which will initiate a fast and accurate resolution based on your product's
service level. Hewlett Packard Enterprise strongly recommends that you register
your device for remote support.
If your product includes additional remote support details, use search to locate that
information.

Remote Support and Proactive Care Information

HPE Get Connected
www.hpe.com/services/getconnected
HPE Proactive Care services
www.hpe.com/services/proactivecare
HPE Proactive Care service: Supported products list
www.hpe.com/services/proactivecaresupportedproducts

64
 HPE Proactive Care advanced service: Supported products list
www.hpe.com/services/proactivecareadvancedsupportedproducts

Proactive Care Customer Information

Proactive Care central
www.hpe.com/services/proactivecarecentral
Proactive Care service activation
www.hpe.com/services/proactivecarecentralgetstarted

Warranty Information
To view the warranty for your product or to view the Safety and Compliance
Information for Server, Storage, Power, Networking, and Rack Products reference
document, go to the Enterprise Safety and Compliance website:
www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional warranty information

HPE ProLiant and x86 Servers and Options

www.hpe.com/support/ProLiantServers-Warranties
HPE Enterprise Servers
www.hpe.com/support/EnterpriseServers-Warranties
HPE Storage Products
www.hpe.com/support/Storage-Warranties
HPE Networking Products
www.hpe.com/support/Networking-Warranties

Regulatory Information
To view the regulatory information for your product, view the Safety and Compliance
Information for Server, Storage, Power, Networking, and Rack Products, available at
the Hewlett Packard Enterprise Support Center:
www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional Regulatory Information

Hewlett Packard Enterprise is committed to providing our customers with information

about the chemical substances in our products as needed to comply with legal
requirements such as REACH (Regulation EC No 1907/2006 of the European

65
 Parliament and the Council). A chemical information report for this product can be
found at:
www.hpe.com/info/reach
For Hewlett Packard Enterprise product environmental and safety information and
compliance data, including RoHS and REACH, see:
www.hpe.com/info/ecodata
For Hewlett Packard Enterprise environmental information, including company
programs, product recycling, and energy efficiency, see:
www.hpe.com/info/environment

Documentation Feedback
Hewlett Packard Enterprise is committed to providing documentation that meets
your needs. To help us improve the documentation, send any errors, suggestions, or
comments to Documentation Feedback ([email protected]). When
submitting your feedback, include the document title, part number, edition, and
publication date located on the front cover of the document. For online help content,
include the product name, product version, help edition, and publication date located
on the legal notices page.

Websites

General Websites
Hewlett Packard Enterprise Information Library
www.hpe.com/info/EIL
Single Point of Connectivity Knowledge (SPOCK) Storage compatibility
matrix
www.hpe.com/storage/spock
Storage white papers and analyst reports
www.hpe.com/storage/whitepapers
For additional websites, see Support and other resources.

66