IEEE ISIE 2006, July 9-12, 2006, Montreal, Quebec, Canada

InfiniB and-Based Real- Time Simulation

of HVDC, STATCOM and SVC Devices
with Custom-Of-The-Shelf PCs and FPGAs
Christian Dufour, Abourida, Belanger
Opal-RT Technologies, 1751 Richardson, suite 2525, Montreal, Quebec, Canada
Tel: 514-935-2323
{christian.dufour, simon.abourida, jean.belanger}gopal-rt.com

Abstract- This paper presents a real-time simulator for large HVDC, STATCOM and SVC are also good examples of
power network based on Custom-Of-The-Shelf technologies, all complex power grid devices and systems. They also present
embedded in the RT-LAB real-time simulation platform. This
platform uses Pentium, Xeon, Opteron-based PCs (multi-CPUs some challenges in terms of computing power and algorithms
and/or dual-core configurations) or even Xilinx FPGA cards for when considering real-time simulation because of course of
computational engines and InfiniBand communication fabric for their size but also for high number of switches contained in
fast inter-PCs communications. The real-time PCs runs under them.
well-known operating systems QNX or RedHawk Linux while the A. Objective ofthe work
main user control interface is either Simulink or LabView.
The paper demonstrates the real-time simulation of complete The power grid real-time simulator was made with the
single-pole 12-pulse HVDC system on dual-CPU, dual-core 2.2 following considerations and objectives:
GHz Opteron PC under 15 microseconds time step. It also * The simulator must ultimately be able to simulate
demonstrates the real-time simulation of complex power system realistically complex power networks like bipolar
devices like SVC, STATCOM and more general power systems HVDC transmission system
like the Kundur network. * The simulator must be very scalable to meet very large
The paper also discusses the latest advances in Hardware-In- grid simulation challenges.
the-Loop simulation like the possibility to program from within * The simulator must be able to accommodate very
Simulink some controllers or devices, like a PMSM drive, directly flexible user interface and modem automatic test
in an FPGA card. This feature is enabled in RT-LAB by the use of
Xilinx System Generator, a Simulink blockset. This FPGA technologies to cope with the complexity of power grid
targeting diminishes further the leap between prototype and design and validation tests.
production-type controller systems because the FPGA can * The simulator must use generically available
implement rapidcontrol functions along with fast protection components and technologies with perennial qualities
systems, like IGBT-current protection, of real controller. like industry wide acceptance (Simulink, Intel/AMD
I. INTRODUCTION
computers, QNX/Linux operating systems, InfiniBand
switching fabric, etc).
Testing of complex High Voltage Direct Current (HVDC) * The simulator must come with flexible and re-
network, Static VAR Compensators (SVC), Static programmable high-performance input-output devices
Compensators (STATCOM) and other FACTS devices control to interface the simulator with modem power electronic
systems under steady state and transient operating conditions is controllers and protection systems; users shall be able
nowadays a common and mandatory practice during the to develop their own custom interface if needed.
control development phase and before the final system
commissioning in order to reduce risks. Hardware-In-the-Loop II. REAL-TIME GRID SIMULATOR DESCRIPTION
(HIL) testing, where the controller is connected in closed loop
with a real-time simulator, are first made with a prototype A. Real-Time Simulation Algorithms
controller and, after that, with real controller that will be The main algorithm used in RT-LAB Grid Simulator is the
installed in the field after successful testing. Several thousands ARTEMIS plug-in to SimPowerSystems (SPS) blockset from
systematic and random tests are often required to test Simulink. ARTEMIS enables real-time simulation for SPS
performance under normal and abnormal operating conditions model because it makes precomputation of systems equations,
and to detect instabilities caused by unwanted interactions provides a set of special discrete solvers to improve
between control functions and the power system that will discretization stability and facilitate network equation
usually included other FACTS systems that may interact with decoupling. This decoupling can be natural like in the case of
the system under tests.

1-4244-0497-5/06/$20.00 © 2006 IEEE 2025

Bergeron traveling-wave power lines that have inherent delays III. REAL-TIME SIMULATION EXAMPLES MADE ON RT-LAB
built-in their model. The decoupling can also be artificially POWER GRID SYSTEM SIMULATOR
added to the model by substituting transformer secondary Real-time simulation of the following power systems cases
inductance by a stubline (one time step propagation time are demonstrated in the paper: 12-pulse single-pole HVDC
equivalent traveling-wave line, see [6] for example) with the system, 48-pulse GTO STATCOM, complex multi-capacitor
same total series line inductance. The simulation algorithms bank SVC system and a two-area, four machine system
used in this paper includes these techniques to achieve cluster (Kundur network) All the models described next are designed
distribution of the computation loads. using ARTEMIS add-on to SimPowerSystems blockset for
Simulink and /or RteDrive blockset and runs in RT-LAB on
Voltage source inverters are best modeled using a either multi-core/multi-CPU PCs or InfiniBand cluster of PCs.
specialized toolbox from Opal-RT called RteDrive that
includes special interpolating IGBT inverter models. It is used A. HVDC transmission system
to model very effectively and precisely most type of voltage Real-time simulation of HVDC transmission is important to
inverters like the one used IGBT drive systems and IGBT- evaluate controller behavior and overall network response to
GTO-MOSFET-based power systems devices like STATCOM. various contingencies like AC-faults (one phase 3-phase), DC-
This blockset is used in combination to RT-Events blockset, faults, thyristor misfires, both at the inverter and the rectifier
which is used to design interpolated firing pulse unit (FPU), to sides. Normal operations of the HVDC system like gamma
the RTeDrive models. extinction angle estimation and working mode shifts are also in
the scope of interest.
B. Real-Time Simulation Hardware
The HVDC transmission model used is rated at 1000 MW,
Similar computational load distribution capability must exist 500 kV, 2kA and is used to transmit power from a 500 kV,
at the hardware level and RT-LAB, from Opal-RT 5000 MVA, 60 Hz network to a 345 kV, 10 000 MVA, 50 Hz
Technologies, is based on this fundamental paradigm. For this network. The rectifier and the inverter are 12-pulse converters
purpose, RT-LAB supports distributed simulation through 2-4- using two 6-pulse thyristor bridges connected in series. The
8-16-CPU (and recently dual-core) shared-memory AMD or rectifier and the inverter are interconnected through a 300 km
Intel-type processors or through PC-cluster with InfiniBand distributed parameter line and two 0.5 H smoothing reactors.
communication links. RT-LAB currently support InfiniBand The transformer tap changers are not simulated and fixed taps
adapter from Mellanox and is soon expected to support are assumed (0.9 at the rectifier and 0.96 at the inverter).
InfiniPath adapter, from PathScale. InfiniPath is an InfiniBand Reactive power required by the converters is provided by a set
compatible adapter used for PC cluster communication and is of capacitor banks plus 11th, 13th and high pass filters for a
reported to have the lowest reported 0-byte-half-trip latency of total of 600 Mvar on each side.
about 1.3 microseconds. Two circuit breakers are used to apply various AC and DC-
fault faults and thyristor misfires can also be tested. At the
The infinite possibilities of tests, contingencies and network inverter side, a resonant RLC circuit is included to study
configurations require effective test managing programs for controller interaction. This circuit is modeled without SPS so
optimization and Monte Carlo studies. RT-LAB TestManager the inverter-side grid impedance Z, defined by short-circuit
is build around TestStand, from National Instruments, and can power, resonance frequency and damping, are continuously
use Python scripts, an open source multi-thread language, to adjustable during run-time.
design test suites and generate test reports automatically. 0.5 H smoothing 0.5 H smoothing
reactor (Q=150) reactor (Q=150)
R= 1 ohms R= 1 ohms

The graphical interface of a real-time simulator is also very

important. Simulink interface (scope, dials, etc...) can quickly
become inept on complex problem like power grid tests. For
example, in repetitive fault tests, one would like the acquisition
and display of data to be synchronized on the fault. TestDrive,
from Opal-RT, is a LabView-based interface with Python-
scripting capability. Requiring only the LabView runtime
engine, it enables users to build on-the-fly LabView display
and control panels by virtually wiring real-time simulation Fig. 1 Single pole 12-pulse HVDC transmission system
signals to graphical displays. TestDrive also have built-in The complete single-pole HVDC transmission system
display triggering capability that enables to display complex depicted in Fig. 1 runs at a time step of 15 gs on a dual-core
waveforms in real-time in synchronization to specific events dual-Opteron 2.2GHz PC running under RT-LAB 8.0.
like a fault or control signal step.

2026
 As an example test, the model was used to run a 0.3 sec. DC time simulation of this model on dual-Xeon PC running under
fault (18-cycle @60 Hz). The resulting line voltage and RT-LAB was achieved at time step of 27 gs in two different
currents as well as the firing angle of the rectifier and inverter- configurations: dual-core, dual-CPU 2.2 GHz Opteron
side converters are displayed in Fig. 2 and 3. The results is computer and 42 gs on a InfiniBand cluster of the Xeon 3.6
somewhat similar to the one reported in [2] in terms on GHz PCs.
rectifier-side over-current and recovery time even if it was BUS1
made on 6-pulse HVDC system and slightly different network
parameters.
1.5

0
- (b) IdRect
2.0
- (g) Id Rect commanded

1-10
g
Q0 Fig. 4 48-pulse GTO STATCOM configuration
m
a) 200-
V BUS1
100
V

-2- 0
2 2.5 3 3.5
Time (s)

Fig. 2 Line voltages, line currents and firing angle at the

rectifier side of the HVDC system
1.5 Il

6500 MVA
3'
0L i
Fig. 5 STATOM compensated power system
-00.5-
1 -
STATCOM signals (3-phase to ground fault at BUS3)
0 >~ 2-

Q0

QL
1 10
1-c
0.5 _ (b)ldlnv 2
(g)ldlnv commanded
>
0
<z
0
5_
_ 0
m 160
iuu N
a)
-a
140 01_
5U)
a 120
100
_
_4 I

3 va)x 0
< 80 (5 us) SPS model
2.5 3 (40 us) SPS model
Time (s)
omm3 025 (40 us) TSB

Fig. 3 Line voltages, line currents and firing angle at the -cz 2 -

v
inverter side of the HVDC system n1
0.25 0.35 0.45
B. 48-Pulse GTO STA TCOM Compensated Power System time (s)
Fig. 6 STATCOM response to a 3-phase fault at BUS3
The static compensator (STATCOM) is a device used to
regulate voltage and improve dynamic stability of power On the STATCOM compensated network of Fig. 5, a 3-
systems. GTO-based STATCOM (Fig. 4) are multilevel line- phase fault to ground was made on the real-time simulation
commuted voltage-sourced inverters that are shunt-connected result is compared to a small time step simulation in Fig. 6.
to a power system bus through a set of transformers. The real- More tests results are available in [7].

2027
C. SVC compensated network depicted runs at a time step of 14 gs on a dual-core dual-
Opteron 2.2GHz PC running under RT-LAB 8.0.
Thyristor-based SVCs are used to regulate the voltage on a
power grid by continuously adjusting injected reactive power D. Kundur network (Electromechanical stability analysis)
at its connection point. This is done by fine firing angle control
of a thyristor-controlled reactor (TCR). Capacitor banks (TSC) The Kundur power system, depicted in Fig. 9, consists of
are switched on and off with thyristors at power frequency to two fully symmetrical areas linked together by two 230 kV
provide offset on the VAR range of the SVC (Fig. 7). lines of 220 km length. It was specifically designed in [3] [4] to
study low frequency electromechanical oscillations in large
SVC is obviously a complex and costly device and therefore interconnected power systems.
required thorough testing. Among numerous contingencies that
can occur in SVCs, out-of-phase TSC misfiring can be I
catastrophic. This phenomenon occurs when a thyristor is
mistakenly fired with the bus voltage at its maximum
amplitude and the stored TSC capacitor voltage at the opposite
polarity. This can cause great over-currents in the thyristor and
up to 3 pu. over-voltage in the capacitor depending on the
timing of the pulse. Without proper MOV protection, the
capacitor could by destroyed and the SVC shutdowned for
maintenance. Using a 3-segment piecewise linear MOV model
(not a standard SPS model), the real-time simulation of the Fig. 9 The Kundur Network
SVC devices with an out-of-phase misfire of a TSC bank was The electromagnetic transient type of simulation made in
carried out and the results displayed in Fig. 8. RT-LAB enables the study of fast and detailed phenomena like
735kV 16kV single-phase faults in the Kundur network and observe their
4I ALI effects on a larger time scale, i.e. on the electromechanical
scale like inter-area power oscillations. It also permits easy
f
7335kV
600C35MVA 333MVA
X=1500
HIL connection to the systems of real network controllers[9].

'+ (lgd l 2t An 8-cycle, 3-phase fault test was carried out and the results
are displayed in Fig. 10. This test was made at real-time speed
To thyristors
TCR
T
TSC1
T
TSC2
T
TSC3 on a dual-core dual-Opteron PC at time step of 18 gs. To
109 Mvar 94 Mvar 94 Mvar 94 Mvar obtain this low sample time, the network was decoupled in two
tasks while all machine controllers and stabilizers where
Fig. 7 SVC compensated network
grouped in a 3rd core. The 4th core was dedicated to TCP/IP
communications with the host PC. The results are somewhat
=2-
identical to phasor-type simulation results (not shown).
CL
0 o 500
0
C-)
c.5 -1
Q 5;
U
L
Out-of-phase _ 400
misfire
U) -3-
Co
2300
0 8-cycle 3-phase fault at line mid-point
Q r a- Pre-fault transit power: 413 WM

m 2 200-
2
a)
b
0
Q
>'-2 -' (b) wol thyristor MOV a) 1
CL
(r) w/ thyristor MOV
F- -4
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2
Time (s) o- 0.8
0
a)
(n (b) Sector 1 voltage
Fig. 8 TSC bank misfire test results (g) Sector 2 voltage
0.6
2 2.5 3 3.5 4 4.5 5 5.5 6
This configuration of SVC contains 24 thyristors in total. Time (s)
Stublines are used to decouple the network equations in several
Fig. 10 Test results for a 3-phase fault at line mid-point.
tasks executable on several CPUs. The complete SVC model

2028
 USER CUSTOM I/0 INTERFACE AND COMPLEX DEVICE TABLE I
SUMMARY OF ACHIEVED SAMPLE TIMES FOR VARIOUS GRID MODELS ON
PROGRAMMATION ON FPGA USING RT-LAB XSG DUAL-CORE DUAL- OPTERON 2.2 GHz PC OR DUAL-XEON 3.6 GHZ PC.
Xilinx System Generator (XSG) is a Simulink blockset that
allows block diagram based programmation of an FPGA card.
In RT-LAB, XSG support enables users to program the FPGA Model Sample time (gs) Target
card of the simulator for custom I/Os with fast current HVDC (12-pulse, single pole) 15 Opteron
regulation loops or motor drives with ultra-low input-output
latency, for example, without any HDL language knowledge. STATCOM 27 Opteron
This also allows for simulation of complex systems with tasks
SVC 15 Opteron
executed in parallel on CPUs and FPGAs.
Kundur 18 Opteron
In [8], a PMSM model based on Park transform with a Xeon (3.6 GHz)
reference frame on the rotor is presented. The PMSM machine STATCOM 42 cluster with
in driven by a 3-phase IGBT inverter. All models are l___________________ |___ InfiniBand
implemented in RT-LAB using Xilinx System Generator,
without any HDL coding. The cited paper explains various The STATCOM timing with InfiniBand (Mellanox) and
aspects of the design of the motor drive models in fixed-point Xeon 3.6 GHz is difficult to compare with the Opteron
representation in XSG, as well as validations against a standard benchmarks. From our experience, the latest available Xeon
PMSM model built in Simulink. CPUs are general slower than the Opteron counterparts,
regardless of GHz, which are most of the time meaningless
Unlike automotive industry controller which are designed when comparing different CPU type.
for mass-scale production, power system controller cost is not The support for the InfiniPath adapter should improve this
a critical aspect of their design. The hardware used for result as a communication time of 5 gs in RT-LAB has been
controller prototyping may even be used for the final controller already obtained for one double number transmission across
and be of PC-type like RT-LAB with mission-critical operating two PCs via the InfiniBand switch.
system QNX for example.
REFERENCES
With the support of XSG, power system control designer can
also include the modelisation of the fast modules of Firing
[1] M. Harakawa, H. Yamasaki, T. Nagano, S. Abourida, C. Dufour and J.
Pulse Units. This can include fast protection system like pulse Belanger, "Real-Time Simulation of a Complete PMSM Drive at 10 us
disable-based current limiters. Another important aspect of Time Step", Proceedings of the 2005 International Power Electronics
XSG support is that the enginner can now program the Conference (IPEC 2005) - April 4-8, 2005, Niigata, Japan.
[2] Vijay K. Sood, "HVDC and FACTS Controllers, Applications of Static
complete controller with minimal knowledge of low-level Converters in Power Systems", Kluwer Power Electronics and Power
HDL language. Systems Series, Kluwer Academic Publishers, ISBN 1-4020-7890-0
[3] P. Kundur, Power System Stability and Control, McGraw-Hill, 1994,
CONCLUSION Example 12.6, p. 813
[4] Klein, Rogers, Moorty and Kundur: "Analytical investigation of factors
This paper has presented a real-time simulator with the influencing PSS performance," IEEE Trans. on EC, Vol. 7 No 3,
ability to simulate realistic power grid circuits and devices September 1992
[5] C. Dufour, J. Belanger, "Discrete Time Compensation of Switching
used by power utilities. Events for Accurate Real-Time Simulation of Power Systems",
The grid circuits are designed in the SimPowerSystems Proceedings of the 27th IEEE Industrial Electronics Society Conference
interface and make use of ARTEMIS special solver and (IECON'O 1), Nov 29-Dec 2 2001, Denver, Colorado, USA
[6] V. Dinavahi, " Modeling and Simulation of Multiple Nonlinearities
models to achieve hard real-time simulation. Table 1 Using TLM", Proceedings of the 6th International Conference on Power
summarizes the various model performance reported in this Systems Transients (IPST 2005), June 19-23, 2005, Montreal, QC,
,

paper. Canada.
[7] C. Dufour, J. Belanger, "Real-time Simulation of a 48-Pulse GTO
Table 1 shows among other things the power of dual-core STATCOM Compensated Power System on a Dual-Xeon PC using RT-
dual-CPU Opteron PCs. It also shows that Infiniband LAB", Proceedings of the 6th International Conference on Power
simulation comes with a cost on sample time because it is Systems Transients (IPST-05), June 19-23, 2005, Montreal, QC, Canada.
[8] C.Dufour, S. Abourida, J. Belanger, " Real-Time Simulation of
slower than shared-memory communication. Permanent Magnet Motor Drive on FPGA Chip for High-Bandwidth
Controller Tests and Validation", Proceeding of the IEEE-ISIE 2006
Conference, Montreal, Quebec, Canada, July 9-13 2006.
[9] R. Majumber, B.C. Pal, C. Dufour, P. Korba, "Design and Real-Time
Implementation of Robust FACTS Controller for Damping Inter-Area
Oscillation", IEEE Transactions on Power Systems, Vol. 21, No. 2, pp.
809-816, May 2006.

2029