WHITE PAPER:

ANALYZING POWER QUALITY DATA WITH PYTHON

Contributed by Caleb Payne, November 2019
LIBRARY DEPENDENCIES
ABSTRACT
The PMI Python library has two very important dependencies.
Power Monitors provides a range of software tools for power The pip tool can be used to install them if they are not already
quality analysis, including ProVision, Canvass and PQ present on the system:
Canvass. These general-purpose tools are built towards
facilitating the most common types of data analysis among • matplotlib – a graphing package for scientific computing.
customers. Sometimes more in-depth or customized approach
is desired. For these scenarios, PMI provides a full-featured • numpy – a high-performance library for linear algebra,
Python 3 library that can be used to extract, analyze and statistics and other mathematical operations.
examine power quality data from PMI PQ recordings. This
white paper shows how to download and install this library; USING THE LIBRARY
how to use the library to extract PQ records from recordings;
and also highlights some of the built-in utility analysis functions The primary module is nsf_recording. This module contains
that can facilitate a customized or automated analysis. the nsf_recording class, used to read NSF data. The NSF file
format is a PMI data format generated directly from ProVision
DOWNLOADING AND INSTALLING THE LIBRARY or with a conversion utility (the utility will convert .XSB files
to the .NSF format on the f ly). See the end of this paper in
Most Python installations ship with a package management the Resources section for more information on obtaining the
utility called pip. This program is typically invoked from conversion tool.
the command line in order to install Python packages from a
centralized (PyPI) repository. It can, however, be used to install Figure 1 demonstrates how to load and plot a stripchart trace
.whl package files that have been downloaded locally. The from an NSF file. Figure 2 demonstrates the output of the
latter is the primary means of installing PMI’s Python analysis load_strip.py demo script.
package.

The .whl file can be found at https://powermonitors.com/

strip_demo.zip. Begin by downloading this package to an easily
navigable directory. Once the file has been downloaded, open a
command prompt and navigate to the directory where the .whl
file now resides. It can be installed with the following command:

pip3 install --user ./ml_canvass-1.0.5-py3-none-any.whl

This will direct the Python package installer to install the

contents of the .whl file for the local user only. In order to make
a global installation, the user invoking the pip command must be
an administrator.

NOTE FOR WINDOWS USERS

Since this library depends on a native compiled binary Figure 1. Loading and Plotting a Stripchart Trace
(libnsf.dll) and different Python installations and Windows
versions use different default library paths, this library will
automatically extract the libnsf.dll (built into the library) into the
working directory for each project if the libnsf.dll library does
not already exist. This extraction will occur during script
runtime for any script that references the nsf_recording class.
No special steps need to be taken by the user in order for this to
happen.

Page 1 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com
WHITE PAPER:
ANALYZING POWER QUALITY DATA WITH PYTHON
Lines 12 through 21 define the stripchart trace request. The
“start_time” and “end_time” elements are integers in time_t
(UNIX time) format. It should be noted that if both are set to
0 and 0, the entire stripchart trace is retrieved. Subsequent
calls with adjusted start and end times can be made to retrieve
only small portions of the trace if required (ex. if the trace is
many millions of points and takes a long time to render).

The channel element is an integer value that is 1-based (i.e.

channel 1 = 1, channel 2 = 2, etc.) with a valid range of [1,4].
The measure element is a bit more tricky. It is also an integer
value, but is 0-based (it’s actually a C enumerated type). The
primary measures that will likely be of use to the user are:

0 = RMS Voltage

Figure 2. Stripchart Graph Generated from NSF Recording 1 = RMS Current

The source for Figure 1 is the resource load_strip.py. The 2 = Real Power
contents of this script will be brief ly described below to aid the
user in understanding the individual pieces. 3 = Reactive Power

The first line of the script, often called the shebang instructs 4 = Apparent Power
the Linux/UNIX shell to execute the script with the given
interpreter (in this instance Python3). The “mma” element is an abbreviation of “min, max, average”
and allows the user to specify which specific trace they desire.
Following (lines 3 through 7) are the imports needed for the The valid values are “min”, “ave” or “max”.
script. The first import is used to bring in the nsf_recording
class from the ml_canvass.nsf_recording package. This is the The “cthresh” element should always remain 0. The
primary class that will be used for retrieving PQ data from remaining elements (“harmonic” and “interharmonic” are self-
PMI data files. The remainder of the imports are for graphing, explanatory. Setting them to 0 disables them.)
auto-increment array generation, date and time formatting and
some system calls. Line 23 is the call to the stripchart_trace method of the nsf_
recording object. This is the call that actually retrieves the
The main() function is used as the application’s entry point stripchart data. Note that the returned object is a dictionary
(see lines 41-45 – this forces the Python interpreter to directly that contains the following keys:
invoke the main() function when the script is executed). This
function is where all of the useful code is executed. interval – number of seconds between each point (this is the
stripchart recording interval that was set with ProVision when
Line 10 is the instantiation of an nsf_recording object. Note the recording was initialized).
that the only parameter to the constructor is a string, which
happens to be the path to the NSF file. This is the only decimationFactor – f loating point value that is not of use in
parameter that needs to be passed by users. (There are other this application.
optional parameters, such as file mode, but that should not be
altered in the event that the write f lag is accidentally set.) points - an array of point values representing the values
recorded in the stripchart trace.
The next line (line 11) retrieves the header report object from
the recording with a call to the aptly named header_report()
method. The return value is actually a hashmap of key-value
pairs. It is used to retrieve the start and end times of the
recording (which will be passed into the stripchart request).

Page 2 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com
WHITE PAPER:
ANALYZING POWER QUALITY DATA WITH PYTHON
The graphing setup begins on line 27. The call to np.arange As with man pages, the help screen can be navigated with
generates a series of integer values, starting at 1 and ending at the standard vi keystrokes (i.e. ctrl+d to page down, ctrl+u to
len(points) – 1. These will be used for the x-axis of the plot. page up, j to scroll down a line, k to scroll up a line and the /
Lines 29 through 37 are self-explanatory (clear the plot, plot the character can be used to search for specific methods or text).
x and y values (points), create a grid on the plot, a legend and a
title and then show the graph). USEFUL FUNCTIONS

LIBRARY DOCUMENTATION The following is a not-at-all-comprehensive list of some of

the functions that are available to the user as part of the nsf_
The PMI Python library (ml_canvass) has extensive built- recording class. For more details on parameters and return
in documentation. This is convenient in that the interactive types, the reader is urged to invoke help(<function name>)
Python interpreter can be invoked and the built-in help() from within the interactive Python interpreter.
function can be used to read manpage style documentation for
the library. recording_header() - returns the 4 lines of text that appear
at the top of a recording header report in ProVision.
Figure 3 shows how to import the nsf_recording class and
browse the help section. Figure 4 shows the actual manpage hookup_type() - returns the hookup type that was specified
style documentation that is generated from the library. when the recording was initialized.

abnormal_voltage() - returns an array of abnormal voltage

events from the recording.

alerts() - returns an array of alerts from the recording.

Figure 3. Interactive Help for the nsf_recording Class cbema() - returns an array of CBEMA records from the
recording.

events() - returns an array of event capture records from the

recording.

f licker() - returns an array of f licker events from the

recording.

outages() - returns an array of outage records from the

recording.

significant_change() - returns an array of significant change

events from the recording.

transient_capture() - returns an array of transient capture

records from the recording.

list_waveforms() - returns an array of waveform capture

metadata from the recording. This list can be used to select
waveforms whose points are to be loaded by the waveform()
function.

waveform(index,channel,measure) – returns waveform

points and some capture-specific meta-data about an
individual waveform capture.

Figure 4. NSF Recording Documentation

Page 3 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com
WHITE PAPER:
ANALYZING POWER QUALITY DATA WITH PYTHON
stripchart_trace(params) – returns a stripchart trace. See
the examples from figures 1 and 2 above for more details.

energy_usage() – returns an array of energy usage records

from the recording.

histogram(measure,channel,variant) – returns histogram

record for the specified measure, channel and variant. Variant
should be one of ‘standard’, ‘nominal’ or ‘abnormal’ with
‘standard’ being recommended.)

daily_profile(measure,channel) – returns a daily profile

trace for the given channel and measure.

loose_neutral() – returns a list of phase imbalance records

from the recording.

max_voltage_report() – returns an array of max voltage

records.

out_of_limits(measure,min_value,max_value,mode) –
generates an out_of_limits report that is an array of timestamp
records where values were outside of the specified min and max
values.

total_demand_distortion(ratio,start,end) – calculates
the total demand distortion given the specified ratio and
start and end times in UNIX timestamp. The start and end
values default to 0, which will return the TDD for the entire
recording.

header_report() – returns a hashmap of key-value pairs

that contain the same information that would be found in the
ProVision header report.

close() – closes the recording when analysis is complete.

DETAILED EXAMPLE

A more detailed example of how to use Python and the ML

Canvass library to “go beyond” the functionality of ProVision
is summarized here. Thesource code for this example (as
well as the demo NSF recording used) can be obtained at the
following URL: https://powermonitors.com/strip_demo.zip

Figure 5 shows a short screenshot or code snippet for the

included stripchart demonstration code. It is recommended
that the reader download the code sample from the URL listed
above in order to follow along.

Figure 5. Screenshot of Code Snipet from strip_demo.py

Page 4 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com
WHITE PAPER:
ANALYZING POWER QUALITY DATA WITH PYTHON
In general terms, this script retrieves the min, average and readings for each individual 24 hour time period is retrieved
max stripchart traces for channel 1 voltage and channel 1 real with subsequent calls to get_points().
power for the duration of the recording. It then plots both sets
of traces (voltage and real power) in two separate interactive Once the points have been retrieved, the total kilowatts for
plots. (The user is encouraged to zoom, pan, inspect, etc. the the given period are calculated with a call to np.sum(points)
graph that is rendered by the application. The interactions / 1000. That takes the sum of all of the values in the points
are intuitive and the x-axes are synchronized so zooming in on array and then divides the sum by 1000 (from watts to
one feature on one plot will force zooming or panning on the kilowatts). The next line calculates the actual kWh value by
corresponding plot.) taking kwh = kw / (24 * day_len). The day_len is actual a
percentage of a 24 hour day based on the start and end time
The recording is from a solar installation. This data shows of the last value from calc_days(). Only the final day is a
the impact of solar generation on voltage regulation during non-24 hour day, but it’s necessary to use the ratio for each
a multi-day span. It is a quick and handy way to show calculation for the sake of accuracy.
those correlations (and there are several instances of voltage
regulators adjusting voltage based on solar output). Finally, all of these values are appended to an array (along
with the start time being appended to a separate x-axis array)
After the first graph closes, the user is presented with a second so that they can be plotted with matplotlib.
graph. This graph is a line graph with the calculated kWh per
day for each of the 24 hour periods in the recording (except for
the final period, which is a fractional day). Closing this graph Figure 6. Selected segment of the resultant graph.
will result in a console-based text report showing the kWh for Shows one day’s worth of solar generation and correlated regulation events.
each given 24 hour period (from start of recording
to end of recording) in a tabular format.

The load_stripchart_traces() function is used

to load each of the six stripchart traces (channel
1 voltage min, max and ave and channel 1 real
power min, max and ave) from the included
recording. The reader will note that there are two
loops (one nested within the other) for retrieving
these traces – the outer loop to select the measure
(0 = voltage, 1 = current, 2 = real power, 3 =
reactive power, 4 = apparent power, etc.) and
the inner loop to select the “mma” (min/max/
average) type for the trace. The resulting points
arrays are wrapped in numpy arrays (for ease of
mathematical operations), appended to a Python
array of return values and then returned as a
dictionary with all of the other pertinent recording
bits as well.

The next function of interest is the kwh_graph()

function. It is used not only to graph the kWh
report, but to calculate the kWh values for each
24 hour period as well. This is done through
three different functions. The first is a generator
function called calc_days() that takes the
recording start and end times and generates
tuples of 24-hour start/end time periods. Those
are passed into the kwh_graph() function when
called in the for day in calc_days(start_
time,end_time): loop. The resulting real power

Page 5 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com
WHITE PAPER:
ANALYZING POWER QUALITY DATA WITH PYTHON
The final section of the kwh_graph() function is used to
generate the tabular report of kWh to the console. The source
and output of this small loop are fairly self-explanatory.

Figure 7. Tabular Output from kWh Report to Console

CONCLUSION

With the PMI Python data library, a user can easily extract the
PQ data needed for further customized analysis. This paper
has provided a brief overview of the library’s functionality,
an example of how to load data from a PQ recording and
suggestions for finding one’s way through the library’s
documentation.

Page 6 WP#334 © 2019 Power Monitors, Inc. • Call Us: (800) 296-4120 • www.powermonitors.com