MODULE 4 (part 2)

Sabira PS
AP/CSE
RCET
Topics covered
• Hadoop ecosystem
• Edge Streaming Analytics
• Network analytics
• Flexible NetFlow Architecture
• Securing IOT
• Common Challenges in OT Security
• Formal Risk Analysis Structures: OCTAVE and FAIR
 Hadoop Ecosystem
• Hadoop plays an increasingly big role in the collection, storage, and
processing of IoT data due to its highly scalable nature and its ability to
work with large volumes of data
• Since the initial release of Hadoop in 2011, many projects have been
developed to add incremental functionality to Hadoop and have
collectively become known as the Hadoop ecosystem
• The following sections describe several of these packages and discuss
how they are used to collect or process data
1. Apache Kafka

• Apache Kafka is a distributed publisher-subscriber messaging system that is built to be

scalable and fast.
• It is composed of topics, or message brokers, where producers write data and consumers
read data from these topics.
• Figure below shows the data flow from the smart objects (producers), through a topic in
Kafka, to the real-time processing engine.
• Due to the distributed nature of Kafka, it can run in a clustered configuration that can
handle many producers and consumers simultaneously and exchanges information
between nodes, allowing topics to be distributed over multiple nodes.
• The goal of Kafka is to provide a simple way to connect to data sources and allow
consumers to connect to that data in the way they would like
Apache Kafka Data Flow
2. Apache Spark
• Apache Spark is an in-memory distributed data analytics platform
designed to accelerate processes in the Hadoop ecosystem.
• The “in-memory” characteristic of Spark is what enables it to run jobs
very quickly.
• At each stage of a MapReduce operation, the data is read and written
back to the disk, which means latency is introduced through each disk
operation.
• With Spark, the processing of this data is moved into high-speed
memory, which has significantly lower latency. This speeds the batch
processing jobs and also allows for near-real-time processing of events.
Apache Spark
• Spark Streaming is an extension of Spark Core that is responsible for
taking live streamed data from a messaging system, like Kafka, and
dividing it into smaller micro batches.
• These microbatches are called discretized streams, or DStreams.
• The Spark processing engine is able to operate on these smaller pieces of
data, allowing rapid insights into the data and subsequent actions.
3. Apache Storm and Apache Flink
• Apache Storm and Apache Flink are other Hadoop ecosystem projects
designed for distributed stream processing and are commonly deployed
for IoT use cases.
• Storm can pull data from Kafka and process it in a near-real-time fashion,
and so can Apache Flink.
• This space is rapidly evolving, and projects will continue to gain and lose
popularity as they evolve
4. Lambda Architecture
• One architecture that is currently being leveraged for Querying both data in motion
(streaming) and data at rest (batch processing) functionality is the Lambda
Architecture.
• Lambda is a data management system that consists of two layers for ingesting data
(Batch and Stream) and one layer for providing the combined data (Serving).
• These layers allow for the packages like Spark and MapReduce, to operate on the data
independently, focusing on the key attributes for which they are designed and optimized.
• Data is taken from a message broker, commonly Kafka, and processed by each layer in
parallel, and the resulting data is delivered to a data store where additional processing or
queries can be run.
Lambda Architecture
3 layers:
• Stream layer: This layer is responsible for near-real-time processing of
events. Technologies such as Spark Streaming, Storm, or Flink are used to
quickly ingest, process, and analyze data on this layer
• Batch layer: The Batch layer consists of a batch-processing engine and
data store. If an organization is using other parts of the Hadoop ecosystem
for the other layers, MapReduce and HDFS can easily fit the bill. Other
database technologies, such as MPPs, NoSQL, or data warehouses, can
also provide what is needed by this layer
• Serving layer: The Serving layer is a data store and mediator that decides
which of the ingest layers to query based on the expected result or view
into the data. If an aggregate or historical view is requested, it may invoke
the Batch layer. If real-time analytics is needed, it may invoke the Stream
layer. The Serving layer is often used by the data consumers to access
both layers simultaneously
 Edge Streaming Analytics
• In the world of IoT, vast quantities of data are generated on the fly and
often need to be analyzed and responded to immediately.
• Not only is the volume of data generated at the edge immense meaning
the bandwidth requirements to the cloud or data center need to be
engineered to match but the data may be so time sensitive that it needs
immediate attention, and waiting for deep analysis in the cloud simply
isn’t possible.
The key values of edge streaming analytics include the
following:

• Reducing data at the edge:

• The aggregate data generated by IoT devices is generally in proportion to
the number of devices.
• The scale of these devices is likely to be huge, and so is the quantity of
data they generate.
• Passing all this data to the cloud is inefficient and is unnecessarily
expensive in terms of bandwidth and network infrastructure
• Analysis and response at the edge: Some data is useful only at the edge
(such as a factory control feedback system). In cases such as this, the data
is best analyzed and acted upon where it is generated.
• Time sensitivity: When timely response to data is required, passing data
to the cloud for future processing results in unacceptable latency. Edge
analytics allows immediate responses to changing conditions
Edge Analytics Core Functions

• To perform analytics at the edge, data needs to be viewed as real-time flows.

• Whereas big data analytics is focused on large quantities of data at rest, edge
analytics continually processes streaming flows of data in motion.
• Streaming analytics at the edge can be broken down into three simple stages:
1. Raw input data: This is the raw data coming from the sensors into the
analytics processing unit.
2. Analytics processing unit (APU): The APU filters and combines data streams
(or separates the streams, as necessary), organizes them by time windows, and
performs various analytical functions.
• Output streams: The data that is output is organized into insightful
streams and is used to influence the behavior of smart objects, and passed
on for storage and further processing in the cloud.

Edge Analytics Processing Unit

In order to perform analysis in real-time, the APU needs to
perform the following functions:

1. Filter: The streaming data generated by IoT endpoints is likely to be very large, and
most of it is irrelevant. For example, a sensor may simply poll on a regular basis to
confirm that it is still reachable
2. Transform: In the data warehousing world, Extract, Transform, and Load (ETL)
operations are used to manipulate the data structure into a form that can be used for
other purposes
3. Time: As the real-time streaming data flows, a timing context needs to be established.
This could be to correlated average temperature readings from sensors on a minute-by-
minute basis
4. Correlate: Streaming data analytics becomes most useful when multiple data streams are
combined from different types of sensors. For example, in a hospital, several vital signs are
measured for patients, including body temperature, blood pressure, heart rate, and
respiratory rate.

Correlating Data Streams with Historical Data

5. Match patterns: Once the data streams are properly cleaned, transformed, and correlated
with other live streams as well as historical data sets, pattern matching operations are used
to gain deeper insights to the data.
For example, say that the APU has been collecting the patient’s vitals for some time and has
gained an understanding of the expected patterns for each variable being monitored.
6. Improve business intelligence: Ultimately, the value of edge analytics is in the
improvements to business intelligence that were not previously available.
For example, conducting edge analytics on patients in a hospital allows staff to respond
more quickly to the patient’s changing needs and also reduces the volume of unstructured
(and not always useful) data sent to the cloud.
Network Analytics
• Another form of analytics that is extremely important in managing IoT
systems is network-based analytics.
• Network analytics has the power to analyze details of communications
patterns made by protocols and correlate this across the network.
• It allows you to understand what should be considered normal behavior in a
network and to quickly identify anomalies that suggest network problems
due to suboptimal paths, intrusive malware, or excessive congestion.
Network management services are:

• Network traffic monitoring and profiling

• Application traffic monitoring and profiling
• Capacity planning
• Security analysis
• Accounting
• Data warehousing and data mining:
Flexible NetFlow (FNF) Architecture

• Flexible NetFlow (FNF) and IETF IPFIX (RFC 5101, RFC 5102) are
examples of protocols that are widely used for networks.
• FNF is a flow technology developed by Cisco Systems that is widely
deployed all over the world.
• Key advantages of FNF are as follows:
➢ Flexibility, scalability, and aggregation of flow data.
➢ Ability to monitor a wide range of packet information and produce new
information about network behavior
➢ Enhanced network anomaly and security detection
➢ User-configurable flow information for performing customized traffic
identification and ability to focus and monitor specific network behavior.
➢ Convergence of multiple accounting technologies into one accounting
mechanism
FNF Components:

1. FNF Flow Monitor (NetFlow cache):

• The FNF Flow Monitor describes the NetFlow cache or information stored
in the cache.
• The Flow Monitor contains the flow record definitions with key fields (used
to create a flow, unique per flow record: match statement) and non-key
fields (collected with the flow as attributes or characteristics of a flow)
within the cache.
2. FNF flow record:

• A flow record is a set of key and non-key NetFlow field values used to
characterize flows in the NetFlow cache.
• Flow records may be predefined for ease of use or customized and user
defined.
• A typical predefined record aggregates flow data and allows users to target
common applications for NetFlow.
3. FNF Exporter:

• There are two primary methods for accessing NetFlow data: Using the show
commands at the command-line interface (CLI), and using an application
reporting tool.
• NetFlow Export, unlike SNMP polling, pushes information periodically to the
NetFlow reporting collector.
4. Flow export timers:
Timers indicate how often flows should be exported to the
collection and reporting server.
5. NetFlow export format:
This simply indicates the type of flow reporting format.
6. NetFlow server for collection and reporting:
This is the destination of the flow export.
It is often done with an analytics tool that looks for anomalies in the traffic
patterns.
Securing IOT
 Common Challenges in OT Security

• 1. Erosion of Network Architecture

• Two of the major challenges in securing industrial environments have been initial design
and ongoing maintenance.
• The initial design challenges arose from the concept that networks were safe due to physical
separation from the enterprise with minimal or no connectivity to the outside world, and the
assumption that attackers lacked sufficient knowledge to carry out security attacks.
• The challenge, and the biggest threat to network security, is standards and best practices
either being misunderstood or the network being poorly maintained.
2. Pervasive Legacy Systems

• Due to the static nature and long lifecycles of equipment in industrial

environments, many operational systems may be deemed legacy
systems.
• In many cases, legacy components are not restricted to isolated network
segments but have now been consolidated into the IT operational
environment.
• From a security perspective, this is potentially dangerous as many
devices may have historical vulnerabilities or weaknesses that have not
been patched and updated, or it may be that patches are not even
available due to the age of the equipment.
3. Insecure Operational Protocols

• Many industrial control protocols, particularly those that are serial based, were
designed without inherent strong security requirements.
• Furthermore, their operation was often within an assumed secure network.
• In addition to any inherent weaknesses or vulnerabilities, their operational
environment may not have been designed with secured access control in mind.
• Some common industrial protocols and their respective security concerns are
given below:
3.1 Modbus

• Modbus is commonly found in many industries, such as utilities and manufacturing

environments, and has multiple variants (for example, serial, TCP/IP).
• It was created by the first programmable logic controller (PLC) vendor, Modicon,
• The security challenges that have existed with Modbus are not unusual.
• Authentication of communicating endpoints was not a default operation because it
would allow an inappropriate source to send improper commands to the recipient.
• Some older and serial-based versions of Modbus communicate via broadcast.
3.2 DNP3 (Distributed Network Protocol)

• DNP3 is found in multiple deployment scenarios and industries.

• It is common in utilities and is also found in discrete and continuous
process systems.
• There is an explicit “secure” version of DNP3, but there also remain
many insecure implementations of DNP3 as well.
• DNP3 has placed great emphasis on the reliable delivery of messages.
• That emphasis, while normally highly desirable, has a specific
weakness from a security perspective.
3.3 ICCP (Inter-Control Center Communications Protocol)

• ICCP is a common control protocol in utilities across North America that is

frequently used to communicate between utilities.
• Unlike other control protocols, ICCP was designed from inception to work
across a WAN.
• Despite this role, initial versions of ICCP had several significant gaps in the
area of security.
• One key vulnerability is that the system did not require authentication for
communication.
Second, encryption across the protocol was not enabled as a default condition,
thus exposing connections to man-in-the-middle (MITM) and replay attacks.
3.4 OPC (OLE for Process Control)

• OPC is based on the Microsoft interoperability methodology Object Linking and

Embedding (OLE).
• This is an example where an IT standard used within the IT domain and personal
computers has been leveraged for use as a control protocol across an industrial
network.
• In industrial control networks, OPC is limited to operation at the higher levels of the
control space, with a dependence on Windows-based platforms.
3.5 International Electrotechnical Commission (IEC) Protocols

• The IEC 61850 standard was created to allow vendor-agnostic engineering

of power utility systems, which would, in turn, allow interoperability
between vendors and standardized communication protocols.
• Three message types were initially defined: MMS (Manufacturing
Message Specification), GOOSE (Generic Object Oriented Substation
• Event), and SV (Sampled Values).
 4. Other Protocols

• Some specialized environments may also have other background control

protocols.
• For example, many IoT networks reach all the way to the individual
sensors, so protocols such as Constrained Application Protocol (CoAP)
and Datagram Transport Layer Security (DTLS) are used, and have to be
considered separately from a security perspective.
5. Device Insecurity

• To understand the nature of the device insecurity, it is important to review

the history of what vulnerabilities were discovered and what types of
devices were affected.
• It is not difficult to understand why such systems are frequently found
vulnerable.
• First, many of the systems utilize software packages that can be easily
downloaded and worked against.
• Second, they operate on common hardware and standard operating
systems,
• such as Microsoft Windows.
• Third, Windows and the components used within those applications are
well known to traditionally IT-focused security researchers.
 How IT and OT Security Practices and
Systems Vary
• The differences between an enterprise IT environment and an industrial-
focused OT deployment are important to understand because they have a
direct impact on the security practice applied to them.
The Purdue Model for Control Hierarchy
• Regardless of where a security threat arises, it must be consistently and
unequivocally treated.
• IT information is typically used to make business decisions, such as
those in process optimization, whereas OT information is instead
characteristically leveraged to make physical decisions, such as closing a
valve, increasing pressure, and so on.
• This model identifies levels of operations and defines each level. The
enterprise and operational domains are separated into different zones and
kept in strict isolation via an industrial demilitarized zone (DMZ):
Enterprise zone:

• Level 5: Enterprise network: Corporate-level applications such as Enterprise

Resource Planning (ERP), Customer Relationship Management (CRM), document
management, and services such as Internet access and VPN entry from the outside
world exist at this level.
• Level 4: Business planning and logistics network: The IT services exist at this level
and may include scheduling systems, material flow applications, optimization and
planning systems, and local IT services such as phone, email, printing, and security
monitoring.
Industrial demilitarized zone:

• DMZ: The DMZ provides a buffer zone where services and data can be
shared between the operational and enterprise zones. It also allows for easy
segmentation of organizational control. By default, no traffic should traverse
the DMZ;everything should originate from or terminate on this area.
Operational zone:

• Level 3: Operations and control: This level includes the functions involved in
managing the workflows to produce the desired end products and for monitoring and
controlling the entire operational system. This could include production scheduling,
reliability assurance, systemwide control optimization, security management, network
management, and potentially other required IT services, such as DHCP, DNS, and
timing.
•
Operational zone:(continue..)

• Level 2: Supervisory control: This level includes zone control rooms, controller
status, control system network/application administration, and other control related
applications, such as human-machine interface (HMI) and historian.
• Level 1: Basic control: At this level, controllers and IEDs, dedicated HMIs, and
other applications may talk to each other to run part or all of the control function.
• Level 0: Process: This is where devices such as sensors and actuators and machines
such as drives, motors, and robots communicate with controllers or IEDs.
Safety zone:

• Safety-critical: This level includes devices, sensors, and other equipment used to
manage the safety functions of the control system.
● One of the key advantages of designing an industrial network in structured levels, as
with the Purdue model, is that it allows security to be correctly applied at each level
and between levels.
● Although security vulnerabilities may potentially exist at each level of the model, it is
clear that due to the amount of connectivity and sophistication of devices and systems,
the higher levels have a greater chance of incursion due to the wider attack surface.
 Formal Risk Analysis Structures: OCTAVE
and FAIR
• Within the industrial environment, there are a number of standards, guidelines, and
best practices available to help understand risk and how to mitigate it.
• The key for any industrial environment is that it needs to address security holistically
and not just focus on technology.
• It must include people and processes, and it should include all the vendor ecosystem
components that make up a control system.
• They are two risk assessment frameworks:
• OCTAVE (Operationally Critical Threat, Asset and Vulnerability Evaluation)
from the Software Engineering Institute at Carnegie Mellon University
• FAIR (Factor Analysis of Information Risk) from The Open Group
• These two systems work toward establishing a more secure environment but with
two different approaches and sets of priorities.
• Knowledge of the environment is key to determining security risks and plays a key
role in driving priorities.
 OCTAVE

• OCTAVE has undergone multiple iterations.

• The version this section focuses on is OCTAVE Allegro, which is intended to be a
lightweight and less burdensome process to implement.
• Allegro assumes that a robust security team is not on standby or immediately at the
ready to initiate a comprehensive security review.
• This approach and the assumptions it makes are quite appropriate, given that many
operational technology areas are similarly lacking in security-focused human assets.
OCTAVE(continue..)

• The first step of the OCTAVE Allegro methodology is to establish a risk measurement
criterion.
• OCTAVE provides a fairly simple means of doing this with an emphasis on impact, value, and
measurement.
• The second step is to develop an information asset profile.
• This profile is populated with assets, a prioritization of assets, attributes associated with each
asset, including owners, custodians, people, explicit security requirements, and technology
assets.
OCTAVE(continue..)

• Within this asset profile, process are multiple substages that complete the definition of
the assets.
• Some of these are simply survey and reporting activities, such as identifying the asset
and attributes associated with it, such as its owners, custodians, human actors with which
it interacts, and the composition of its technology assets.
• The third step is to identify information asset containers.
• This is the range of transports and possible locations where the information might reside.
• This references the compute elements and the networks by which they communicate.
OCTAVE(continue..)
• The fourth step is to identify areas of concern.
• At this point, we depart from a data flow, touch, and attribute focus to
one where judgments are made through a mapping of security-related
attributes to more business-focused use cases.
• In the fifth step, where threat scenarios are identified.
• Threats are broadly (and properly) identified as potential undesirable
events.
• This definition means that results from both malevolent and accidental
causes are viable threats.
OCTAVE(continue..)

• At the sixth step risks are identified. Within OCTAVE, risk is the possibility of an
undesired outcome.
• This is extended to focus on how the organization is impacted.
• The seventh step is risk analysis, with the effort placed on qualitative evaluation of
the impacts of the risk.
• Here the risk measurement criteria defined in the first step are explicitly brought
into the process.
OCTAVE(continue..)

• Finally, mitigation is applied at the eighth step.

• There are three outputs or decisions to be taken at this stage.
• One may be to accept a risk and do nothing, other than document the situation,
potential outcomes, and reasons for accepting the risk.
• The second is to mitigate the risk with whatever control effort is required.
• The final possible action is to defer a decision, meaning risk is neither accepted
nor mitigated.
 FAIR

• FAIR (Factor Analysis of Information Risk) is a technical standard for risk

definition from The Open Group.
• While information security is the focus, much as it is for OCTAVE, FAIR has
clear applications within operational technology.
• Like OCTAVE, it also allows for non-malicious actors as a potential cause for
harm, but it goes to greater lengths to emphasize the point.
• FAIR Factors

• As FAIR comes into action, it sets its sight on the

complication of multiple factors that can establish the
risks in a given ecosystem. FAIR explains that a deeper
analysis of all these factors is important because it
provides a broader picture of every risk and helps one to
understand how these new risks are linked with existing
risks.
 How Does FAIR Work?
• FAIR is based on a highly strategic approach that involves:

• Finding the systems at risk and categorizing them into different

categories
• Spotting the possible risks for a system
• Giving the severity score to every risk from every category
• Analyzing the control ecosystem under consideration
• Finalizing the risk rating
• Organizations referring to FAIR risk assessments have the same
processing.
• ‍
 Stage #1 - Knowing The Elements Causing
Risks
• This is the first stage of the evaluation process and it
comprises two actions. The first action is to know which
all assets could be under attack and have a risk possibility.
The second action of this stage is to know the threat
resources.
• The foundation of this stage is the probability-based
model that helps in figuring out the probability of a
risk and its related damage. This stage should collect
feedback from top management, CISO, and other
cybersecurity professionals.
 Stage #2 - Evaluating Loss Event
Occurrence Rate
• In the second stage, all the action happening revolves
around collecting data related to metrics like Loss Event
Frequency, Threat Event Frequency, Control Strength,
and Threat Capability. Each of these metrics provides a
deeper insight into cybersecurity risks. For instance,
Threat Event Frequencies are the estimated frequencies
of threat agent actions that are capable of causing any
damage in a given time.
 Stage #3 - Analyzing Probable Loss
Magnitude Or PLM
• It revolves around PLM. PLM solving queries like an
anticipated loss to an organization from primary and
secondary loss events, what are worst-case scenarios,
and so on.

• This stage takes care of both the primary and secondary

losses concerning the stakeholders. Let’s understand
what these two loss types mean.
• Primary loss is incurred because of any direct loss event.
It could involve lost revenue or any outdated asset of an
organization.

• By secondary loss, we meant the loss incurred because of

events that are not directly linked with primary
stakeholders but still have a certain impact on the
organization and its key operations. Secondary risks have
definite potential to inflict and reduce the performance of
an organization.
 Stage #4 - Devising And Expressing The
Risks
• The last stage of the FAIR’s procedure involves articulating the risks to everyone who
wants to use this assessment for decision-making. The methods involved in the process are:

• Offering a crisp classification method for the risk factors

• Using a viable measurement method so that it’s possible to define the responsible risk
factors
• Offering a computational engine that is capable to link all the predetermined risks factors
• This stage involves the use of a simulation model using which computational engines can
accurately figure out the risks.
FAIR(continue..)

• For many operational groups, it is a welcome acknowledgement of existing

contingency planning.
• Unlike with OCTAVE, there is a significant emphasis on naming, with risk taxonomy
definition as a very specific target.
• FAIR places emphasis on both unambiguous definitions and the idea that risk and
• associated attributes are measurable.
• At its base, FAIR has a definition of risk as the probable frequency and probable
magnitude of loss.
FAIR(continue..)
• With this definition, a clear hierarchy of sub-elements emerges, with one side of the
taxonomy focused on frequency and the other on magnitude.
• FAIR defines six forms of loss, four of them externally focused and two internally focused.
• Of particular value for operational teams are productivity and replacement loss.
• Response loss is also reasonably measured, with fines and judgments easy to measure but
difficult to predict.
• Finally, competitive advantage and reputation are the least measurable.