Checking Function-Level

Kernel Control
Flow Integrity
for Cloud Computing
SARTHAK BODAKE
TINKU KUMAR
2021BCY0024
2021BCY0022
“With Cloud Computing, it is no longer a question
AKHILESH ZENDE 2021BCY0033 of If, but rather When and How. “
INTRODUCTION Title: Overview of Control Flow Data
Collection

Brief Explanation: The data collection

module is fundamental to our detection
method, utilizing Virtual Machine
Introspection (VMI) technique to trace
function-level control flow and
extract VM kernel memory.

We focus on system call control flows

for monitoring critical kernel
operations, considering functions as
the natural abstraction of execution
routines.
Significance of domain
1. Cloud Computing Significance:

Resource Isolation and Management:

Domains in cloud computing help

achieve effective resource
isolation, allowing multiple users
or applications to coexist on the
same physical infrastructure.

This ensures efficient resource

utilization and improves overall
system performance.
Significance of domain
2. Cloud Security Significance: 3. Kernel Level Security for System Calls using
Kernel Functions:
Isolation of Workloads:
Control Flow Monitoring:
Domains play a crucial role in
cloud security by isolating Kernel-level security for system calls involves
workloads. monitoring the control flow within the kernel.

This prevents unauthorized access This is significant for detecting and preventing
and interference between different malicious activities, such as rootkits, that may
users or applications, enhancing attempt to compromise system integrity through
the overall security posture of the unauthorized manipulation of system call flows.
cloud environment.
Significance of domain
Kernel Level
Sandboxing:

This doesn’t require

any change to the
application binary or
the way a process is
invoked.

This can introduce

significant performance
overhead, particularly
for systems which rely
heavily on system
calls.
Significance of domain User Level Sandboxing:

This information,
along with the system
call arguments, is
sent to the policy
engine.

This approach
typically has higher
overhead thana kernel-
based solution.

Also, the application

which requires
isolation will need to
run under a parent
sandboxing process.
Significance of domain
Delegation Based Sandboxing:

In this approach, when a user

process invokes a system call
the system makes a callback to
an emulation library in the
user space, which provides
isolation service.

The callback function in the

emulation library calls a user-
space delegation agent, which
actually executes the system
call on behalf of the user
application.
Literature review
Sr Research Paper Year Journal/ Paper Findings and Novelty
No Conference

1 Ye, Lin, Xiangzhan Yu, Lei Yu, Bin 2018 IEEE In this paper, we propose a
Guo, Dongyang Zhan, Xiaojiang Du, function-level kernel CFI checking
and Mohsen Guizani. "Checking approach to meet the performance
Function-Level Kernel Control Flow requirements in the cloud. By
Integrity for Cloud combining the static memory
Computing." IEEE Access 6 (2018): analysis and the dynamic tracing,
41856-41865. our system can achieve high
detection capability with low
overhead.

2 Tabrizchi, Hamed, and Marjan The narrative review presented in

Kuchaki Rafsanjani. "A survey on 2020 IEEE this survey provides cloud security
security challenges in cloud issues and requirements, identified
computing: issues, threats, and threats, and known vulnerabilities.
solutions." The journal of
supercomputing 76, no. 12 (2020):
9493-9532.
MOTIVATION and research gap
❖ With the advancement of cloud computing, the control flow integrity (CFI) of
virtual machines 'kernel becomes more and more important for the security of
cloud services.

❖ Many CFI checking and protecting approaches have been proposed. Among them,
dynamic analysis approaches have the best detection capability, but they are
rarely used because of the high overhead introduced to the virtual machine to be
monitored.

❖ In this paper, we propose a function-level kernel CFI checking approach to meet

the performance requirements in the cloud. By combining the static memory
analysis and the dynamic tracing, our system can achieve high detection
capability with low overhead.
MOTIVATION and research gap
❖ Since the analysis and tracing targets of our system are kernel functions, our
system incurs lower overhead to the monitored virtual machines than the
instruction-level monitors.

❖ We propose two models to describe the kernel control flows. After building the
secure control flow database by learning the normal behaviors, we can detect
abnormal control flows in real time.

❖ With the help of virtualization and virtual machine introspection techniques, we

implement a prototype system in the hardware virtualization environment. From the
evaluation, our system has high detection capability with reasonable overhead.
Objectives from the paper
❖ Threat Model and System Overview

❖ Control Flow Data Collection

❖ Control Flow Separation
❖ Instruction Analysis
❖ Instruction Injections
❖ Code Block Obtaining and Protection

❖ Abnormality Detection and Control Flow

❖ Verified Function Detection
❖ Short Sequences

❖ Optimization
❖ Monitoring Location
❖ Instruction Injection Timing
❖ Evaluation
THREAT MODEL AND SYSTEM OVERVIEW
What is Control flow integrity (CFI) ? It's a security
technique that ensures
“CFI is like a security checkpoint on a programs or systems
highway, diligently checking every vehicle only follow legitimate
•It's a security technique that ensures programs or systems only follow legitimate execution paths.
(instruction) to ensure it has proper execution paths.
•CFI can ensure VM security.
authorization and
•CFI follows
specifically focuses the designated
on the kernel, the core part of the operating system with privileged access
route” CFI can ensure VM
security.

CFI specifically
focuses on the kernel,
the core part of the
operating system with
privileged access
THREAT MODEL AND SYSTEM OVERVIEW
Memory analysis Dynamical tracing

Memory analyzing methods inspect Dynamical tracing methods involve tracing

the execution memory directly to the execution of the target system and
detect any abnormal control analyzing the extracted execution
flows. information to check CFI in real time.

Low overhead on the system. High overhead due to tracing instructions

or branches
It can miss the execution
information which is very Captures rich execution information for
important aspect of analysis. better detection.
THREAT MODEL AND SYSTEM OVERVIEW
Virtual Machine Introspection (VMI) Technique

It traces and analyze kernel function-level memory and execution information out
of the virtual machine to check kernel CFI.

Monitor targets at kernel functions instead of instructions, the monitored

virtual machine is intercepted only when a new kernel function is executed.
Therefore low overhead.
THREAT MODEL AND SYSTEM OVERVIEW
Cloud VMI virtualizes the VMI interface as a cloud service that is offered to
cloud customers.

CLAW is a cloud based application whitelisting system, which leverages VMI

technique to guarantee that only pre-approved application binaries are allowed to
run in virtual machines.
THREAT MODEL AND SYSTEM OVERVIEW
Threat Model

It investigates potential risks to the integrity of the operating system (OS)

kernel, particularly focusing on kernel rootkits.

These rootkits typically tamper with kernel addresses and alter the normal
behavior of system calls to conceal themselves, eavesdrop on users, or manipulate
the kernel.

The primary method used by rootkits is hijacking control flows,

THREAT MODEL AND SYSTEM OVERVIEW
The primary method used by
rootkits is hijacking control
flows, which can be achieved
through three main approaches:

• Modifying addresses stored in

the system call table
• Modifying subroutines of
system calls
• Modifying kernel objects to
redirect control flow
THREAT MODEL AND SYSTEM OVERVIEW
Modifying addresses stored in
the system call table:

Rootkits can replace pointers

to normal system call routines
with their own malicious
routines.

This allows them to control

system behavior intentionally,
such as hiding their presence
from detection.
THREAT MODEL AND SYSTEM OVERVIEW
Modifying subroutines of system calls:

Rootkits can alter the subroutines of system calls, making it difficult to

detect the modifications since they can occur anywhere in the system call
routine.

Modifying kernel objects to redirect control flow:

Modifying kernel objects to redirect control flow: Certain attacks, like direct
kernel object manipulation (DKOM), involve tampering with kernel objects to
change the control flow of the kernel.
CONTROLFLOW AND DATA COLLECTION
CONTROL FLOW SEPERATION
There are many running control flows generated from different processes simultaneously, which
demonstrates inter twined and complex relationships among functions. For operating system
kernels, using the system calls are the best way to differentiate the control flows.

Therefore, the first step is to find the starting and ending point to obtain a complete
control flow of a system call.

Each system call has its own starting and ending points, which can be served as the basis to
separate control flows from each other.

Starting with Windows XP and Linux 2.6, an alternative system call entry mechanism named “fast
system Call” is introduced where companion instructions SYSENTER/SYSEXIT [16] are invoked to
enter/exit the system call.
CONTROLFLOW AND DATA COLLECTION
INSTRUCTION ANALYSIS
In order to identify the functions, we first analyze the instructions to find out the positions
of all functions in the control flow, which involves a static analysis and dynamic tracing
process.

In the static analysis process, we analyze the instructions from the entrance of each function
to find the next function in the control flow. We also need to obtain the code block between
the start and end of a function for further analysis.

Categories of Instruction:

• Branching Instructions
• Non- Branching Instructions
CONTROLFLOW AND DATA COLLECTION
INSTRUCTION ANALYSIS
With the help of VMI, it recognizes the address and content of the instruction to further inspect
the target address that the program jumps to based on the semantics of the instruction.

This procedure continues until it finishes parsing the whole content of the function. However,
this method can only analyze the direct branch instructions.

There are two kinds of branch instructions:

direct branch instructions and indirect branch instructions.

An indirect branch is also known as a computed jump, indirect jump and register-indirect jump, so
we cannot get the target address through static analysis.

In order to build a complete control flow, we leverage dynamic tracing to follow up the indirect
branches.
CONTROLFLOW AND DATA COLLECTION
INSTRUCTION INJECTIONS
Since the indirect branch instructions are transparent to the VMM, they do not involve VMM
operations.

To trace these instructions from VMM, we lift the execution of these instructions to the VMM
layer by employing the Intel Virtualization Technology (Intel VT) such that the VMM can be
aware of their states.

Intel VT offers a set of hardware extensions that make virtualization practical and efficient
on the x86 platform.

To facilitate virtualization, CPU operation can work in two different modes: VMX root operation
and VMX non-root operation.
CONTROLFLOW AND DATA COLLECTION
INSTRUCTION INJECTIONS

Thus, in this paper we make use of interrupts to trap functions for Intel VT switches.

INT 3 instruction is a privileged instruction causing VM exit events. By leveraging INT 3

instruction, we can inject breakpoints to the monitored VM kernel to cause a VM exit
transition
CONTROLFLOW AND DATA COLLECTION
CODE BLOCK OBTAINING AND PROTECTION
After obtaining all the functions of the system call, we get the memory code of each function
for integrity checking.

To this end, we map the memory to the secure VM by leveraging the Xen toolkit. The code of
functions can be obtained by reading the mapped memory according to their virtual addresses.

Checking the code blocks can ensure the integrity of all executed codes during the monitored
system call.

In addition, we leverage the Intel Extended Page Table (Intel EPT) technique to protect the
code blocks of the functions.

As a result, we need not to read them again when they are re-executed. Furthermore, we can also
prevent the time-of-check-time-of-use (TOCTOU) attacks.
ABNORMALITY DETECTION CONTROL FLOWS
VERIFIED FUNCTION DETECTION
The verified function detection aims at the validation of the memory addresses and code blocks
of different functions in a control flow, based on the fact that each memory address and code
block in a valid control flow should be valid.

In other words, as long as the whitelist of valid memory addresses and code block can be
learned or maintained in a trusted environment, the control flow containing the memory
addresses or code blocks absent from the white list can be determined as a compromised one.

For instance, when some rootkits replace the normal functions with its own defined ones, the
presence of ‘alien’ memory addresses and code blocks in a control flow can indicate the
abnormality of the control flow.

In the learning stage, even though two flows F1 (A1 → A2 → A3) and F2 (A2 → A3 → A1) have
different execution sequences, three valid memory addresses A1, A2, A3 and their code blocks
C1, C2, C3 are learned equivalently from F1 and F2.
ABNORMALITY DETECTION CONTROL FLOWS
SHORT SEQUENCE
Automaton is usually used to describe the relationships between different states. However,
building an automaton for the system call control flows is difficult.

.This model uses the sequence of the division and the formation of a unique short sequence to
model normal behaviors.

In the detection, the sequence to be detected is also divided into short sequences and compared
with the normal ones, and the proportion of non-matching is used to determine the exception.
ABNORMALITY DETECTION CONTROL FLOWS
SHORT SEQUENCE

By following the same routine of the learning phase, a control flow is divided into several short
sequences in the detection phase.

Otherwise, once a short sequence does not exist in the database, the control flow is identified as
an invalid one. However, considering that it is impossible to record all valid short sequences, we
set a threshold for the ratio between the number of absent short sequences and the total number of
short sequences. When its ratio exceeds the threshold, the control flow will be identified as
invalid.
 Optimization
There are three ways to improve the performance of our system.
Monitoring location
• we change the monitoring location from the start and end addresses of the fast
system call mechanism to the entry address of the specific system call.
• This adjustment ensures more controlled triggering of monitoring, avoiding
unnecessary performance losses.
 Optimization
2. Instruction injecting timing
• we modify the timing of instruction injection into the target system call.
• Instead of injecting instructions when the detection system starts, we initialize
injection and analysis only when the system call begins to run.
• This prevents overhead when the system call is inactive.

.
 Optimization
3. Polling mechanism
• we implement a polling mechanism, turning the detection system on and off at
predefined intervals.
• This improvement is based on the observation that malicious behaviors often
persist for a while, allowing effective detection with reduced overhead
distributed over time.
• Balancing detection accuracy and system performance requires careful consideration
of the polling interval.
Evaluation
It is done on the basis of Effectiveness and performance.
Effectiveness:
• Validation through Manual Examination: The system's effectiveness was confirmed by
manually examining 15 rootkits.
• High Detection Success: It successfully identified and detected all instances of
rootkits, showcasing its capability to pinpoint malicious activities.
 Evaluation
Effectiveness:
• Low False Positive Rate: The system exhibited a commendable 1% false positive
rate, indicating its precision in distinguishing between malicious and non-
malicious activities.
• Comprehensive analysis: The system's validation included carefully looking at
source codes and tracking how the program runs. This highlights the system's
strong capability to accurately identify malicious behavior.
 Evaluation
Performance
• Reasonable Performance: The system performed reasonably well during evaluations.
• Overhead Measurements: The increase in execution time, measured through system
calls and file operations, ranged from 7.5% to 22%.
• Polling Optimization Strategy: To minimize the impact on performance, the system
used a polling strategy. This reduced the overall loss to less than 1% when using
larger intervals.
 Evaluation
Performance
• Efficient Detection: Despite performance considerations, the approach ensured
efficient detection while keeping the system responsive.
• Acceptable Increase in Execution Time: The overhead analysis revealed a reasonable
increase in execution time, ranging from 7.5% to 22%. This indicates that the
system did not significantly slow down the computer.
 Evaluation
Performance
• Mitigating Performance Impact: To address potential performance impacts, a polling
optimization strategy was implemented. This strategy aimed to reduce the overall
loss in system performance.
CONCLUSION
❖ We Have Discussed Threat Model and System Overview

❖ Then We had a Extensive Review of Control Flow Data Collection

❖ We Moved to Control Flow Separation
❖ And then Completed Instruction Analysis
❖ Also the study of Instruction Injections
❖ Code Block Obtaining and Protection

❖ Then we moved our Attention towards Abnormality Detection and Control Flow
❖ We also Covered Verified Function Detection
❖ And Also Short Sequences

❖ Finally We had a Brief Review of Optimization

❖ Monitoring Location
❖ Instruction Injection Timing
❖ At Last on ending node we discussed Evaluation
REFERENCES
1. Ye, Lin, Xiangzhan Yu, Lei Yu, Bin Guo, Dongyang Zhan, Xiaojiang Du, and Mohsen Guizani. "Checking
Function-Level Kernel Control Flow Integrity for Cloud Computing." IEEE Access 6 (2018): 41856-41865.

2. Tabrizchi, Hamed, and Marjan Kuchaki Rafsanjani. "A survey on security challenges in cloud computing:
issues, threats, and solutions." The journal of supercomputing 76, no. 12 (2020): 9493-9532.
[1] Z. Liao and Y. Luo, ‘‘A stack-based lightweight approach to detect [7] J. Zeng, Y. Fu, and Z. Lin, ‘‘PEMU: A pin highly compatible
kernellevel rookits,’’ in Proc. IEEE Int. Conf. Prog. Inform. Comput., Dec. out-of-VM dynamic binary instrumentation framework,’’ ACM SIGPLAN
2015, pp. 602–607. Notices, vol. 50, no. 7, pp. 147–160, 2015.
[2] A. Prakash, H. Yin, and Z. Liang, ‘‘Enforcing system-wide control flow
[8] J.-L. Danger, S. Guilley, T. Porteboeuf, F. Praden, and M.
integrity for exploit detection and diagnosis,’’ in Proc. 8th ACM SIGSAC Symp.
Timbert, ‘‘HCODE: Hardware-enhanced real-time CFI,’’ in Proc.
Inf., Comput. Commun. Secur., 2013, pp. 311–322.
4th Program Protection Reverse Eng. Workshop, 2014, Art. no. 6.
[3] T. Garfinkel and M. Rosenblum, ‘‘A virtual machine introspection based
architecture for intrusion detection,’’ in Proc. NDSS, vol. 3. 2003, pp. 191– [9] J. Li, Z. Wang, T. Bletsch, D. Srinivasan, M. Grace, and X.
206. Jiang, ‘‘Comprehensive and efficient protection of kernel control
data,’’ IEEE Trans. Inf. Forensics Security, vol. 6, no. 4, pp.
[4] O. S. Hofmann, A. M. Dunn, S. Kim, I. Roy, and E. Witchel, ‘‘Ensuring 1404–1417, Dec. 2011.
operating system kernel integrity with OSck,’’ ACM SIGARCH Comput. Archit.
News, vol. 39, no. 1, pp. 279–290, 2011. [10] C. Malone, M. Zahran, and R. Karri, ‘‘Are hardware
performance counters a cost effective way for integrity checking of
[5] J. Rhee, R. Riley, D. Xu, and X. Jiang, ‘‘Kernel malware analysis with un-
programs,’’ in Proc. 6th ACM Workshop Scalable Trusted Comput.,
tampered and temporal views of dynamic kernel memory,’’ in Proc. Int. Workshop
2011, pp. 71–76.
Recent Adv. Intrusion Detection. Berlin, Germany: Springer, 2010, pp. 178–197.

[6] N. L. Petroni, Jr., and M. Hicks, ‘‘Automated detection of persistent [11] X. Wang and R. Karri, ‘‘NumChecker: Detecting kernel control-
kernel control-flow attacks,’’ in Proc. 14th ACM Conf. Comput. Commun. Secur., flow modifying rootkits by using hardware performance counters,’’
2007, pp. 103–115. in Proc. 50th ACM/EDAC/IEEE Des. Automat. Conf., May/Jun. 2013, pp.
1–7.
THANK YOU