Processor Architecture Security

Part 1: Processor Security

and Secure Processors

Jakub Szefer
Assistant Professor
Dept. of Electrical Engineering
Yale University

ACACES 2019 – July 14th - 20th, 2019

Slides and information available at: https://caslab.csl.yale.edu/tutorials/acaces2019/
ACACES Course on Processor Architecture Security
1
© Jakub Szefer 2019
Principles of Computer Architecture

Traditional computer architecture has six principles regarding processor design:

E.g. caching frequently used data in a small but fast memory helps hide data
access latencies.
• Caching E.g. break processing of an instruction into smaller chunks that can each be
executed sequentially reduces critical path of logic and improves
• Pipelining performance.

E.g. predict control flow direction or data values before they are actually
• Predicting computed allows code to execute speculatively.

• Parallelizing E.g. processing multiple data in parallel allows for more computation to be
done concurrently.
• Use of indirection E.g. virtual to physical mapping abstracts away physical details of the system.

• Specialization E.g. custom instructions use dedicated circuits to implement operations that
otherwise would be slower using regular processor instructions.

What are principles for securing processors?

ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 2
Processor Security and Secure Processors

Processor security focuses on ensuring

Confidentiality, Integrity, and Availability
from attacks by intelligent adversaries General
• Reliability is about random errors Purpose
• Security is about “smart” attackers Processors

Course focus: architecture and hardware

• Many attacks exist on software Secure
Processors Security
• Focus on attacks abusing hardware Co-processors,
Accelerator, or
Secure processors: Monitor
• Subset of processors with extra security features
• Provide extra logical isolation for software
• Vulnerable to similar attacks as regular processors
Security Attacks
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 3
Outline

Part 1: Processor Security and Secure Processors

• Present features of secure processors
• Contrast to conventional processors

Part 2: Side and Covert Channels

• Detail side and covert channel attacks on conventional processors and secure processors
• Focus on timing channels

Part 3: Securing Caches, Buffers, TLBs, and Directories

• Present defenses for timing channels in the memory hierarchy
• Solutions for conventional processors and secure processors

Part 4: Transient Execution Attacks and Mitigations

• Discuss attacks leveraging transient execution (and timing channels) and defenses
for conventional processors and secure processors
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 4
Secure Processor Architectures

Secure Processor Architectures extend a processor with hardware (and related software) features
for protection of software
• Protected pieces of code and data are now commonly called Enclaves
• But can be also Trusted Software Modules, whole Operating Systems, or Virtual Machines
• Focus on the main processor in the system
• Others focus on co-processors, cryptographic accelerators, or security monitors
• Add more features to isolate secure software from other, untrusted software
• Includes untrusted Operating System or Virtual Machines
• Many also consider physical attacks on memory
• Isolation should cover all types of possible ways for information leaks
• Architectural state
Most recent threats, i.e. Spectre, etc.
• Micro-architectural state
• Due to spatial or temporal Side and covert channel threats
sharing of hardware

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 5
Brief History of Secure Processor Architectures

Starting with a typical baseline processor, many secure architectures have been proposed

Starting in late 1990s or early 2000s, academics have shown increased interest in secure processor
architectures:
XOM (2000), AEGIS (2003), Secret-Protecting (2005), Bastion (2010),
NoHype (2010), HyperWall (2012), Phantom (2013), CHERI (2014), Sanctum (2016),
Keystone (about 2017), Ascend (2017), MI6 (2018)

Commercial processor architectures have also included security features:

LPAR in IBM mainframes (1970s), Security Processor Vault in Cell Broadband Engine (2000s),
ARM TrustZone (2000s), Intel TXT & TPM module (2000s), Intel SGX (mid 2010s),
AMD SEV (late 2010s)

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 6
Baseline (Unsecure) Processor Hardware

Typical computer system with no secure components nor secure processor

architectures considers all the components as trusted:

Processor Chip
Information can be extracted
Core Core from memory or memory
… contents can be modified.
$ $ $ $

$ Snooping on the system

bus is possible to extract
Uncore
information.

Compromised or malicious
Memory devices can attack other
I/O Devices
components of the system.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 7
Baseline (Unsecure) Processor Software

Typical computer system uses ring-based protection scheme, which gives most privileges
(and most trust) to the lowest levels of the system software:

App App App Compromised or malicious

App App App
Ring 3 App App App OS can attack all the
applications in the system.
Guest Guest … Guest
Ring 0 OS OS OS Compromised or malicious
Hypervisor can attack all
Ring -1 Hypervisor (VMM) the OSes in the system.

Hardware

Image:
https://commons.wikimedia.org/wiki/File:Priv_rings.svg
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 8
Logical Isolation with New Privilege Levels

Modern computer systems define protections in terms of privilege level or protection rings,
new privilege levels are defined to provide added protections.

Ring 3 Application code, least privileged.

Rings 2 and 1 Device drivers and other semi-privileged
code, although rarely used.
Ring 0 Operating system kernel.
Ring -1 Hypervisor or virtual machine monitor (VMM),
most privileged mode that a typical system
administrator has access to.
Ring -2 System management mode (SMM),
typically locked down by processor manufacturer
Ring -3 Platform management engine, retroactively named “ring -3”,
actually runs on a separate management processor. Image:
https://commons.wikimedia.org/wiki/File:Priv_rings.svg
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 9
Extend Linear Trust to the New Protection Levels

The hardware is most privileged as it is the lowest level in the system.

• There is a linear relationship between

protection ring and privilege (lower ring App App App
App
App App
Ring 3 App App App
is more privileged)
Guest Guest … Guest
• Each component trusts all the software Ring 0 OS OS OS
“below” it Ring -1 Hypervisor (VMM)
Ring -2 SMM
Ring -3 SecE
Security Engine (SecE)
Hardware
can be something like
Intel’s ME or AMD’s PSP.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 10
Add Horizontal Privilege Separation

New privileges can be made orthogonal to existing protection rings.

Normal Privileged
• E.g. ARM’s TrustZone’s “normal” and “secure” worlds Operation Operation

• Need privilege level (ring number)

and normal / secure privilege
Security levels from a lattice:
Ring -1
Privileged

Ring -1
Ring 0
Normal
Privileged

Ring 0
Normal Ring 3
Privileged

Ring 3
Normal Image:
https://commons.wikimedia.org/wiki/File:Priv_rings.svg
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 11
Breaking Linear Hierarchy of Protection Rings

Examples of architectures that do and don’t have a linear relationship between

privileges and protection ring level:

App
App App
App App
App App
App
Encl
Ring 3 App Ring 3 App
TSM
Ring 3 App
ave Ring 3 App

Guest Guest Guest Guest

Ring 0 OS Ring 0 OS Ring 0 OS Ring 0 OS

Ring -1 HV Ring -1 HV Ring -1 HV Ring -1 HV

Ring -2 SMM Ring -2 SMM Ring -2 SMM Ring -2 SMM
Ring -3 SecE Ring -3 SecE Ring -3 SecE Ring -3 SecE

Hardware Hardware Hardware Hardware

Normal Computer E.g. Bastion E.g. SGX E.g. SEV

ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 12
Providing Protections with a Trusted Processor Chip

Key to most secure processor architecture designs is the idea of trusted processor chip as the
security wherein the protections are provided.

Processor Chip
Core Core Whole processor chip is
… trusted.
$ $ $ $

$ Memory is untrusted.
Uncore
System bus is untrusted.

Devices are untrusted.

Memory I/O Devices

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 13
Limitations of Secure Processors

Threats which are outside the scope of secure processor architectures:

TCB hardware and software is prone to
bugs just like any hardware and software.
• Bugs or Vulnerabilities in the TCB
• Hardware Trojans and Supply Chain Attacks Modifications to the processor after the
design phase can be sources of attacks.
• Physical Probing and Invasive Attacks
At runtime hardware can be probed to
extract information from the physical
realization of the chip.

Threats which are underestimated when designing secure processor architectures:

• Side Channel Attacks Information can leak through timing,
power, or electromagnetic emanations
from the implementation
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 14
TEE and TCB

The Trusted Computing Base (TCB) is the set of hardware and software that is responsible
for realizing the TEE:
• TEE is created by a set of all the components in the TCB
• TCB is trusted to correctly implement the protections
• Vulnerability or successful attack on TCB nullifies TEE protections

• TCB is trusted
• TCB may not be trustworthy, if is not verified or is not bug free

The goal of Trusted Execution Environments (TEEs) is to provide protections for

a piece of code and data from a range of software and hardware attacks.
• Multiple mutually-untrusting pieces of protected code can run on a system at the same time

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 15
TEEs and Software They Protect

Different architectures mainly focus on protecting Trusted Software Modules (a.k.a. enclaves)
or whole Virtual Machines.

Some TEEs have support

App App App
App for protecting whole virtual
App App
Encl
App App
ave App machines.

Guest Guest … Guest

OS OS OS Other TEEs support
Trusted Software Modules,
Hypervisor (VMM) a.k.a. enclaves
SMM
SecE

Hardware

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 16
Protections Offered by Secure Processor Architectures

Security properties for the TEEs that secure processor architectures aim to provide:

• Confidentiality Confidentiality is the prevention of the disclosure of secret or sensitive

information to unauthorized users or entities.
• Integrity
Integrity is the prevention of unauthorized modification of protected
information without detection.

• Availability is usually not provided usually

Confidentiality and integrity protections are from attacks by other components (and hardware) not in
the TCB. There is typically no protection from malicious TCB.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 17
Protections Categorized by Architecture

Secure processor architectures break the linear relationship (where lower level
protection ring is more trusted):

SecE
SMM and SecE are always
trusted today, no
architecture explores
design where these levels
are untrusted. ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 18
Protecting State of the Protected Software

Protected software’s state is distributed throughout the processor. All of it needs to be protected
from the untrusted components and other (untrusted) protected software.

• Protect memory through encryption Processor Chip

and hashing with integrity trees
Core Core
• Flush state, or isolate state, …
$ $ $ Reg. FU State
of functional units in side processor cores
• Isolate state in uncore Cache
and any security modules
SecE Uncore
• Isolate state in I/O and other subsystems

Memory I/O Devices

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 19
Ideal No Side-Effects Execution

Secure processor architectures ideally have no side-effects which are visible to the untrusted
components whenever protected software is executing.

1. System is in some state Processor Chip

before protected software runs
Core Core
…
$ $ $ Reg. FU State
2. Protected software runs
modifying system state Cache
SecE Uncore
3. When protected software is
interrupted or terminates
the state modifications
are erased
Memory I/O Devices

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 20
No Protections from Protected Software

The software (code and data) executing within TEE protections is assumed to be benign
and not malicious:

• Goal of Secure Processor Architectures is to create minimal TCB that realizes a TEE
within which the protected software resides and executes

• Secure Processor Architectures can not protect software if it is buggy or has vulnerabilities

Code bloat endangers invalidating assumptions about benign protected software.

Attacks from within protected software should be defended.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 21
Hardware TCB as Circuits or Processors

Key parts of the hardware TCB can be implemented as dedicated circuits or

as firmware or other code running on dedicated processor

• Custom logic or hardware Processor Chip

state machine:
Core Core
• Most academic proposals …
$ $ $ Reg. FU State
• Code running on dedicated Cache
processor:
• Intel ME = ARC processor SecE Uncore
or Intel Quark processor
• AMD PSP = ARM processor

Memory I/O Devices

Vulnerabilities in TCB “hardware” can lead
to attacks that nullify the security protections
offered by the system.
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 22
Ensuring Trustworthy TCB Execution

Trustworthiness of the TCB depends on the ability to monitor the TCB code
(hardware and software) execution as the system runs.

TCB should be monitored to ensure it is trustworthy.

Monitoring of TCB requires mechanisms to:

• Fingerprint and authenticate TCB code
• Monitor TCB execution
• Protect TCB code (on embedded security processor)
• Virtual Memory, ASLR, …

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 23
Root of Trust for TCB

Security of the system is derived from a root of trust.

• A secret (cryptographic key)

only accessible to TCB components Processor Chip
Core Core
…
• Derive encryption and signing keys $ $ $ Reg. FU State
from the root of trust
KR Cache
SecE Uncore
Hierarchy of keys can be derived
from the root of trust

KR
Memory I/O Devices
Km KSK KPK …

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 24
Root of Trust and Processor Key

Each processor requires a unique secret.

• Burn in at the factory by the manufacturer

(but implies trust issues with manufacturer Processor Chip
and the supply chain) Core Core
• E.g. One-Time Programmable (OTP) fuses …
$ $ $ Reg. FU State

KR Cache
• Use Physically Uncloneable Functions
(but requires reliability) SecE Uncore
• Extra hardware to derive keys from PUF
• Mechanisms to generate and distribute
certificates for the key
Memory I/O Devices

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 25
Derived Keys and Key Distribution

Derived form the root of trust are signing and verification keys.

• Public key, KPK, for encrypting data

to be sent to the processor
• Data handled by the TCB Cert
ID, KPK

• Signature verification key, KVK, for checking Cert Cert

data signed by the processor ID, KVK ID
ID, KPK
• TCB can sign user keys
Processor Chip Cert
ID, KVK
• Key distribution for PUF based KR ID
designs will be different
KSigK KSK

Emoji Image:
• Need infrastructure! https://www.emojione.com/emoji/1f3ed
https://www.emojione.com/emoji/1f469-1f4bc
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019
https://www.emojione.com/emoji/1f3e2 26
Software Measurement

With an embedded signing key, the software running in the TEE can be “measured” to attest to
external users what code is running on the system.

App App App

App
App App
App App App

Guest Guest … Guest Some

OS OS OS When all levels architectures,
are trusted, e.g. SGX or
Hypervisor (VMM) compute SEV, “skip”
cryptographic untrusted
SMM
hashes over layers when
SecE code and data computing
of each level. hashes
Hardware

Emoji Image:
https://www.emojione.com/emoji/1f469-1f4bc
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 27
Using Software Measurement

Trusted / Secure / Authenticated Boot:

• Abort boot when wrong measurement is obtained
• Or, continue booting but do not decrypt secrets
• Legitimate software updates will change measurements, may prevent correct boot up
Remote attestation:
• Measure and digitally sign measurements that are sent to remove user
Data sealing (local or remote):
• Only unseal data if correct measurements are obtained
TOC-TOU attacks and measurements:
• Time-of-Check to Time-of-Use (TOC-TOU) attacks leverage the delay between when a
measurement is taken, and when the component is used
• Cannot easily use hashes to prevent TOC-TOU attacks
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 28
Need for Continuous Monitoring of Protected Software

Continuous monitoring is potential solution to TOC-TOU:

• Constantly measure the system, e.g. performance counters, and look for anomalies
• Requires knowing correct and expected behavior of system
• Can be used for continuous authentication

Attacker can “hide in the noise” if they change the execution of the software slightly and do not affect
performance counters significantly.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 29
Memory Protections in Secure Processors

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 30
Sources of Attacks on Memory

Memory is vulnerable to different types of attacks:

a) Untrusted software running no the processor
b) Physical attacks on the memory bus, other devices snooping on the bus, man-in-the-middle
attacks with malicious device
c) Physical attacks on the memory (Coldboot, …)
d) Malicious devices using DMA or other attacks

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 31
Types of Attacks on Memory

Different types of attacks exist (very similar to attacks in network settings):

Passive attack, try to read data contents.

• Snooping
Active attack, inject new memory
commands to try to read or modify data.
• Spoofing
Active attack, combine portions of legitimate
memory commands into new memory
• Splicing commands (to read or modify data).

Active attack, re-send old memory

• Replay command (to read or modify data).

Active attack, DoS on memory bus,

• Disturbance repeated memory accesses to age circuits,
repeated access to make Rowhammer, etc.
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 32
Confidentiality Protection with Encryption

Contents of the memory can be protected with encryption. Data going out of the CPU is encrypted,
data coming from memory is decrypted before being used by CPU.

a) Encryption engine (usually AES in CTR mode) encrypts data going out of processor chip
b) Decryption engine decrypts incoming data Pre-compute encryption pads, then only
need to do XOR; speed depends on how
well counters are fetched / predicted.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 33
Integrity Protection with Hash Trees

Hash tree (also called Merkle Tree) is a logical three structure, typically a binary tree, where two
child nodes are hashed together to create parent node; the root node is a hash that depends on
value of all the leaf nodes.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 34
Integrity Protection with Hash Trees

Memory blocks can be the leaf nodes in a Merkle Tree,

On-chip (cached) nodes are
the tree root is a hash that depends assumed trusted, used to
on the contents of the memory. speed up verification.
Cache

Counters are included in Hash tree nodes are stored in

hashes for freshness. (untrusted) main memory.
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 35
Integrity Protection with Bonsai Hash Trees

Message Authentication Codes (MACs) can be used instead of hashes, and a smaller
“Bonsai” tree can be constructed.

Key for MAC

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 36
Integrity Protection of Selected Memory Regions

• For encryption, type of encryption does not typically depend on memory configuration
• For integrity, the integrity tree needs to consider:
• Protect whole memory E.g., Bastion’s memory integrity tree
• Protect parts of memory (e.g. per application, per VM, etc.) (Champagne, et al., HPCA ‘10)
• Protect external storage (e.g. data swapped to disk)

Memory Data on Disk

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 37
Memory Access Pattern Protection

Snooping attacks can target extracting data (protected with encryption)

or extracting access patterns to learn what a program is doing.
• Easier in Symmetric multiprocessing (SMP) due to shared bus
• Possible in other configuration if there are untrusted components

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 38
Memory Access Pattern Protection

Access patterns (traffic analysis) attacks can be protected with use Oblivious RAM, such as Path
ORAM. This is on top of encryption and integrity checking.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 39
Leveraging 2.5D and 3D Integration

With 2.5D and 3D integration, the memory is brought into the same package as the main processor
chip. Further, with embedded DRAM (eDRAM) the memory is on the same chip.
• Potentially probing attacks are more difficult
• Still limited memory (eDRAM around 128MB in 2017)

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 40
Security of Non-Volatile Memories and NVRAMs

• Non-volatile memories (NVMs) can store data even when there is no power
• Non-volatile random-access memory (NVRAM) is a specific type of NVM that is suitable to serve
as a computer system’s main memory, and replace or augment DRAM

• Many types of NVRAMs:

• ReRAM – based on memristors, stores data in resistance of a dialectric material
• FeRAM – uses ferroelectric material instead of a dialectric material
• MRAM – uses ferromagnetic materials and stores data in resistance of a storage cell
• PCM – typically uses chalcogenide glass where different glass phases have different resistances

Security considerations
• Data remanence makes passive attacks easier (e.g. data extraction)
• Data is maintained after reboot or crash (security state also needs to be correctly restored after
reboot or crash)

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 41
Features of Systems using NVRAMs

Persistence:
• Data persists across reboots and crashes, possibly with errors Persistence
domain
• Need atomicity for data larger than one memory word example
(either all data or no data is “persisted”)
• E.g. Write Pending Queue (WPQ) – memory controller
has non-volatile storage or enough stored charge to write
pending data back to the NV-DIMM or NVRAM

Granularity of persistence:
• Hide non-volatility from the system: simply use memory as DRAM replacement
• Expose non-volatility to the system: allow users to select which data is non-volatile
• Linux support through Direct Access (DAX) since about 2014
• Developed for NV-DIMMs (e.g., battery backed DRAM, but works for NVRAMs)
Image:
https://www.usenix.org/system/files/login/articles/login_summer17_07_rudoff.pdf
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 42
Integrity Protection of NVRAMs

• For integrity, the integrity tree needs to additionally consider:

• Atomicity of memory updates for data and related security state (so it is correct after reboot or a crash)
• Which data in NVRAM is to be persisted (i.e. granularity)

NVRAM Data on Disk

Persistent Data
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 43
Encrypted, Hashed, Oblivious Access Memory Assumption

Off-chip memory is untrusted and the contents is assumed to be protected from the snooping,
spoofing, splicing, replay, and disturbance attacks:

• Encryption – snooping and spoofing protection

• Hashing – spoofing, splicing, replay (counters must be used), and disturbance protection
• Oblivious Access – snooping protection

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 44
Multiprocessor and Many-core Secure Processors

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 45
Multiprocessor Architectures

Symmetric Multi Processing (SMP) and Distributed Share Memory (DSM) also referred to as
Non-Uniform Memory Access (NUMA) offer two ways of connecting many CPUs together.

Other components Individual processors are still

on the same system trusted
are untrusted

SMP DSM / NUMA

Emoji Image:
https://www.emojione.com/emoji/1f479
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 46
SMP Protections

Encrypt traffic on the bus between processors

• Each source-destination pair can share a hard-coded key
• Or use distribute keys using public key infrastructure (within a computer)

Use MACs for integrity of messages

• Again, each source-destination pair can share a key

Use Merkle trees for memory protection

• Can snoop on the shared memory bus to update the tree root node
as other processors are doing memory accesses
• Or per-processor tree

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 47
DSM / NUMA Protections

Encrypt traffic on the bus between processors

• Again need a shared key

Use MACs for integrity of messages

• Again, each source-destination pair can share a key

Use Merkle trees for memory protection

• No-longer can snoop on the traffic (DSM is point to point usually)

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 48
Many-core Trust Boundary

Trusted processor chip boundary is reduced in most research focusing on many-core security

Processor Chip
The processor core is still
trusted

Other cores, interconnect,

or routing elements are
untrusted (malicious IP
core)

Probing of internal
interconnect still
assumed out-of- Memory I/O Devices
scope. Emoji Image:
https://www.emojione.com/emoji/1f479
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 49
Architecture and Hardware Security Intersection

With many-core chips, the threats architects worry about start to overlap
with hardware security researchers’ work

• Untrusted 3rd party intellectual property (IP) cores

• Malicious foundry
• Untrusted supply chain

Architecture solutions (add encryption, add hashing, etc.) complement

defenses developed by hardware security experts (split manufacturing, etc.).

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 50
Protected Inter-processor Communication

In addition to the existing assumption about protected memory communication,

designs with multiple processors or cores assume the inter-processor communication will be
protected:

• Confidentiality
• Integrity
• Communication pattern protection

Emoji Image:
https://www.emojione.com/emoji/1f479
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 51
Performance Challenges

Interconnects between processors are very fast:

• E.g. HyperTransport specifies speeds in excess of 50 GB/s

• AES block size is 128 bits
• Encryption would need 3 billion (giga) AES block encryptions or decryptions per second

• Tricks such as counter mode encryption can help

• Only XOR data with a pad
• But need to have or predict counters and generate the pads in time

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 52
Designing Secure Processors

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 53
Principles of Secure Processor Architecture Design

Four principles for secure processor architecture design based on existing designs and also on
ideas about what ideal design should look like are: • Architectural state
• Micro-architectural state
• Due to spatial or temporal
1. Protect Off-chip Communication and Memory sharing of hardware

2. Isolate Processor State among TEE Execution and other Software

3. Allow TCB Introspection
4. Authenticate and Continuously Monitor TEE and TCB

Additional design suggestions:

• Avoid code bloat
• Minimize TCB
• Ensure hardware security (Trojan prevention, supply chain issues, etc.)
• Use formal verification
ACACES Course on Processor Architecture Security
© Jakub Szefer 2019 54
Protect Off-chip Communication and Memory

Off-chip components and communication are untrusted, need protection with encryption, hashing,
access pattern protection.

Open research challenges: Processor Chip

• Performance Core Core

…
$ $ $ Reg. FU State
• Key distribution
Cache
SecE Uncore
Enc

E.g. encryption defends Memory I/O Devices

Cold boot style attacks on
main memory.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 55
Isolate Processor State among TEE Execution

When switching among protected software and other software or other protected software, need to
flush the state, or save and restore it, to prevent one software influencing another.

Open research challenges: Processor Chip

• Performance Core Core

…
$ $ $ Reg. FU State
• Finding all the state to flush or clean
• Isolate state during concurrent execution Cache

• ISA interface to allow state flushing SecE Uncore

E.g. flushing state helps Memory I/O Devices

defend Spectre and
Meltdown type attacks.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 56
Allow TCB Introspection

Need to ensure correct execution of TCB, through open access to TCB design, monitoring,
fingerprinting, and authentication.

Processor Chip
Open research challenges: Core Core
…
• ISA interface to introspect TCB $ $ $ Reg. FU State
• Area, energy, performance costs Cache
due extra features for introspection
SecE Uncore
• Leaking information about
TCB or TEE

E.g. open TCB design can Memory I/O Devices

minimize attacks on ME or
PSP security engines

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 57
Authenticate and Continuously Monitor TEE and TCB

Monitoring of software running inside TEE, e.g. TSMs or Enclaves, gives assurances about the state
of the protected software.
Likewise monitoring TCB ensures protections are still in place.
App
App
App
TSM
Open research challenges:
• Interface design for monitoring Guest
OS
E.g. continuous monitoring
• Leaking information about TEE
of a TEE can help prevent
HV
TOC-TOU attacks.
SMM
SecE

Hardware

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 58
Pitfalls and Fallacies
E.g. recent attacks on industry processors.
• Pitfall: Security by Obscurity
Most actually realized architectures use a security
processor (e.g. ME or PSP).
• Fallacy: Hardware Is Immutable
E.g. original SGX did not claim side channel
• Pitfall: Wrong Threat Model protection, but researchers attacked it.
Most designs are one-size-fits all solutions.
• Pitfall: Fixed Threat Model
E.g. today’s devices will be in the field for many years,
• Pitfall: Use of Outdated or Custom Crypto but do not use post-quantum crypto.
Most architectures underestimate side channels.
• Pitfall: Not Addressing Side Channels
Performance-, area-, or energy-only focused designs
• Pitfall: Requiring Zero-Overhead Security ignore security.
E.g. rather than partition a problem, large code pieces
• Pitfall: Code Bloat are ran instead TEEs; also TCB gets bigger and
bigger leading to bugs.
• Pitfall: Incorrect Abstraction
Abstraction (e.g. ISA assumptions) does not match
how device or hardware really behaves.

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 59
Pitfalls and Fallacies
Defending only speculative attacks does not ensure
• Pitfall: Focus Only on Speculative Attacks classical attacks are also protected

• …

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 60
Challenges in Secure Processor Design

A number of challenges remain in research on secure processor designs:

Verific Perfor
ation mance
Side
TOC-
Chann
els Fast TOU
Crypto
&
State Hash Attack
Flushi Recov
ng Data ery ISA
Reman Interfa
ence ce
Usabili
ty
TCB
Cost
Key
Distrib
ution

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 61
Thank You!

Related reading…

Jakub Szefer, ”Principles of Secure Processor

Architecture Design,” in Synthesis Lectures on
Computer Architecture, Morgan & Claypool
Publishers, October 2018.

https://caslab.csl.yale.edu/books/

ACACES Course on Processor Architecture Security

© Jakub Szefer 2019 62