Computer

Architecture
(Presentation)

PRESENTED BY:
2021-R2019-EE-619(L)
2021-EE-623
2021-EE-608
TOPIC
Difference Between
CPU ,GPU
And TPU plus Tensor cores
and Cuda cores
o Basic concepts

o Performance Difference

o Choice between CPU,GPU

And TPU

o Companies manufacture
CPU(Central Processing
Unit)
CPU is the brain of computer
It handles all the essential
tasks, from running your
operating system to opening
apps
Think of CPU as the
multitasking wizard that keeps
everything in check
GPU (Graphics
Processing Unit)

 GPUs specialize in handling

graphics-related tasks
They make those stunning
visuals in games and videos
possible
They’re not just for gaming
though; they also lend a hand
in tasks like video editing and
rendering
TPU(Tensor Processing Unit)

TPUs are Google’s special

sauce designed for machine
learning tasks
They excel at crunching
numbers lightning-fast, making
them perfect for AI-related
tasks like image and speech
recognition
They’re like the brain boosters
for AI applications
 Performance Differences

CPU GPU TPU

 Great for general-  King of graphical  AI’s best friend,

purpose tasks but tasks but may not lightning-fast for
might struggle with be as efficient for machine learning
heavy graphics or AI general computing. tasks.
work.
 GPU can handle tens  TPU can handle
 CPU can handle tens of thousands of upto 128000
of operation per operation per cycle operations per cycle
cycle
Which to choose
for your
personal
device?
Answer is simple
its depend on
specific needs of
the user.
 CHOICE BETWEEN CPU, GPU AND TPU

CPU GPU TPU

 If you’re a multitasker  Gamers and content  If AI or machine
who does a bit of creators should lean learning is your
everything, a towards beefier GPUs thing, consider
powerful CPU for that extra oomph devices integrated
combined with a in visuals with TPUs.
decent GPU should
suffice. • Example Device: • Example Device:
ASUS ROG Zephyrus Google Coral Dev
• Example Device: Dell G14 Board
XPS 15
• CPU: AMD Ryzen 9 • Integrated TPU:
• CPU: Intel Core i7 or processor Google Edge TPU
i9 processor
• GPU: NVIDIA GeForce
• GPU: NVIDIA GeForce RTX 30 series (e.g.,
GTX or RTX series RTX 3060 or higher)
Which Companies
make CPU,GPU
And TPU?
 COMPANIES MAKE CPU, GPU AND TPU

CPU GPU TPU

 Intel, AMD, Qualcomm,  NVIDIA, AMD, Broadcom  Google, Coral (owned by
NVIDIA, IBM, Samsung, Limited, Imagination Google), HAILO
Apple, Hewlett-Packard, Technologies (PowerVR)
VIA, Atmel, etc.
Tensor Cores
Tensor cores are specialized hardware
components designed to accelerate tensor
operations, which are commonly used in deep
learning and other machine learning workloads.
They are specifically designed to perform
tensor matrix multiplications and other tensor
operations with high throughput and efficiency.
Tensor cores operate on lower precision formats
like FP16 (16-bit floating-point)
Tensor cores are predominantly found in
NVIDIA's high-end GPUs aimed at deep learning
and AI applications, such as the Tesla V100,
A100, and GeForce RTX 30 series.
Tensor cores implementation in processors

CPU GPU TPU

 CPUs typically do not have  In GPUs, tensor cores are a  TPUs are specifically
dedicated tensor cores like key feature in accelerating designed by Google for
GPUs or TPUs. Instead, tensor deep learning workloads. accelerating machine
operations are handled by learning workloads in its
the CPU's general-purpose data centers, particularly
cores.  GPUs with tensor cores for tasks like neural
can perform these network inference and
 To execute tensor operations, operations with higher training.
the CPU utilizes its SIMD throughput and energy
(Single Instruction, Multiple efficiency compared to  TPUs incorporate a large
Data) capabilities and vector traditional GPU cores. number of tensor
processing units
processing units, which are
highly optimized for
performing tensor
operations at a large scale.
CUDA Cores

CUDA cores are the building blocks of NVIDIA

GPUs.
CUDA cores focus on more traditional
computational tasks across various industries
like gaming, scientific research, and video
editing.
CUDA cores are found in NVIDIA's consumer
GPUs (GeForce series), workstation GPUs
(Quadro series), and data center GPUs (Tesla
series)
 Cuda Cores implementation in processors

CPU GPU TPU

 CPUs do not have CUDA  CUDA Cores are the primary  TPUs do not have CUDA
cores, as CUDA is a parallel processing units in NVIDIA cores, as CUDA is a
computing platform GPUs, responsible for executing proprietary parallel
developed by NVIDIA general-purpose computations computing platform
specifically for GPUs and graphics rendering tasks. developed by NVIDIA for
GPUs.
 However, CPUs can execute  They are programmable cores
CUDA-accelerated capable of executing a wide
applications by offloading range of parallel computing
computations to the GPU. tasks, including matrix
operations, image processing,
and scientific simulations
 WHAT IS
PERFORMANCE
DIFFERENCE
.BETWEEN THESE
2 CORES ?
 LET,S SEE SOME DIFFERENCES

TENSOR CORES CUDA CORES

 Takes the hit in accuracy but deliver superior  CUDA cores deliver very high accuracy, but at the
accelerated computational speed cost of compute speed.

 Tensar cores can perform multiple operations in  CUDA cores perform a single operation in one
one clock cycle clock cycle

 Cloud services such as GPU- as-a-Service will  Accelerated rendering of 3D- graphics in Cloud-
prefer Tensor cores over CUDA cores. based games can leverage CUDA cores

 Tensor cores have significantly high computational  CUDA cores are slower vis-a- vis Tensor cores.
power and are, therefore, a perfect choice for Data Thus, Data scientists and ML Engineers do not
science and ML development and algorithm prefer them for extensive ML training.
training.
Thank you