AI for Embedded

Systems

Dr.D.Vaithiyanathan
Assistant Professor
Department of Electronics and Communication Engineering

National Institute of Technology Delhi

Overview

• Recent Trends in VLSI Design

• Challenges in Embedded Systems

• Review of Artificial Intelligence

• AI based Real-Time Scheduling for

Embedded Systems
 HISTORY OF IC DESIGN
• One transistor, three resistors, one capacitor.

• Sixty years back, Jack Kilby assembled those

components together on one semiconductor. The
world’s first integrated circuit.

• Jack did more than invent the integrated circuit

that day.

• Jack Kilby invented the future.

• He received the noble Prize for this invention in

October 2000.
 Moore’s Law
• In 1969, Gorden Moore stated that,
the number of transistors per chip
will double every 18 months !!!

• And it is happening !!!!!!!

• Now Moore's law has become self sustaining.

• Infact our world follows Moore’s Law.

Xeon Haswell E-5, launched in 2014, sports over 5 billion
 Recent Technology Nodes
The current lowest technology node in production is 7nm and one under
development is 5nm.

There is also a test chip produced in 3nm.

We don't know how far industry will be able to follow Moore's law.

It is becoming more and more commercially unviable proposition to fund

this development.

In fact, it costs $271 million to design a 7nm system-on-a-chip, which is

about nine times the cost to design a 28nm device, according to
Gartner.

“Not that many people can afford to (design chips at 10nm and 7nm)
unless they have a high-volume runner and can see a return-on-
investment,” said Samuel Wang, an analyst with Gartner
 Recent Technology Nodes
In fact foundries are coming up with intermediate nodes like 8nm which is
shrieked version of 10nm.

It may not cost as high as 7nm but still give some area and power
advantage.

Other way to work around this is stacked die or 3D IC which allow you to
get more transistors in same area.

A monolithic 3D ICs rather than a conventional planar implementation, as

this can provide 30% power savings, 40% performance boost, and cut the
cost by 5-10% without changing over to a new node.

There is good amount of research around this but I don't see that
commercially popular yet, reason mostly because of thermal issues and
cost.
 SMALL DIMENSIONS EFFECTS
Difficulty of MOSFET Devices in nano scale regime
Increased Leakage current
Large Sub-threshold swing
Low Switching Speed
Short Channel Effects
Low reliability

Solution
FinFET


Tunnel Field Effect Transistor



Junction-less Transistor (JLT)



Carbon Nanotube Transistor (CNTFET)



Nano-wire Field Effect Transistor


NEW INNOVATIVE DEVICE STRUCTURES

For conventional MOS structure, as the channel length shrinks, the gate
does not have full control over the channel which is not desirable.

One of its effects is to cause more sub-threshold leakage from drain to

source, which is not good from power consumption point of view.

In conventional MOS, the gate cannot control leakage path which is far
removed from it. This can be improved using various MOS structures
which allow the scaling of a transistor beyond conventional MOS scaling
limit.

The two new MOS structures, Silicon on Insulator (SOI) and FinFET
are introduced. The main objective of both the structures is to
maximize gate-to-channel capacitance and minimize drain-to-channel
capacitance.
FINFET IN MICROELECTRONICS INDUSTRY

Intel introduced Trigate FETs at the 22 nm node in the Ivy-Bridge

processor in 2012.

Other foundries that are offering FinFET technology are TSMC, Global
Foundry, and Samsung.

In 2014, TSMC announced that it has produced its first fully functional
ARM-based networking processor with 16nm FinFET technology.

STMicroelectronics released its first Fully Depleted Silicon on Insulator

(FD-SOI) chips for mobile processor at 28nm in 2012.

Foundries that are offering FD-SOI technology are IBM, Global Foundry,
and Samsung.

Some of the products using SOI technology are AMD's processor,

PowerPC microprocessor and Sony's PlayStation.
 WHAT NEXT?

Both FinFET and SOI structure have better gate control and lower
threshold voltage with less leakage.

But, when we move to lower technology node say below 10nm

node, the issue of leakage starts again.

This leads to many other issues like threshold flattening, increase

in power density, and thermal dissipation.

FinFET structure is less efficient in terms of heat dissipations, as

heat can easily be accumulated on the fins.

These concerns can lead to a new class of design rule - Design

for Thermal, unlike other design rule like Design for
Manufacturability.
Recent Devices in Microelectronics Industry

Increasing resistivity of copper with scaling and rising demands on current

density requirements are driving the need to identify new wiring solutions for deep
nanometer scale VLSI technologies.

Metallic carbon nanotubes (CNTs) are promising candidates that can potentially
address the challenges faced by copper and thereby extend the lifetime of
electrical interconnects.

Carbon Nanotube (CNT) FET, Gate-All-Around Nanowire FET or FinFETs with

compound semiconductors may prove as promising solutions in future technology
nodes.
Recent Development in Microelectronics Industry

In 2018 the lowest technology node in production is 7nm and one

under development is 5nm.

There is also a test chip produced in 3nm.

Samsung, for one, plans to ship its 10nm finFET technology by

year’s end. The company will focus on 10nm for now, although it
also is working on 7nm in R&D.

In contrast, TSMC will move into 10nm production in early 2017,

with 7nm slated to ship in 2018. TSMC sees 10nm as a shorter
node and is emphasizing 7nm.

Meanwhile, Intel will move into 10nm production by mid-2017, with

7nm slated for 2018 or 2019, sources said.

In addition, Global Foundries will also be in the mix at 7nm.

Economic Impact & Industry Trends
Over the years, we have seen a wide range of advancements in semiconductor
design services.

The Semiconductor Industry Association (SIA) announced that the global

semiconductor industry posted sales of $468.8 billion in 2018 – the industry’s
highest-ever annual total and an increase of 13.7 percent over the 2017 sales.

As the demand for semiconductor technology node design services continues to

increase and the industry witnesses a broader range of new technology innovations,
we can clearly see a move toward lower geometries (7nm, 10nm, 12nm, 16nm,
etc.).

The key drivers behind this trend are benefits in terms of the power, area, plus
various other features that become possible with lower geometries.

The proliferation of lower geometry design has fuelled business

in a number of areas, especially in the sectors of mobility,
communication, IoT, cloud, AI for hardware platforms (
ASIC, FPGA, boards).
Embedded Systems
Overview
Outline
What is an embedded system

Characteristics and Classification of Embedded

Systems
Systems-on-a-Chip

Distributed Systems

Internet-of-Things

Embedded Systems Design Challenges

Real-Time Embedded Systems

Microprocessors for Embedded Systems
Computing systems are everywhere

Most of us think of “desktop” computers

PC’s
Laptops
Mainframes
Servers

But there’s another type of computing system

Far more common...
 Embedded Systems Overview
Embedded Computing Systems
 Computing systems embedded within electronic
devices Computers are in here...

 Hard to define. Nearly any computing system and here...

other than a desktop computer
and even here...
 Billions of units produced yearly, versus
millions of desktop units

 Perhaps 50 per household and per automobile

 A lot more programming is done for embedded

systems than desktop computers or servers Lots more of these,
though they cost a lot
less each.
 A “short list” of Embedded Systems
Anti-lock brakes Modems
Auto-focus cameras MPEG decoders
Automatic teller machines Network cards
Automatic toll systems Network switches/routers
Automatic transmission On-board navigation
Avionic systems Pagers
Battery chargers Photocopiers
Camcorders Point-of-sale systems
Cell phones Portable video games
Cell-phone base stations Printers
Cordless phones Satellite phones
Cruise control Scanners
Curbside check-in systems Smart ovens/dishwashers
Digital cameras Speech recognizers
Disk drives Stereo systems
Electronic card readers Teleconferencing systems
Televisions
Electronic instruments
Temperature controllers
Electronic toys/games
Theft tracking systems
Factory control
TV set-top boxes
Fax machines
VCR’s, DVD players
Fingerprint identifiers
Video game consoles
Home security systems
Video phones
Life-support systems Washers and dryers
Medical testing systems

And the list goes on and on

Definitions
Broad definition:
Any computer system that is not a general-purpose computer
 That would include robots, and all portable devices

Narrow definition:
A computer system (software and hardware) that interacts
with its physical environment, mainly without human
intervention
 That would exclude printers, modems, portable devices such as dvd
and mp3 players, etc.
Some common characteristics of embedded
systems
Single-functioned
Executes a single program, repeatedly

Tightly-constrained
Low cost, low power, small, fast, etc.

Reactive and real-time

Continually reacts to changes in the system’s environment
Must compute certain results in real-time without delay
Considerations in embedded system design

An embedded system receives input from its environment

through sensors, processes this input and acts upon its
environment through actuators

Besides the usual software and hardware design issues the

embedded system designer must consider the properties of
the sensors and actuators and the environment itself

The ultimate test of an embedded systems are the laws of

physics
Processing Elements used in Embedded Systems
 Microcontroller:
 Cheap
 Optimized for control applications
 Low processing power
 Low power consumption

 General-Purpose Processor:
 More expensive
 Medium processing power
 Suitable but not optimized for any application
 High power consumption

 Digital Signal Processor:

 Optimized for DSP applications (high-end audio, video and image
processing)

 FPGA:
 Good processing power
 Longer development time
 Medium cost
 Low power consumption

 ASIC:
 High processing power
 Very low power consumption
 Expensive at low volume
 Optimized for specific application (hardware accelerators)
 Long development time
Design challenge – optimizing design metrics
Obvious design goal:
Construct an implementation with desired functionality

Key design challenge:

Simultaneously optimize numerous design metrics

Design metric
 A measurable feature of a system’s implementation
Optimizing design metrics is a key challenge
Design challenge – optimizing design metrics
Common metrics
Unit cost: the monetary cost of manufacturing each copy of the system,
excluding NRE cost

NRE cost (Non-Recurring Engineering cost): The one-time

monetary cost of designing the system

Size: the physical space required by the system

Performance: the execution time or throughput of the system

Power: the amount of power consumed by the system

Flexibility: the ability to change the functionality of the system without

incurring heavy NRE cost
Design challenge – optimizing design metrics
Common metrics (continued)
Time-to-prototype: the time needed to build a
working version of the system

Time-to-market: the time required to develop a system

to the point that it can be released and sold to customers

Maintainability: the ability to modify the system after

its initial release

Correctness, safety, many more

Design metric competition -- improving one may worsen others

 Expertise with both software

Power
and hardware is needed to
optimize design metrics
Performance Size

 Not just a hardware or

software expert, as is common
NRE cost

A designer must be
Digital camera chip comfortable with various
CCD

A2D
CCD preprocessor Pixel coprocessor D2A technologies in order to
lens choose the best for a given
JPEG codec Microcontroller Multiplier/Accum application and constraints
DMA controller Display ctrl

Memory controller ISA bus interface UART LCD ctrl

 Time-to-market: a demanding design
metric

 Time required to develop a

product to the point it can be
sold to customers
Revenues ($)

 Market window
Period during which the
product would have highest
sales
Time (months)
 Average time-to-market
constraint is about 8 months
 Delays can be costly
 Losses due to delayed market entry
 Simplified revenue model
Peak revenue
 Product life = 2W, peak at W
Revenues ($)

Peak revenue from  Time of market entry defines a

delayed entry
On-time triangle, representing market
Market Market penetration
rise fall  Triangle area equals revenue
Delayed

 Loss
D W 2W  The difference between the on-
On-time Delayed Time
entry entry
time and delayed triangle areas
 Losses due to delayed market entry (cont.)
 Area = 1/2 * base * height
Peak revenue  On-time = 1/2 * 2W * W
Revenues ($)

Peak revenue from

 Delayed = 1/2 * (W-D+W)*(W-D)
delayed entry
 Percentage revenue loss =
On-time
Market Market (D(3W-D)/2W2)*100%
rise fall  Try some examples
Delayed

D W 2W – Lifetime 2W=52 wks, delay D=4 wks

On-time Delayed Time – (4*(3*26 –4)/2*26^2) = 22%
entry entry – Lifetime 2W=52 wks, delay D=10 wks
– (10*(3*26 –10)/2*26^2) = 50%
– Delays are costly!
Classification of Embedded Systems
Centralized vs distributed
Real-time vs non real-time

Hard deadline
 Failsafe
 Fail-operational

Soft deadline
Firm deadline

Battery powered vs mains powered

System-on-Chip vs discrete element
 Real-time (reactive) systems
Systems that are bound by a real-time constraint
(“deadline”) in their operation
If the deadline is not met it is usually considered a
system failure, even if the output is eventually correct
Deadlines are usually relative to an event
Hard deadlines: Anti-lock brakes,
Soft deadlines: Digital video
Not the same as high-performance systems, because
often running faster than real-time requirement is not
necessary or desired
Real-time constraints
te + to < tc
te: execution time
to: overhead time
tc: constraint time
What is real-time? Is there any other
kind?

A real-time computer system is a computer system

where the correctness of the system behavior depends
not only on the logical results of the computations, but
also on the physical time when these results are
produced.

By system behavior we mean the sequence of outputs in

time of a system.
Real-time means reactive
 A real-time computer system must react to stimuli from its
environment
 The instant when a result must be produced is called a deadline.

 If a result has utility even after the deadline has passed, the
deadline is classified as soft, otherwise it is firm.
 If severe consequences could result if a firm deadline is missed,
the deadline is called hard.
 Example: Consider a traffic signal at a road before a railway
crossing. If the traffic signal does not change to red before the
train arrives, an accident could result.
Fail-Safe hard-deadline RT
systems
 If a safe state can be identified and quickly reached upon the
occurrence of a failure, then we call the system fail-safe.

 Failsafeness is a characteristic of the controlled object, not the

computer system.
 In case a failure is detected in a railway signaling system, it is possible to
set all signals to red and thus stop all the trains in order to bring the
system to a safe state.

 In failsafe applications the computer system must have a high

error-detection coverage.

 Often a watchdog, is required to monitor the operation of the

computer system and put it in safe state.
Fail-Operational hard-deadline RT
systems

In fail-operational applications, there is no safe state

a flight control system aboard an airplane.

The computer system must remain operational and

provide a minimal level of service even in the case of a
failure to avoid a catastrophe
Typical Embedded System
Components
Sensors: Allow the system to “read” the environment

Processing Elements: Control of the embedded system

Actuators: Allow the system to act on its environment

Network Connection: Local or internet, allowing

exchange of information
Example
Assuming a real-time system that processes samples at
a f= 10 MHz sampling rate, and a to= 20 ns, select the
most appropriate implementation among the following:
A processor running at 500 MHz, requiring 100 cycles at a
cost of 50$
An FPGA running at 200 MHz, requiring 10 cycles at a cost of
60$
A DSP running at 500 MHz, requiring 20 cycles at a cost of
100$
An ASIC running at 2 GHz, requiring 20 cycles at a cost of
500$
Are the following embedded systems? If yes,
classify them. Which characteristics of an
embedded system do they have or do not have?
Printer
DVD player
Mobile phone
Tablet
Netbook/laptop
Automobiles
 Embedded System
Medical
Automotive
 Embedded Systems are
everywhere
 Ubiquitous, invisible
Communications
Military  Hidden (computer inside)
 Dedicated purpose

 Microprocessor
 Intel: 4004, ..8080,.. x86
Comsumer Industrial
 Freescale: 6800, .. 9S12,.. PowerPC
 ARM, DEC, SPARC, MIPS,
PowerPC, Natl. Semi.,…

Embedded system  Microcontroller

Microcontroller LM3S or TM4C Electrical,  Processor+Memory+
mechanical,
Processor chemical, I/O Ports (Interfaces)
I/O Ports or
RAM optical
devices
ROM DAC Analog
Bus ADC signals

1-54
An embedded system example – Digital camera
Digital camera chip
CCD

CCD preprocessor Pixel coprocessor D2A

A2D

lens

JPEG codec Microcontroller Multiplier/Accum

DMA controller Display ctrl

Memory controller ISA bus interface UART LCD ctrl

• Single-functioned -- always a digital camera

• Tightly-constrained -- Low cost, low power, small, fast
• Reactive and real-time -- only to a small extent
Embedded Software Development Requires as Much/More Design Effort Than
Hardware
A System-on-a-Chip: Example

Courtesy: Philips
Design at a crossroad - System-on-a-Chip
• Embedded applications
where cost, performance,
Multi- and energy are the real
500 k Gates FPGA

Analog
Spectral issues!
RAM + 1 Gbit DRAM
Imager • DSP and control intensive
Preprocessing • Mixed-mode
• Combines programmable
C and application-specific
64 SIMD Processor system modules
Array + SRAM +2 Gbit • Software plays crucial role
DRAM
Image Conditioning Recog-
nition
100 GOPS
The Future of Embedded Systems
In the past an embedded system was more or less isolated

In the past decade Wireless Sensor Networks have

changed that

Today embedded systems + internet connection = Internet

of Things

Near future: Embedded systems + internet + mobile

devices + cloud computing + artificial intelligence + ? =
smart environment
IoT forecasts
Global Internet of Things (IoT) market reached USD
598.2 Billion in 2015

is expected to reach USD 724.2 Billion by 2023

the market is projected to register a CAGR of 13.2%

The number of connected IoT (Internet of Things)

devices, sensors and actuators will reach over 46 billion
in 2021
 IoT
Embedded system with internet connection

Not quite as simple as it sounds

Increased need for security

Safety issues

Direct machine to machine (M2M) communication

Case Studies of Distributed
Embedded Systems
Outline
Case study 1: RFID

Case study 2: Wireless sensor networks

Case study 3: Internet of things

RFID Tags
Developed to automate the process of object
identification
electronic tags (called RFID tags) can be read from a
small distance by an RFID reader
An RFID reader does not require a direct line-of-sight to
the RFID tag.
The RFID tag stores the unique Electronic Product
Code (EPC) of the attached object.
RFID Tag dimensions
Since an RFID tag has to be attached
to every object, the cost of an RFID
tag is a major issue.

RFID tags come in various shapes and

sizes and continue to decrease in size

RFID tags are implantable and

implants have been approved in
humans as well as animals.
RFID Reader
The RFID reader can act as a gateway to the Internet and
transmit the object identity, together with the read-time
and the object location (i.e., the location of the reader) to
a remote computer system that manages a large database.

It is thus possible to track objects in real-time

Applications: toll gates, hospitals and large

organizations, public transportation systems, tracking of
animals, libraries
Electronic Product Code
 A typical EPC has a length of 96 bits and contains the following fields:
 Header (8 bits): defines the type and the length of all subsequent fields.
 EPC Manager (28 bits): specifies the entity (most often the manufacturer) that
assigns the object class and serial number in the remaining two fields.
 Object Class (24 bits): specifies a class of objects (similar to the optical bar code).
 Object Identification Number (36 bits): contains the serial number within the
object class.

 The EPC is unique product identification, but does not reveal anything
about the properties of the product.

 Two things that have the same properties, but are designed by two different
manufacturers, will have completely different EPCs.
Passive RFID tags
 Passive RFID Tags. No power supply. They get the power needed
for their operation from energy harvested out of the electric field
that is beamed on them by the RFID reader. The energy required
to operate a passive tag of the latest generation is below 30 mW
and the cost of such a tag is below 5 ¢.

 Due to the low level of the available power and the cost pressure
on the production of RFID tags, the communication protocols of
passive RFID tags do not conform to the standard Internet
protocols. Specially designed communication protocols between
the RFID tag and the RFID reader that consider the constraints of
passive RFID tags have been standardized by the ISO (e.g., ISO
18000-6C also known as the EPC global Gen 2) and are
supported by a number of manufacturers.
Active RFID tags
 Active RFID Tags
 have their own on-board power supply
 The lifetime of an active tag is limited by the lifetime of the battery
 typically in the order of a year.

 Active tags can transmit and receive over a longer distance

 typically in the order of hundreds of meters,
 can have sensors to monitor their environment
 sometimes support standard Internet communication protocols.

 An active RFID tag resembles a small embedded system

 More expensive
WSN
A set of sensor nodes that
each contains
a sensor

a microcontroller

a wireless communication
controller
WSN node
A sensor node can acquire a variety of physical,
chemical, or biological signals to measure properties of
its environment.
WSN node constraints
Sensor nodes are resource constrained.

They are powered either by a small battery or by energy

harvested from its environment
have limited computational power, a small memory, and
constrained communication capabilities.
WSN deployment and operation
a number (from few tens to millions) of sensor nodes
are deployed, either systematically or randomly, in a
sensor field to form an adhoc self-organizing
network

The WSN collects data about the targeted

phenomenon and transmits the data via an adhoc
multi-hop communication channel to one or more
base stations that can be connected to the Internet.
WSN function
 Phase 1: detect neighbors and establish communication
 Phase 2: learn about
 the arrangement in which the nodes are connected to each other,
 the topology of nodes
 build up ad-hoc multi-hop communication channels to a base station

 In case of the failure of an active node, it must reconfigure the network

 Applications:
 remote environment monitoring,
 surveillance,
 medical applications,
 ambient intelligence,
 Military

 The utility of a wireless sensor network is in the collective emergent intelligence

of all active sensor nodes, not the contribution of any particular node.
 Primary concern for WSN: energy
 A WSN is operational as long as a minimum number of nodes is
active and the connectivity of the active nodes to one of the base
stations is maintained.
 In battery-powered sensor networks, the lifetime of the network
depends on the energy capacity of the batteries and the power-
consumption of a node.
 When a sensor node has depleted its energy supply, it will cease to
function and cannot forward messages to its neighbors any more.
 The design of the nodes, the communication protocols, and the
design of the system and application software for sensor networks
are primarily determined by this quest for energy efficiency and
low cost.
WSN + RFID = the future?
RFID infrastructure for the interconnection of
autonomous low-cost RFID-based sensor nodes has been
proposed
nodes operate without a battery and harvest the energy
either from the environment or the electromagnetic
radiation emitted by the RFID reader.
Potential for long-lasting, low-cost ubiquitous sensor
nodes that may revolutionize many embedded
applications.
IoT component: Smart object
A smart object is a cyber-physical system or an
embedded system, consisting of a thing (the physical
entity) and a component (the computer) that processes
the sensor data and supports a wireless
communication link to the Internet.

Example: smart refrigerator keeps track of the

availability and expiry date of food items and places
orders
IoT issues
 The novelty of the IoT is not in the functional capability of a
smart object

 Novelty exists in the expected size of billions or even trillions of

smart objects that bring about novel technical and societal issues
that are related to size.

 issues are:
 authentic identification of a smart object,
 autonomic management and self-organization of networks of smart objects,
 diagnostics and maintenance,
 intrusion of privacy

 Safety issues
 Autonomous mobile robots and self-driving cars
Key technologies for IoT
low-power wireless communication: no need of a
physical connection.

GPS: makes a smart object location- and time-aware

 Smart object categories
Goal: an autonomic smart object that
has access to a domain specific knowledge base
is empowered with reasoning capabilities to orient itself in the
selected application domain.

Based on the capability level of a smart object it can be

activity aware
policy aware
process aware
 Ultimate vision: smart planet
everyday things around us with an identity in cyberspace
capable of acquiring information and intelligence

the world economy and support systems will operate

more smoothly and efficiently
Social and legal issues in IoT
But the life of the average citizen will also be affected by
changing the relation of power between those that have
access to the acquired information and can control the
information and those that do not.

IoT devices can be hacked with significant dangers to

safety and property
IoT drivers
The IoT should extend the interoperability of the
internet to the universe of heterogeneous smart objects.

IoT must establish a uniform access pattern to things in

the physical world.
 Logistics
 The first commercial application of
a forerunner of the IoT, the RFID is
in the area of logistics
 There are many quantitative
advantages in using RFID
technology in supply-chain
management:
 the movement of goods can be tracked in
real-time,
 shelf space can be managed more
effectively
 inventory control is improved
 the amount of human involvement in the
supply chain management is reduced
considerably.
Energy savings
 Already today, embedded systems contribute to energy savings in many
different sectors of our economy and our life.
 increased fuel efficiency of automotive engines,
 improved energy-efficiency of household appliances,
 reduced loss in energy conversion

 The future: of IoT devices opens many new opportunities for energy savings:
 Smart buildings: individual climate and lighting control in residential buildings
 Smart grids: reduced energy loss in transmission by the installation of smart grids,
 Smart meters: better coordination of energy supply and energy demand

 Other energy savings:

 Physical meetings replaced by virtual meetings
 delivery of information goods such as the daily paper, music, and videos by the
Internet
Security and safety
Automated IoT based access control systems to
buildings and homes

IoT-based surveillance of public places

Smart passports and IoT based identifications (e.g., a

smart key to access a hotel room or a smart ski lift
ticket)
Car-to-car and car-to-infrastructure communication
will alert the driver of dangerous traffic scenarios
Industrial
 computerized observation and monitoring of industrial
equipment
 reduces maintenance cost
 improves the safety in the plant

 A smart object can monitor its own operation and call for
preventive or spontaneous maintenance in case a part wears
out or a physical fault is diagnosed

 Automated fault-diagnosis and simple maintenance are

absolutely essential prerequisites for the wide deployment of
the IoT technology in the domain of ambient intelligence.
Medical
 The wide deployment of IoT technology in the medical domain is
anticipated.
 Health monitoring (heart rate, blood pressure, etc.)
 precise control of drug delivery by a smart implant

 Body area networks that are part of the clothing can monitor the
behavior of impaired persons and send out alarm messages if an
emergency is developing.

 Smart labels on drugs can help a patient to take the right medication at
the right time and enforce drug compliance.

 Example: A heart pacemaker can transmit important data via a

Bluetooth link to a mobile phone that is carried in the shirt pocket. The
mobile phone can analyze the data and call a doctor in case an
emergency develops.
Technical issues: internet integration
Guaranteeing the safety and information security of IoT-based
systems is considered to be a difficult task.

Many smart objects will be protected from general Internet

access by a tight firewall to avoid that an adversary can
acquire control of a smart object.
Naming and identification
 A well-thought-out naming architecture in order to be able to
identify a smart object and to establish an access path to the
object is essential.

 Isolated Objects. The following three different object names

have to be distinguished when we refer to the simple case of an
isolated object:
 Unique object identifier (UID) refers to the physical identity of a specific
object.
 The Electronic Product Code (EPC) of the RFID community is such a
UID.
 Object type name refers to a class of objects that ideally have the same
properties.
 Object role name. In a given use context, an object plays a specific role
that is denoted by the object role name.
Composite object naming
Composite Objects. Whenever a number of objects are
integrated to form a composite object, a new whole, i.e.,
new object is created that has an emerging identity that
goes beyond the identities of the constituent objects.
IoT vs cloud computing
 Smart objects that have access to the Internet can take advantage of
services that are offered by the cloud

 The division of work between a smart object and the cloud will be
determined, to a considerable degree, by privacy and energy
considerations

 If the energy required to execute a task locally is larger than the energy
required to send the task parameters to a server in the cloud, then the
task is a candidate for remote processing.

 However, there are other aspects that influence the decision about work
distribution: autonomy of the smart object, response time, reliability,
and security.
SoC Design
Key topics of system-on-a-chip
Divide-and-conquer view of SoC
Design
Programmable Processor
Rapid development in the Multimedia algorithm
research – No. of approaches to meet the diverse
computing requirement.

Achieving high computational and energy efficiencies.

A predominant approach is to incorporate

programmable core.

Several different algorithms can be executed on the

same hardware.

97 May 5, 2024
Programmable Processor
The functionality of a specific system can be easily
upgraded by a change in software.

This will create a versatile platform that can follow new

generations of applications and standards.

Some popular programmable core implementations use

RISC and/or DSP cores.

Even extend existing programmable cores with

multimedia enhancements.
98 May 5, 2024
 Hardware-Software Co-Design
Many applications can be divided into two portions.
(i)Complex data-dependent and decision-making
procedures.
(ii)Computationally intensive and regular tasks.

We can employ fast dedicated hardware modules to

perform regular computation-intensive tasks.

For almost all multimedia SoC designs, there is a

common co-design methodology.

99 May 5, 2024
A common software-hardware co-design methodology

100 May 5, 2024

 Co-design Methodology
1. Software and hardware partitioning.

2. Software and hardware synchronization.

3. Algorithm optimization.

4. Software optimization.

5. Dedicated hardware design.

The methodology is applicable to video decoders, audio

decoders and other future multimedia applications.
101 May 5, 2024
IP Reuse
 In order to deploy successful SoC products in a timely manner, we must build
new SoC’s from circuit blocks that have been designed for previous ones.

 By using existing and high-performance IP, Soc designers not only can save
time and resources, but also can create a mind blowing solutions that users want.

 IP does not just refer to hardwired logic or hardware design.

 Software development starts to be on the critical path in time-to-market for SoC.

 In today’s business environment, semiconductor companies must minimize risk

and shorten time-to-market for their customers.

 IP libraries must include behavioral model description, so that the entire

hardware/software co-design can be simulated and verified at the early design
stage.

102 May 5, 2024

 IP Reuse
 It is be great to be able to quickly assemble new software stacks
(e.g., OS, complier, libraries) from reusable software.
 Most semiconductor companies have mature hardware reuse
methodologies, the majority of them have not yet reused
software components.
 Embedded software design can easily take more resources than
hardware design.

103 May 5, 2024

Cross-Disciplinary Vertical Integration
 In order to vertically integrate hardware, software, algorithms,
applications, and their interfaces, architecting a SoC design
requires cross-disciplinary knowledge.

 Divide at the very beginning of the system design.

 It gives best performance for each individual component and to

get the best overall performance.

 The whole system depends heavily on the initial design of the

architecture.

104 May 5, 2024

 Cross-Disciplinary Vertical Integration
 Partition the hardware and software.

 Combine cost-effective silicon with high-performance software

stacks is the key differentiator of a successful SoC design.

 This requires tight interaction between algorithm/software

design and hardware design.

 Understanding the application becomes critical, including

analyzing the algorithm based on the performance specification.

 Exploiting optimization techniques to reduce the bottlenecks in

memory capacity, data bus bandwidth, power dissipation, and so
forth.
105 May 5, 2024
 Modern System Design Trends
Power Efficiency
1. Transistor power dissipation is not falling at the same rate
as the gate density is increasing.

2. Rather than relying on processor technology, modern

designs start addressing the problem at the circuit level and
the architecture level.

3. Like take an advantage of idle cycles

4. Reduce the system-level power dissipation by shutting off

parts of the system that are not used.
106 May 5, 2024
Modern System Design Trends
Power Efficiency
5. In the past heuristics were often used to predict workloads
and turn modules on and off.

6. For better efficiency, modern designs further utilize the

knowledge from the applications.

7. Another important techniques to reduce power

consumption of clock distribution.

8. Synchronous design are easy to implement, but main

drawback is power consumption for clock distribution.
107 May 5, 2024
Modern System Design Trends
Power Efficiency
9. In recent years, mesochronous designs have been
overcome the clock distribution drawbacks.

10.Mesochronous designs use a single clock frequency, but

different blocks at different phase. Clock edges are not
align.

11.There is a latency penalty for transferring data between

blocks, this kind of designs have an advantages of the low
power consumption for the global clock distribution.

108 May 5, 2024

Modern System Design Trends
Temperature-aware design
1. In recent days power density and cooling costs rising
exponentially. So temperature-aware design has become a
necessity.

2. The temperature variations and hotspots accounts for over

50% of electronic failures.

3. Thermal variations can also lead to significant timing

uncertainty.

4. How to design and place the “hot” and “cool” components

together in the architecture design.
109 May 5, 2024
 Modern System Design Trends
Temperature-aware design
5. Traditional cooling system design, which is based on worst-case
analysis.

6. It will over-provision the real requirement and will not be

economical.

7. Modern designs start incorporating techniques that control or

reduce heat dissipation.

8. In particular, runtime techniques that can regulate operating

temperature when the package’s capacity is exceeded.

9. Emerging 3D stacking technology introduces more freedom and also

places more constraints on future designs. May 5, 2024
110
Modern System Design Trends
Other parameters
1. Power Efficiency
2. Multi-processor SoCs
3. Reconfigurable Logic
4. Design for verification and for Testing

System Design Challenges

1. Scalable or Reusable Architecture
2. IP Integration
3. Network-on-chip
4. Embedded Memory
5. Reliability
111 May 5, 2024
 Power Efficiency
Transistor power dissipation is not falling at the same
rate as the gate density is increasing.

Rather than relying on processor technology, modern

designs start addressing the problem at the circuit and
architecture level.

Some power management techniques are proposed to

take advantages of idle cycles.

System-level power dissipation by shutting off parts of

the system that are not used on only when it requires.
112 May 5, 2024
Power Efficiency
For better efficiency, modern designs further utilize the
knowledge from the applications.
Important technique to reduce power consumption is to
reduce power consumption of clock distribution.
Synchronous design are widely used, easy to implement and
well supported by logic synthesis tools. But it consumes
more power for clock distribution.
Mesochronous designs use a single clock frequency, but
different block may be at different phase.

113 May 5, 2024

Power Efficiency
Power density and cooling costs rising exponentially,
temperature-aware design has become a necessity.
Chip reliability and performance are increasingly impacted
by the operating temperature.
The temperature variations and hotspots account for over
50% of electronic failures.
Thermal variations can also lead to significant timing
uncertainty, prompting wider timing margins, and poorer
performance.

114 May 5, 2024

 Power Efficiency
 Placing the “hot” and “Cool” components together becomes
another important aspect in the architecture design.

 The traditional cooling system design, which is based on worst-

case analysis, will over-provision the real requirement and it will
not be economical.

 Modern designs starts incorporating techniques that control or

reduce heat dissipation, in particular runtime techniques that can
regulate operating temperature, when packaging capacity is
exceeded.

 Emerging 3D stacking technology introduces more freedom and

115 May 5, 2024
also places more constraints on future designs.
Multi-Processor SoCs
 To further increase performance without substantially increasing power
consumption, parallel processing can be used at the instruction level at VLIW
and at data level SIMD.

 Multi-processor SoCs (MPSoCs) start becoming popular, as it is a very power-

efficient and increase the processing capabilities. More and more processing
units in near future.

 Multiprocessors on the same chip do not have to be the same.

 A very powerful means to accelerate multimedia processing is to adapt

programmable processors to specific algorithms via specialized instructions for
frequently occurring and high-complexity operations.

 Emerging 3D stacking technology introduces more freedom and also places more
constraints on future designs.
116 May 5, 2024
Multi-Processor SoCs
 A design can integrate multiple programmable cores,
 Which are individually optimized to a particular characteristic of different
application fields in order to deliver high performances.
 They complement each other with flexibility at reduced system cost.

 These designs are often referred to as heterogeneous MPSoCs.

 Multi-core processors become more prevalent in vendors solutions,

application developers must change the algorithm to explore the
full potential of multi-core processors.

 There will be a essential need for parallel algorithms and

programming. Without a properly parallelized program, future
multi-core processors won’t deliver their best performance.
117 May 5, 2024
Reconfigurable Logic
 To create high-performance, versatile platforms, some architectures
start incorporating logic operations and interconnects that can be
reconfigured during run time.

 Adding reconfigurable logic to the SoC provides flexibility for

changing functionality after fabrication.

 Compared to programmable processors, these architectures offer the

potential to achieve higher performance and power-efficiency with
greater flexibility.

 To boost the impact of reconfigurable SoCs, some research work has

been done to extract the parallelism from the application/algorithms
and map the parallelism into reconfigurable architecture efficiently.
118 May 5, 2024
Reconfigurable Logic
In general, there are two styles reconfigurable computing.

1.Fine-grained reconfigurable logic offers extremely flexibility

at the cost of efficiency in are an power.

2.Coarse-grained reconfigurable architectures, on the other

hand are capable of executing word or sub-word level
operations (e.g. Addition, multiplication, shift) instead of the
bit-level operations.

119 May 5, 2024

 Design for Verification and for Testing
 Traditionally, 70% of time and energy in chip design cycles is spent
on verification.

 Typically, when there is small change in a component, we need to re-

verify timing for the entire chip design.

 One way to avoid that is to create clear boundaries and routing

channels between the components. In this case, changes in one block
do not affect the timing of others.

 Mesochronous clocking is a way to separate different logic blocks

from one another.

 As we integrate a billion transistor onto a single chip, it takes much

more time to test the chip as whole, in order to verify all the state
120
machines and logic blocks in a design. May 5, 2024
Design for Verification and for Testing
 As we are likely to see increasing variability in the behaviour of the
transistors statically and dynamically, build-in self-tests in each
logic block become essential.

 Additionally, if the IPs integrated onto a chip come from more than
once source, IP providers and SoC integrators must work closely
together to define effective test strategies.

 Each IP block must have a wrapper so that it can be isolated from

other parts of the system while it is being tested.

121 May 5, 2024

 System Design Challenges
The new generation of SoC architecture must meet more open
and challenging design issues, as follows:

1. Scalable or Reusable Architecture

2. IP Integration

3. Network-on-Chip

4. Embedded Memory

5. Reliability

122 May 5, 2024

Scalable or Reusable Architecture
We shifted our IC design paradigm from full custom design to
standard cells, even to IP reuse, the next-generation system
design paradigm shift should be the reuse of architecture.

The growing complexity of SoCs, IP reuse may not be

enough.

Reuse must happen at a much higher level that it used to reuse

even at architecture level.

Creating a design that can be efficiently reused requires a

great deal of effort.

123 May 5, 2024

Scalable or Reusable Architecture
As multimedia applications growing in a faster phase, we need
more and more computational capability.

For example, video coding standards have been evolved from

MPEG-2 to H.264.

The picture resolutions have been increased from DVD

(720x480) to HDTV (1280x720 or 1920x1080).

To ensure fast design turnaround time without completely

redesigning the whole system, a scalable architecture is highly
desired.

124 May 5, 2024

Scalable or Reusable Architecture
As multimedia applications growing in a faster phase, we need
more and more computational capability.

For example, video coding standards have been evolved from

MPEG-2 to H.264.

The picture resolutions have been increased from DVD

(720x480) to HDTV (1280x720 or 1920x1080).

To ensure fast design turnaround time without completely

redesigning the whole system, a scalable architecture is highly
desired.

125 May 5, 2024

IP Integration
 Integration requires more than simply placing components spatially together on a
single chip. A few issues, for example are outlined below:

1. How to integrate analog IP safely; in particular, how to deal with noise from the analog
domain to the digital domain or vice versa.

2. How to deal with black-box IP.

3. How to handle its I/O requests in a timely manner without over-provisioning resources for
it.

4. How to migrate IPs from one process technology to the next one as quickly as possible. Is
synthesizable soft IP better than customized hard IP even though customized hard IP may be
more efficient.

5. How to effectively test and verify the whole system when the IPs come from different
sources.

These are the challenges beyond simply placing components together. We need good strategies
126 for the integration of hardware and software IP components. May 5, 2024
 Network-on-Chip
 The performance of next-generation SoCs will be limited by the ability to
efficiently connect the functional blocks together, and to accommodate their
communication requirements.

 As applications require more computational power in the next few years, it is

clear that we need multiple processors, processing elements, and function units.

 Hence, communication becomes the critical path in the system.

 The need for synergy of processing elements and interconnect architectures

becomes critical as we face tighter power budgets in the pursuit of performance
targets.

 First, the delay and energy of communication wires do not scale down linearly as
the transistor size shrinks in the future. The interconnect will consume a
significant amount of chip energy.
127 May 5, 2024
 Network-on-Chip
 Second, because processor and interconnect architecture share a common power
and thermal budget, optimizing the power consumption of each in isolation can
have a significant impact on the other’s power and performance.

 Furthermore, the unreliability of deep submicron on-chip wire communication

stimulates more challenges.

 While many communication architectures make use of buses or crossbars, there

are some recent proposals on network-on-chip (NoC) architectures.

 One approach is to employ a packet-switched interconnect.

 The concept is similar to traditional large-scale wide-are networks, but in this

case, on-chip router-based networks are used.

128 May 5, 2024

Network-on-Chip
 Programmable cores access the network via packet-switched interfaces and have
their packets forwarded to their destinations through a multiple-hop routing path.

 Nevertheless, the on-die communication architectures cannot just mimic the off-
die communication.

 We need different designs, for eg., while compression of contents in saved power
consumption for off-die bus communication, compression may need more power
for on-die and short-distance communicaiton.

 Because of communication localities and the latencies, circuit-switched NoC is

sometimes more attractive than the traditional packet-switched NoC.

 An architect must examine all these factors to determine the best one for the
design.

129 May 5, 2024

 Embedded Memory
 As external memory bandwidth becomes a major bottleneck, more on-die
high-speed memory, such as cache or local buffer, will be deployed.

 Sometimes, embedded memory such as SRAM, DRAM, flash, ROM will

be integrated onto the chip.

 The amount of memory integrated into an ASIC-type SoC design has

increased from 20% in 1999 to 70% to 100% in 2019.

 Challenges arise when trying to balance efficiency and power. SRAM

provides high performance, while flash memory is the best solution in
terms of power consumption.

 The amount and the placement of each kind of memory in the SoC will
greatly affect access efficiency and power.
130 May 5, 2024
Embedded Memory
 Additionally, cache may introduce indeterminate delay, cache coherence, and
memory consistency challenges.

 Therefore, the unpredictable latencies associated with caches must be carefully

accounted for.

 An alternative solution is to use a software-control local buffer instead of cache.

 An example of this is the cell architecture developed by Sony, Toshiba, and IBM.
A software control local buffer reduces the uncertainty when it comes to latency,
but of its use makes the software more complex.

 Furthermore, because digital, mixed-signal, RF, and memory blocks are tightly
integrated, the power and substrate noise may cause sensitive blocks to suffer
from functional failures.

131 May 5, 2024

 Embedded Memory
 Because of the unique characteristics of the memory circuit and layout.,
the power and substrate noise may cause functional failures.

 Making embedded memories noise-tolerant will be vital to a successful

SoC design.

 3D stacking memory is an alternative exciting technology to increase

bandwidth substantially.

 However, despite its promising advantages, the thermal issue of multiple

dies stacking together is of serious concern.

 In short, a high-performance and low-power SoC architecture must

balance the tradeoffs from various memory options.
132 May 5, 2024
Reliability
 Reliability is likely to be a major focus of research as transistor sizes
become smaller than 10nm. We are likely to see increasing variability in
the behaviour of the transistors.

 Smaller feature sizes lead to more failures over time from electrostatic
overstressing and electro-migration. While transistors might fail, the
entire system cannot fault.

 We must explore mechanisms to compensate for this underlying

variability in transistor behaviour.

 To address these challenges, research is being carried out from fabrication

to software.

 Some of the basic fault tolerance principles that we used in the era of
mainframe computers can potentially be applied to today’s designs.
133 May 5, 2024
 Reliability
 For example, we can detect when a circuit fails and shift the work
to another circuit through hardware/firmware based self-
management.

 Yet another possible solution is to use statistical computing

techniques, which use probabilistic models to derive reliable results
from unreliable components.

 Nonetheless, none of these techniques is a clear answer on how to

efficiently address reliability issues.

 The bottom line is that there need to be an architectural solution.

134 May 5, 2024

But how do we integrate all requirements?
Smartphone - Galaxy
S2

Future!!
!
Early mobile phones - Ericsson

SoC

DSP, Microprocessor and Memory

are all integrated into a single SoC!
Moore’s Law More than just transistors?
SoC Architecture
 Hardware
Analog: ADC, DAC, PLL, TxRx, RF…etc. Digital: Processor,
Interface, Accelerator…etc.
Storage: SRAM, DRAM, FLASH, ROM…etc.

 Software: OS, Application

SoC Device
SoC Design Considerations
 Architecture strategy
 Design-for-test strategy
 Validation strategy
 Synthesis and backend strategy
 Integration strategy
Need of SoC
Growth of the Technology
Architecture Strategy


Central processing core


 DSP cores On chip bus

 Easy plug-and-play IPs I/O, peripherals

Platform-based design methodology
Alternative Computing Subsystem
Control-dominated subsystem.

 controls & coordinates system

tasks
 performs reactive tasks (e.g.
user interface)

Data-dominated Subsystem
 regular & predictable

 well-defined DSP kernels

transformational tasks
with high parallelism
SOC Complexity / Abstraction
Conquer the SoC Complexity
 Use a known real entity
 A pre-designed component (IP, VC reuse)

 A platform (architecture reuse)

 Partition
 Based on functionality

 Hardware and software

 Modeling

 At different level
 Consistent and accurate
Example-Set Top Box Controller
IBM’s SoC
Generic Wireless / Computing
Emotion Engine in PS2
Traditional Embedded System
Power Supply
Ethernet Audio CLK
CLK
MAC Codec

GP I/O Interrupt
Controller
Timer
Address
Decode
Unit
CPU UART
L
(uP / DSP) Co- C
Memory Proc. custom
CLK Controller IF-logic

SRAM SRAM SRAM Display

SDRAM SDRAM Controller
Traditional Embedded System
Power Supply
Ethernet Audio CLK
CLK
MAC FPGA Codec

GP I/O Interrupt
Controller
Timer
Address
Decode
Unit
CPU UART
L
(uP / DSP) Co- C
Memory Proc. custom
CLK Controller IF-logic

SRAM SRAM SRAM Display

SDRAM SDRAM Controller
Configurable System on Chip (CSoC)
Audio
Codec EPROM

Power Supply

L
C

SRAM SRAM SRAM SDRAM SDRAM

Artificial Intelligence
 Areas of AI and Some
Dependencies
Knowledge
Search Logic Representation

Machine
Learning Planning

Expert
NLP Vision Robotics Systems
 What is Artificial Intelligence ?
• making computers that think?
• the automation of activities we associate with human
thinking, like decision making, learning ... ?
• the art of creating machines that perform functions that
require intelligence when performed by people ?
• the study of mental faculties through the use of
computational models ?
What is Artificial Intelligence ?
• the study of computations that make it possible to
perceive, reason and act ?
• a field of study that seeks to explain and emulate
intelligent behaviour in terms of computational
processes ?
• a branch of computer science that is concerned with
the automation of intelligent behaviour ?
• anything in Computing Science that we don't yet
know how to do properly ? (!)
 What is Artificial Intelligence ?

THOUGHT Systems that thinkSystems that think

like humans rationally

Systems that act Systems that act

BEHAVIOUR like humans rationally

HUMAN RATIONAL
 Systems that act like humans:
Turing Test
• “The art of creating machines that
perform functions that require
intelligence when performed by people.”
(Kurzweil)
• “The study of how to make computers do
things at which, at the moment, people
are better.” (Rich and Knight)
 Systems that act like humans

?
• You enter a room which has a computer terminal. You
have a fixed period of time to type what you want into
the terminal, and study the replies. At the other end of
the line is either a human being or a computer system.

• If it is a computer system, and at the end of the period

you cannot reliably determine whether it is a system or
a human, then the system is deemed to be intelligent.
 Systems that act like humans

• The Turing Test approach

– a human questioner cannot tell if
• there is a computer or a human answering his
question, via teletype (remote communication)
– The computer must behave intelligently

• Intelligent behavior
– to achieve human-level performance in all
cognitive tasks
 Systems that act like humans
• These cognitive tasks include:
– Natural language processing
• for communication with human
– Knowledge representation
• to store information effectively & efficiently
– Automated reasoning
• to retrieve & answer questions using the stored
information
– Machine learning
• to adapt to new circumstances
 The total Turing Test
• Includes two more issues:
– Computer vision
• to perceive objects (seeing)
– Robotics
• to move objects (acting)
 What is Artificial Intelligence ?

THOUGHT Systems that thinkSystems that think

like humans rationally

Systems that act Systems that act

BEHAVIOUR like humans rationally

HUMAN RATIONAL
 Systems that think like humans:
cognitive modeling
• Humans as observed from ‘inside’
• How do we know how humans think?
– Introspection vs. psychological experiments
• Cognitive Science

• “The exciting new effort to make computers think …

machines with minds in the full and literal sense”
(Haugeland)

• “[The automation of] activities that we associate with

human thinking, activities such as decision-making,
problem solving, learning …” (Bellman)
What is Artificial Intelligence ?

THOUGHT Systems that thinkSystems that think

like humans rationally

Systems that act Systems that act

BEHAVIOUR like humans rationally

HUMAN RATIONAL
 Systems that think ‘rationally’
"laws of thought"
• Humans are not always ‘rational’
• Rational - defined in terms of logic?
• Logic can’t express everything (e.g. uncertainty)
• Logical approach is often not feasible in terms of
computation time (needs ‘guidance’)
• “The study of mental facilities through the use of
computational models” (Charniak and
McDermott)
• “The study of the computations that make it
possible to perceive, reason, and act” (Winston)
What is Artificial Intelligence ?

THOUGHT Systems that thinkSystems that think

like humans rationally

Systems that act Systems that act

BEHAVIOUR like humans rationally

HUMAN RATIONAL
 Systems that act rationally:
“Rational agent”
• Rational behavior: doing the right thing
• The right thing: that which is expected to
maximize goal achievement, given the
available information
• Giving answers to questions is ‘acting’.

• I don't care whether a system:

– replicates human thought processes
– makes the same decisions as humans
– uses purely logical reasoning
 Systems that act rationally
• Logic  only part of a rational agent, not all of
rationality
– Sometimes logic cannot reason a correct
conclusion
– At that time, some specific (in domain) human
knowledge or information is used

• Thus, it covers more generally different situations

of problems
– Compensate the incorrectly reasoned
conclusion
 Systems that act rationally

• Study AI as rational agent –

2 advantages:
– It is more general than using logic only
• Because: LOGIC + Domain knowledge
– It allows extension of the approach with more
scientific methodologies
 Rational agents
An agent is an entity that perceives and acts

This course is about designing rational agents

Abstractly, an agent is a function from percept histories to actions:

[f: P*  A]
For any given class of environments and tasks, we seek the agent
(or class of agents) with the best performance

Caveat: computational limitations make perfect rationality

unachievable

 design best program for given machine resources

• Artificial
– Produced by human art or effort, rather than
originating naturally.
• Intelligence
– is the ability to acquire knowledge and use it" [Pigford and
Baur]

• So AI was defined as:

– AI is the study of ideas that enable computers to be
intelligent.
– AI is the part of computer science concerned with
design of computer systems that exhibit human
intelligence(From the Concise Oxford Dictionary)
From the above two definitions, we can see
that AI has two major roles:
– Study the intelligent part concerned with
humans.
– Represent those actions using computers.
 Goals of AI
• To make computers more useful by letting
them take over dangerous or tedious tasks
from human

• Understand principles of human

intelligence
 The Foundation of AI
• Philosophy
– At that time, the study of human intelligence
began with no formal expression
– Initiate the idea of mind as a machine and its
internal operations
 The Foundation of AI
Mathematics formalizes the three main area
of AI: computation, logic, and probability
Computation leads to analysis of the problems
that can be computed
complexity theory

Probability contributes the “degree of belief” to

handle uncertainty in AI

Decision theory combines probability theory and

utility theory (bias)
 The Foundation of AI

• Psychology
– How do humans think and act?
– The study of human reasoning and acting
– Provides reasoning models for AI
– Strengthen the ideas
• humans and other animals can be considered as
information processing machines
 The Foundation of AI
• Computer Engineering
– How to build an efficient computer?

– Provides the artifact that makes AI application

possible

– The power of computer makes computation of

large and difficult problems more easily

– AI has also contributed its own work to computer

science, including: time-sharing, the linked list
data type, OOP, etc.
 The Foundation of AI
• Control theory and Cybernetics
– How can artifacts operate under their own
control?
– The artifacts adjust their actions
• To do better for the environment over time
• Based on an objective function and feedback from
the environment
– Not limited only to linear systems but also
other problems
• as language, vision, and planning, etc.
 The Foundation of AI
• Linguistics
– For understanding natural languages
• different approaches has been adopted from the
linguistic work
– Formal languages
– Syntactic and semantic analysis
– Knowledge representation
 The main topics in AI
Artificial intelligence can be considered under a number of
headings:
– Search (includes Game Playing).
– Representing Knowledge and Reasoning with it.
– Planning.
– Learning.
– Natural language processing.
– Expert Systems.
– Interacting with the Environment
– (e.g. Vision, Speech recognition, Robotics)
Some Advantages of Artificial
Intelligence

– more powerful and more useful computers

– new and improved interfaces
– solving new problems
– better handling of information
– relieves information overload
– conversion of information into knowledge
 The Disadvantages

– increased costs
– difficulty with software development - slow
and expensive
– few experienced programmers
– few practical products have reached the
market as yet.
 Search
• Search is the fundamental technique of AI.
– Possible answers, decisions or courses of action are structured
into an abstract space, which we then search.
• Search is either "blind" or “uninformed":
– blind
• we move through the space without worrying
about what is coming next, but recognising the
answer if we see it
– informed
• we guess what is ahead, and use that information
to decide where to look next.
• We may want to search for the first answer that satisfies our goal,
or we may want to keep searching until we find the best answer .
Knowledge Representation & Reasoning
• The second most important concept in AI

• If we are going to act rationally in our environment, then we must have

some way of describing that environment and drawing inferences from
that representation.
– how do we describe what we know about the world ?
– how do we describe it concisely ?
– how do we describe it so that we can get hold of the right piece of
knowledge when we need it ?
– how do we generate new pieces of knowledge ?
– how do we deal with uncertain knowledge ?
 Knowledge

Declarative Procedural

• Declarative knowledge deals with factoid questions (what is the capital of India?
Etc.)
• Procedural knowledge deals with “How”
• Procedural knowledge can be embedded in declarative knowledge
 Planning
Given a set of goals, construct a sequence of actions that achieves those
goals:
– often very large search space
– but most parts of the world are independent of most other parts
– often start with goals and connect them to actions
– no necessary connection between order of planning and order of
execution
– what happens if the world changes as we execute the plan and/or our
actions don’t produce the expected results?
 Learning
• If a system is going to act truly appropriately,
then it must be able to change its actions in
the light of experience:
– how do we generate new facts from old ?
– how do we generate new concepts ?
– how do we learn to distinguish different
situations in new environments ?
 Interacting with the Environment
• In order to enable intelligent behaviour, we will have to
interact with our environment.
• Properly intelligent systems may be expected to:
– accept sensory input
• vision, sound, …
– interact with humans
• understand language, recognise speech,
generate text, speech and graphics, …
– modify the environment
• robotics
 History of AI
• AI has a long history
– Ancient Greece
• Aristotle
– Historical Figures Contributed
• Ramon Lull
• Al Khowarazmi
• Leonardo da Vinci
• David Hume
• George Boole
• Charles Babbage
• John von Neuman
– As old as electronic computers themselves (c1940)
The ‘von Neuman’ Architecture
 History of AI
• Origins
– The Dartmouth conference: 1956
• John McCarthy (Stanford)
• Marvin Minsky (MIT)
• Herbert Simon (CMU)
• Allen Newell (CMU)
• Arthur Samuel (IBM)
• The Turing Test (1950)
• “Machines who Think”
– By Pamela McCorckindale
 Periods in AI
• Early period - 1950’s & 60’s
– Game playing
• brute force (calculate your way out)
– Theorem proving
• symbol manipulation
– Biological models
• neural nets
• Symbolic application period - 70’s
– Early expert systems, use of knowledge
• Commercial period - 80’s
– boom in knowledge/ rule bases
 Periods in AI cont’d
• period - 90’s and New Millenium
• Real-world applications, modelling, better
evidence, use of theory, ......?
• Topics: data mining, formal models, GA’s, fuzzy
logic, agents, neural nets, autonomous systems
• Applications
– visual recognition of traffic
– medical diagnosis
– directory enquiries
– power plant control
– automatic cars
 Fashions in AI
Progress goes in stages, following funding booms and crises: Some examples:
1. Machine translation of languages
1950’s to 1966 - Syntactic translators
1966 - all US funding cancelled
1980 - commercial translators available

2. Neural Networks
1943 - first AI work by McCulloch & Pitts
1950’s & 60’s - Minsky’s book on “Perceptrons” stops nearly all work on nets
1986 - rediscovery of solutions leads to massive growth in neural nets research

The UK had its own funding freeze in 1973 when the Lighthill report reduced AI work
severely -Lesson: Don’t claim too much for your discipline!!!!
Look for similar stop/go effects in fields like genetic algorithms and evolutionary
computing. This is a very active modern area dating back to the work of Friedberg in
1958.
Symbolic and Sub-symbolic AI
• Symbolic AI is concerned with describing and
manipulating our knowledge of the world as explicit
symbols, where these symbols have clear relationships to
entities in the real world.

• Sub-symbolic AI (e.g. neural-nets) is more concerned

with obtaining the correct response to an input stimulus
without ‘looking inside the box’ to see if parts of the
mechanism can be associated with discrete real world
objects.
 AI Applications

• Autonomous Planning & Scheduling:

– Autonomous rovers.
 AI Applications
• Autonomous Planning & Scheduling:
– Telescope scheduling
 AI Applications
• Autonomous Planning & Scheduling:
– Analysis of data:
 AI Applications
• Medicine:
– Image guided surgery
 AI Applications
• Medicine:
– Image analysis and enhancement
 AI Applications
• Transportation:
– Autonomous vehicle control:
 AI Applications
• Transportation:
– Pedestrian detection:
 AI Applications
Games:
 AI Applications
• Games:
 AI Applications
• Robotic toys:
 AI Applications
Other application areas:
• Bioinformatics:
– Gene expression data analysis
– Prediction of protein structure

• Text classification, document sorting:

– Web pages, e-mails
– Articles in the news

• Video, image classification

• Music composition, picture drawing
• Natural Language Processing .
• Perception.
Intelligent Embedded
Systems
(AI + ES)
Embedded System (ES)
• Embedded system have penetrated our life
to a point ensure our daily business in their
absence.

• ES have some common decisive

requirements among others the real time
constraints, the reduced energy
consumption and the reliability assurance.
Embedded System (ES)
• First, ES are qualified as Real Time (RT) systems. A RT
system is any system where its correction depends not
only results of computations, but also on the time instant
at which these results become available.

• RT system can be hard (i.e. critical), soft, firm or any

combinations of them.

• RT system typically incorporate a RTOS, latter is

responsible of many vital activities for ES such as tasks
scheduling, Input/output management, security and so
on.
Intelligent Embedded Systems (IES)
• Nowadays, ES are more complex, more open
and networked and integrate parts that can
function in hostile, dynamic environments.

• Probably with uncertain or partial knowledge

autonomously, simulating a bit of some human
intellectual activities such as reasoning,
learning, memorization, perception, decision-
making, self-adaption and self-optimization.
Intelligent Embedded Systems (IES)
• The exponential progress in the hardware
technology conducting to the appearance of multi-
core and parallel computing on chips.

• Very high performance processing and

reconfigurable hardware render embedding AI in
ES possible.

• The coupling between AI and ES is not trivial at

all, since the two field have different philosophies.
Intelligent Embedded Systems (IES)
• AI deals with more complex cognitive,
theoretically with unlimited resources tasks,
ES are by nature reactive and have limited
resources.

• The integration of AI into ES lead to the

emergence of what we call “intelligent
embedded systems” .
Intelligent Embedded Systems (IES)
The most famous AI models and methods as
– Artificial Neural Networks (ANN)
– Reinforcement learning
– Multi agent systems (MAS)
– Swarm intelligence
– Genetic Algorithms
– Fuzzy logic
– Constraint Programming
– Game Theory
– Cellular Automata
– Artificial Immune Systems (AIS)
Intelligent Embedded Systems (IES)
• In 2003 Elmenreich, identified some potential reasons
for using an intelligent solution for ES among them

– dependability,
– efficiency,
– autonomy,
– easy modeling,
– maintenance costs
– insufficient alternatives
Intelligent Embedded Systems (IES)
• IES are ES having the capacity of reasoning
about their external environments even in the
presence of uncertainty and adapt their
behavior accordingly.

• IES have some main characteristics such as

self-learning, self-optimizing and self-
repairing. Therefore, IES can include
knowledge-based technology.
Intelligent Embedded Systems (IES)
• This gives rise to the possibility of developing systems
that can learn from their environment and that can
change their own control programs to adapt to new
situations.

• Intelligent WSN (wireless sensors networks), intelligent

vehicles, and robots are becoming very popular.

• IES applications are growing more and more covering a

large spectrum of domains such as farming, smart
buildings, education and transport.
AI Based Methods to Resolve
Real-Time Scheduling for
Embedded Systems
Classification of RT scheduling
algorithms for ES
Classification of Energy-aware
algorithms
Classification of AI methods used
to resolve RT scheduling