Technologies and platforms for Artificial Intelligence
____________________
• Technological evolution and AI revolution
Hi and welcome back! In this lecture we will highlight the importance of the interplay between
hardware and software for effective and efficient AI applications. From the AI perspective, hardware
and software are two sides of the same coin. We started analyzing this interplay by looking at the range
of different technological solutions for AI comprising Internet of Things, Embedded PCs, Personal
Computer, Edge Computing Systems, and Data centers. These technological solutions are nowadays part
of our computing ecosystems and, explicitly or implicitly, they are always part of the game. But what
about AI? How crucial are these technologies in the AI revolution? Let’s see some examples. Data centers
always played a crucial role in AI and, in particular, in the training and inference of very large neural
networks, thanks to the very high computing and memory abilities. For example, data centers allowed
to train Watson, the IBM system that won Jeopardy TV game show in 2011, or AlphaGo, the software
that beat in 2016 the number 1 GO player in the world, or also Libratus, the AI that defeated in 2017
four Texas hold ‘em World's top players. But let’s move to 2020, when the largest and more complex
neural network ever designed has been presented. This neural network, called GPT-3, is meant for
natural language processing and comprises 175 billion parameters. Here data centers played a crucial
role in the training of GPT-3 since its training on regular computers would have required several
hundreds of years. Edge computing systems represent the technological solution for autonomous
vehicles and autonomous robots, just to name a few, while there is one interesting anecdote to tell
about Personal Computers. We all know about Deep Blue, the computer that defeated the world chess
champion Garry Kasparov in 1997. This amazing news opened a new era in the relationship between AI
and Humans. Interestingly, the computational and memory abilities of Deep Blue are in line with what
we have today in our Personal Computers. Last but not least, embedded Personal Computers and IoT
systems, being able to pervasively operate in the environment, provide the technological infrastructure
for smart home and building, smart cities and e-health just to name a few. Intelligent objects and smart
speakers are relevant examples in this field. We mentioned the interplay between hardware and
AI104 – Technologies and platforms for Artificial Intelligence| Audio transcription 1/2
software, but how can we deal with such a large technological heterogeneity in the available hardware
solutions? We are moving from tiny battery-powered objects to large-scale datacenters. How can the
intermediate AI software layer comprising frameworks, platforms and tools operate in such a wide range
of hardware technologies? The answer is simple and complex at the same time. The easy one is that we
have different solutions depending on the target reference hardware. The complex one is that the
choice of the software to use strictly depends on the hardware, hence requiring a hardware-software
co-design of the AI system. Let’s see some examples: In data centers and edge computing systems we
have enough computational and memory ability to consider all the most-powerful versions of off-the-
shelf AI frameworks, platforms and tools. Examples of these AI software solutions are the well-known
TensorFlow, PyTorch, Microsoft cognitive toolkit and ONNX. Typically, these solutions provide both
inference and training functionalities for machine and deep learning solutions. Embedded PCs, being
less powerful than datacenters and edge computing systems, rely on simplified versions of AI software
solutions. TensorflowLite, Core ML, Azure IoT Edge and AWS Greengrass are examples of frameworks,
platforms, and tools for Embedded Personal Computers. Differently from what is available for
datacenters and edge computing, the AI solutions for embedded Personal Computers often provide only
the inference ability for machine and deep learning models, while the training of such models is
generally carried out on more powerful computing units. Finally, IoT units require specifically-designed
AI solutions, because they are severely constrained on computation, memory and power consumption.
For this purpose, approximated and quantized versions of AI frameworks, platforms and tools must be
considered such as TinyML and STM32Cube.AI As we explored, the technological evolution of AI hardware
made giant steps to support more and more complex neural networks and models. Unfortunately, the
algorithmic revolution is going faster. For example, the recently introduced GPT-3, as we already saw,
is the largest network ever developed and trained with more than 175 billion parameters. As you can
see, this trend is steadily growing and the complexity of such neural networks doubles every 3 months
and a half. The only reference we have for the hardware technological evolution is the Moore’s law that
estimates the doubling of hardware performance every 18 months. The gap between the algorithm and
the technological evolution is huge. How can we deal with this? The answer resides in considering
additional hardware specifically designed for AI. And thanks to this specialized hardware able to
parallelize the code and rely on hardware accelerated operations, we can boost the performance of
machine and deep learning solutions by reducing the inference and training time. Graphics Processing
Units (the GPU), Tensor Processing Units (the TPU) and the Field-Programmable Gate Array (the FPGA)
are examples of these hardware accelerators for AI. It is worth mentioning that, due to their complexity
and power consumption, these hardware accelerators are typically present in datacenters and edge
computing systems. Seldom we can find them in embedded PCs, while IoT units are not considered
available technological solution for hardware accelerators. That’s said, let’s start exploring the
technological families for AI.
AI104 – Technologies and platforms for Artificial Intelligence| Audio transcription 2/2
