TDC2019 Intel Software Day - Inferencia de IA em edge devices
2 likes569 views
This document discusses Intel's compiler optimizations and how they may differ depending on the microprocessor. It notes that: - Intel's compilers may optimize differently for non-Intel microprocessors, including optimizations for SSE2, SSE3, and SSSE3 instruction sets. - Intel does not guarantee the availability, functionality, or effectiveness of any optimization on non-Intel microprocessors. - Microprocessor-dependent optimizations are intended for use with Intel microprocessors only. Certain non-Intel specific optimizations are also reserved for Intel microprocessors.
TDC2019 Intel Software Day - Inferencia de IA em edge devices
- 2. Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness or any optimization on microprocessors not manufactured by Intel. Microprocessor- dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804. 2
- 3. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com. Performance estimates were obtained prior to implementation of recent software patches and firmware updates intended to address exploits referred to as "Spectre" and "Meltdown." Implementation of these updates may make these results inapplicable to your device or system. Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. This document contains information on products, services, and processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. Any forecasts of goods and services needed for Intel’s operations are provided for discussion purposes only. Intel will have no liability to make any purchase in connection with forecasts published in this document. Arduino* 101 and the Arduino infinity logo are trademarks or registered trademarks of Arduino, LLC. Altera, Arria, the Arria logo, Intel, the Intel logo, Intel Atom, Intel Core, Intel Nervana, Intel Xeon Phi, Movidius, Saffron, and Xeon are trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others. Copyright ® 2018 Intel Corporation. All rights reserved. 3
- 4. This document contains information on products, services, and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications, and roadmaps. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Learn more at intel.com, or from the OEM or retailer. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit www.intel.com/performance. Cost-reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. Statements in this document that refer to Intel’s plans and expectations for the quarter, the year, and the future are forward-looking statements that involve a number of risks and uncertainties. A detailed discussion of the factors that could affect Intel’s results and plans is included in Intel’s SEC filings, including the annual report on Form 10-K. The products described may contain design defects or errors, known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on request. Performance estimates were obtained prior to implementation of recent software patches and firmware updates intended to address exploits referred to as "Spectre" and "Meltdown." Implementation of these updates may make these results inapplicable to your device or system. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. Intel, the Intel logo, Pentium, Celeron, Atom, Core, Xeon, Movidius, Saffron, and others are trademarks of Intel Corporation in the United States and other countries. *Other names and brands may be claimed as the property of others. Copyright © 2018, Intel Corporation. All rights reserved. 4
- 5. 1. Amalgamation of analyst data and Intel analysis. 2. IDC FutureScape: Worldwide Internet of Things 2017 Predictions (link) 3. IDC FutureScape: Worldwide Internet of Things 2015 Predictions (link) 50% 45% of data will be stored, analyzed, and acted on at the edge22018 2019 of IoT deployments will be network constrained3 By2020 Average internetuser 1.5GB data/day Smart hospital 3TB data/day Autonomous automobile 4TB data/day Connected airplane 40TB data/day Smart factory 1PB data/day 5 5
- 7. AutonomousVehicles ResponsiveRetail Manufacturing EmergencyResponse Financialservices MachineVision Cities/transportation Publicsector 7
- 8. “Computer vision is concerned with the automatic extraction, analysis and understanding of useful information from a single image or a sequence of images.” Source: http://www.bmva.org/visionoverview
- 9. • Image quality (low quality) • Light • Viewpoint / Orientation • Movement deformation • Occlusion • Object variation • Camouflage • Optical Illusions
- 10. • In a nutshell: a color image is a three dimensional array (one element for each pixel, one layer for each color) Computational image manipulation is basically array-based math
- 11. • Open and loose definition: “an interesting part of an image” • Some OpenCV algorithms from version 2.x are patented (removed on OpenCV 3.x)
- 12. 1 Based on selection and connections of computational filters to abstract key features and correlating them to an object. Works well with well defined objects and controlled scene. Difficult to predict critical features in larger number of objects or varying scenes. Traditional Computer Vision Based on application of a large number of filters to an image to extract features. Features in the object(s) are analyzed with the goal of associating each input image with an output node for each type of object. Values are assigned to output node representing the probability that the image is the object associated with the output node. Deep Learning Computer Vision
- 13. SOURCE: Gradient Based Learning Applied to Document Recognition - http://yann.lecun.com/exdb/publis/psgz/lecun-98.ps.gz Feature extraction Classification
- 16. © 2019 Intel Corporation AIIsthedrivingforce Foresight Predictive Analytics Forecast Prescriptive Analytics Act/adapt Cognitive Analytics Hindsight Descriptive Analytics insight Diagnostic Analytics AnalyticsCurveDatadeluge(2019) 25GBper month InternetUser 1 50GBper day SmartCar 2 3TB per day SmartHospital 2 40TBper day AirplaneData 2 1pBper day SmartFactory 2 50PBper day CitySafety 2 WhyAInow? 1. Source: http://www.cisco.com/c/en/us/solutions/service-provider/vni-network-traffic-forecast/infographic.html 2. Source: https://www.cisco.com/c/dam/m/en_us/service-provider/ciscoknowledgenetwork/files/547_11_10-15-DocumentsCisco_GCI_Deck_2014-2019_for_CKN__10NOV2015_.pdf 16 Insights Business Operational Security
- 17. Source: Forrester Research – Artificial Intelligence: Fact, Fiction. How Enterprises Can Crush It; What’s Possible for Enterprises in 2017 Aiadoptionisjustbeginning 58% of business and technology professionals said they're researching AI, but only… 12%said they are currently using AI systems. In a recent Forrester Research survey…
- 18. Machine Learning How do you engineer the best features? Machinelearning 𝑁 × 𝑁 Arjun NEURAL NETWORK 𝒇 𝟏, 𝒇 𝟐, … , 𝒇 𝑲 Roundness of face Dist between eyes Nose width Eye socket depth Cheek bone structure Jaw line length …etc. CLASSIFIER ALGORITHM SVM Random Forest Naïve Bayes Decision Trees Logistic Regression Ensemble methods 𝑁 × 𝑁 Arjun DeepLearning How do you guide the model to find the best features?
- 19. Deeplearning:Trainingvs.inference Lots of labeled data! Training Inference Forward Backward Model weights Forward “Bicycle”? “Strawberry” “Bicycle”? Error Human Bicycle Strawberry ?????? Data set size Accuracy Didyouknow? Training requires a very large data set and deep neural network (i.e. many layers) to achieve the highest accuracy in most cases
- 20. The complexity of the problem (data set) dictates the network structure. The more complex the problem, the more ‘features’ required, the deeper the network.
- 21. © 2019 Intel Corporation SpeedupdevelopmentUsingopenAIsoftware 21 1 An open source version is available at: 01.org/openvinotoolkit *Other names and brands may be claimed as the property of others. Developer personas show above represent the primary user base for each row, but are not mutually-exclusive All products, computer systems, dates, and figures are preliminary based on current expectations, and are subject to change without notice. TOOLKITS App developers libraries Data scientists Kernels Library developers Intel® Distribution of OpenVINO™ Toolkit1 Nauta (Beta) Deep learning inference deployment on CPU/GPU/FPGA/VPU for Caffe*, TensorFlow*, MXNet*, ONNX*, Kaldi* Open source, scalable, and extensible distributed deep learning platform built on Kubernetes Intel-optimized Frameworks And more framework optimizations underway including PaddlePaddle*, CNTK* & others Python R Distributed • Scikit- learn • Pandas • NumPy • Cart • Random Forest • e1071 • MlLib (on Spark) • Mahout Intel® Distribution for Python* Intel® DAAL Intel distribution optimized for machine learning Intel® Data Analytics Acceleration Library (incl machine learning) Intel® nGraph™ Compiler (Beta) Open source compiler for deep learning model computations optimized for multiple devices (CPU, GPU, NNP) from multiple frameworks (TF, MXNet, ONNX) Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) Open source DNN functions for CPU / integrated graphics Visit: www.intel.ai/technology Machinelearning Deeplearning * * * * *
- 22. Today DigitalSecurity&Surveillance 2018+ ScalingtoIndustrial&Retail,EnabledbySWTools OpenVINO™ 22
- 23. What’sInsideIntel®DistributionofOpenVINO™toolkit OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc. OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos Intel® Architecture-Based Platforms Support OS Support: CentOS* 7.4 (64 bit), Ubuntu* 16.04.3 LTS (64 bit), Microsoft Windows* 10 (64 bit), Yocto Project* version Poky Jethro v2.0.3 (64 bit), macOS* 10.13 & 10.14 (64 bit) Intel® Deep Learning Deployment Toolkit Traditional Computer Vision Model Optimizer Convert & Optimize Inference Engine Optimized InferenceIR OpenCV* OpenVX* Optimized Libraries & Code Samples IR = Intermediate Representation file For Intel® CPU & GPU/Intel® Processor Graphics Increase Media/Video/Graphics Performance Intel® Media SDK Open Source version OpenCL™ Drivers & Runtimes For GPU/Intel® Processor Graphics Optimize Intel® FPGA (Linux* only) FPGA RunTime Environment (from Intel® FPGA SDK for OpenCL™) Bitstreams Samples An open source version is available at 01.org/openvinotoolkit (some deep learning functions support Intel CPU/GPU only). Tools & Libraries Intel® Vision Accelerator Design Products & AI in Production/ Developer Kits Open Model Zoo (30+ Pre-trained Models) Samples
- 24. Expedite development, accelerate deep learning inference performance, and speed production deployment. 24 Pretrained Models in Intel® Distribution of OpenVINO™ toolkit Binary Models Face Detection Binary Pedestrian Detection Binary Vehicle Detection Binary ResNet50 Binary Age & Gender Face Detection–standard & enhanced Head Position Human Detection–eye-level & high-angle detection Detect People, Vehicles & Bikes License Plate Detection: small & front facing Vehicle Metadata Human Pose Estimation Action recognition – encoder & decoder Text Detection & Recognition Vehicle Detection Retail Environment Pedestrian Detection Pedestrian & Vehicle Detection Person Attributes Recognition Crossroad Emotion Recognition Identify Someone from Different Videos–standard & enhanced Facial Landmarks Gaze estimation Identify Roadside objects Advanced Roadside Identification Person Detection & Action Recognition Person Re-identification–ultra small/ultra fast Face Re-identification Landmarks Regression Smart Classroom Use Cases Single image Super Resolution (3 models) Instance segmentation and more…
- 25. 25 Use Model Optimizer & Inference Engine for public models & Intel pretrained models • Object Detection • Standard & Pipelined Image Classification • Security Barrier • Object Detection SSD • Neural Style Transfer • Object Detection for Single Shot Multibox Detector using Asynch API+ • Hello Infer Classification • Interactive Face Detection • Image Segmentation • Validation Application • Multi-channel Face Detection
- 26. Optimize/ Heterogeneous Inference engine supports multiple devices for heterogeneous flows. (device-level optimization) Prepare Optimize Model optimizer: Converting Optimizing Preparing to inference (device agnostic, generic optimization) Inference Inference engine lightweight API to use in applications for inference. MKL- DNN cl-DNN CPU: Intel® Xeon®/Intel® Core™/Intel Atom® GPU FPGA Myriad™ 2/X DLA Intel® Movidius™ API Train Train a DL model. Currently supports: Caffe* Mxnet* TensorFlow* ONNX* Extend Inference engine supports extensibility and allows custom kernels for various devices. Extensibility C++ Extensibility OpenCL™ Extensibility OpenCL™/TBD Extensibility TBD ApplicationdevelopmentwithOpenVINO™Toolkit
- 27. • Simple and unified API for inference across all Intel® architecture • Optimized inference on large Intel® architecture hardware targets (CPU/GEN/FPGA) • Heterogeneous support allows execution of layers across hardware types • Asynchronous execution improves performance • Futureproof/scale development for future Intel® architecture processors Inference Engine Common API PluginArchitecture Inference Engine Runtime Intel® Movidius™ API Intel® Movidius™ Myriad™ 2 DLA Intel Integrated Graphics (GPU) CPU: Intel® Xeon®/Intel® Core™/Intel Atom® clDNN Plugin Intel® MKL-DNN Plugin OpenCL™Intrinsics* FPGA Plugin Applications/Service Intel® Arria® 10 FPGA Intel® Movidius™ Plugin
- 28. • Power/Performance Efficiency Varies – Running the right workload on the right piece of hardware higher efficiency – Hardware acceleration is a must – Heterogeneous computing? • Tradeoffs – Power/performance – Price – Software flexibility, portability PowerEfficiency Computation Flexibility Dedicated Hardware GPU CPU X1 X10 X100 Vision Processing Efficiency Vision DSPs FPGA
- 29. 29 HighPerformance&LowPowerforAIInference Intel®neuralcomputestick2 8X¹HIGHER PERFORMANCE On deep neural networks compared to Intel® Movidius™ Neural Compute Stick UPTO + Intel® Movidius™ Myriad™ X VPU Intel® Distribution of OpenVINO™ toolkit Optimized by Powered by MORE CORES. MORE AI INFERENCE. Start quickly with plug-and-play simplicity Develop on common frameworks and out-of-box sample applications Prototype on any platform with a USB port Operate without cloud compute dependence Boost productivity Simplify prototyping Discover efficiencies
- 30. 30
- 31. Intel Distribution for OpenVINO: http://software.intel.com/openvino-toolkit OpenVINO OpenSource: http://01.org/openvinotoolkit Implementações de referência: https://intel.ly/2SwRigI OpenVINO Workshop: https://bit.ly/2Eb6k3e 31