Convolutional neural networks (CNNs) are powerful techniques for AI application development, offering the advantage of accuracy in image-recognition problems. In this talk, Intel software engineer Dmitry Matveev analyzes the performance and scalability of several software development tools that each provide high-performance CNN-based deep learning inference on Intel® architecture.
In just under 30 minutes, Dmitry focuses on two typical data science problems: Image Classification1 and Object Detection2.
The experiment plan:
- Prepare a set of trained models for several dev tools, including Intel® Distribution of OpenVINO™ toolkit, Intel® Optimization of Caffe*, and OpenCV.
- Select a large set of images from each dataset to ensure the performance analysis delivers accurate results; experimentally determine the most appropriate parameters (e.g., batch size and the number of CPU cores used).
- Carry out computational experiments on Endeavor, NASA’s shared-memory supercomputer based on 2nd Generation Intel® Xeon® Scalable Processors (formerly Cascade Lake).
Leveraging the above experiment, this session covers:
- OpenVINO™ toolkit performance, including comparing it to other similar software for CNN-based deep learning inference.
- Analysis of OpenVINO toolkit scaling efficiency using dozens of CPU cores in a throughput mode.
- Results of Intel® AVX-512 VNNI (Vector Neural Network Instructions) performance acceleration in Intel Xeon Scalable Processors.
- Analysis of modern CPU utilization in CNN-based deep learning inference using the Roofline model included in Intel® Advisor.
Check it out.
Download the software
- Intel® Distribution of OpenVINO™ toolkit
- Intel® Advisor standalone download (or get it as part of Intel® Parallel Studio XE or Intel® System Studio)
- Intel® Optimization of Caffe*
1 Image Classification model: ResNet-50; dataset: ImageNET
2 Object Detection model: SSD300; dataset: PASCAL VOC 2012