We can now better scrutinise manufacturers’ claims about AI chip performance improvements thanks to AnTuTu’s latest benchmark.
If you’ve ever read a comprehensive smartphone review, you’ve likely heard of AnTuTu. The company’s smartphone benchmarking tool is often used for testing and comparing the CPU and 3D performance of devices.
With dedicated AI chips now appearing in devices from the mid-range to flagships, AnTuTu has decided it’s time for a benchmark to determine their performance.
In a blog post, AnTuTu says its benchmark uses two categories – ‘Image Classification’, and ‘Object Recognition’.
AI News tested AnTuTu’s benchmark on a Huawei Mate 20 Pro which currently ranks second on AnTuTu’s general performance leaderboard for Android devices. Huawei often brags about the AI performance of its flagship devices.
The first test classifies 200 images as fast as possible using the Inception v3 neural network:
In the second test, 600-frame video is reviewed using the MobileNet SSD neural network:
AnTuTu then delivers an overall benchmark score, along with the scores for each category.
Here is how our Mate 20 Pro fared:
- Overall – 65,222
- Image Classification – 41,717
- Object Detection – 23,505
Each of the categories is further broken down into scores for ‘speed’ and ‘accuracy’. If accuracy is traded for speed, then a lower score will be given.
AnTuTu says this helps to prevent cheating by devices processing the data fast but without providing the right answers. Smartphone manufacturers have been caught artificially-inflating their benchmarks in the past; so it provides added confidence in the results.
For a general look at the AI features in the Mate 20 Pro, see our video below:
You can download the AI benchmark from AnTuTu here.
Interested in hearing industry leaders discuss subjects like this and their use cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, Blockchain Expo, and Cyber Security & Cloud Expo.
