Google “Achieve high speed neural network with sparse inference”

It is reported that Google has significantly increased the inference speed by speeding up the mobile machine learning software library TensorFlow Lite and the neural network inference optimization library (XNNPACK).

Sparse is a word that means shrewd. In the interpretation of big data, etc., there are many situations where the entire data is large, but only a small part of meaningful data is present. Sparse modeling is used to improve MRI, X-ray, and CT resolution, and to increase the speed and precision of three-dimensional structure calculations in a methodology that selects and analyzes only meaningful data with these characteristics.

Google’s newly announced machine learning software library for mobile, TensorFlow Lite, and neural network inference optimization library have been updated and a new sparse collection fragment has been realized. With the new update, it is possible to achieve a significant increase in inference speed so that it can detect whether the model to be analyzed is sparse.

In real-time processing of Google Meat, a video conferencing application, the sparse model realizes lower processing time and higher FPS than the existing model. According to Google, it has succeeded in speeding up the processing speed by 30% by reducing the size of 70% while maintaining the image quality of the subject.

Similarly, in the hand shape detection model (MediaPipe Hands), we achieved a 50% speed improvement through sparse. In the existing model, the processing speed is around 60ms, but the sparse model reached a speed of less than 25ms, which is a little faster than the existing model.

Google said that sparseization is a simple and powerful technique to improve neural network CPU inference, so it will continue to do this kind of research in the future. Related information can be found here.