AI self-assessment of AI reliability is developed

AI is being used in a variety of situations, such as detecting the sound of COVID-19 coughing and generating human-like text. A new technology has been developed that allows the AI to judge whether the calculation result calculated by this AI is correct.

Since AI is also applied to fields related to life risk, such as autonomous vehicles, it is important to evaluate the reliability of AI calculation results. For this reason, a CSAIL researcher at the MIT Computer Science and Artificial Intelligence Research Institute devised a system that evaluates the reliability of AI calculation results on AI itself. The developed reliability evaluation system aims to recognize to the AI itself whether the AI is making wrong judgments with DER (Deep Evidential Regression). The goal is to efficiently and reliably detect 1% mistakes.

So far, there are systems for evaluating the reliability of AI, and there have been cases where AI is used in the system. However, any existing system takes a tremendous amount of time to evaluate its reliability. Meanwhile, DER can quickly evaluate the reliability and can evaluate the reliability at the same time as the AI calculation result.

DER assumes that the test in the research phase is used for autonomous driving technology, and outputs the results of reliability determination at the same time as the results using AI that determines the image depth. The AI used in the study succeeded in judging with the same precision as the existing AI. In addition, AI said that when the calculation result was wrong, the AI was outputting that the calculation result was the least reliable. In other words, AI succeeded in performing the calculation result and reliability evaluation simultaneously and precisely.

In addition, AI equipped with DER was used outdoors to learn from indoor images, but AI is warning that the learned data can be applied. This is because the AI recognizes that it is different from usual, and it can be considered that this flag is used for the purpose of detecting abnormalities in the medical field.

The research team explains that DER can be applied to a variety of AIs and has the potential to reveal the amount of errors and missing data that occur when using AI by evaluating reliability by using it in AI evaluation using a learning model. AI is widely used in fields related to human life such as medical fields and autonomous driving of automobiles. Related information can be found here .