OpenAI’s state-of-the-art machine vision AI is fooled by handwritten notes

Researchers at the machine learning lab OpenAI have done Discovered That their state-of-the-art computer vision system can be defeated by more sophisticated devices than pens and pads. As depicted in the image above, simply typing the name of an object and pasting it onto another is enough to make the software look inaccurately.

“We refer to these attacks The printing The attack, “Write to researchers at OpenAI blog post. “By exploiting the model’s ability to read text convincingly, we find that even pictures of text written by hand can often fool the model.” They note that such attacks are similar to “repulsive pictures” that can fool commercial machine vision systems, but are much simpler to produce.

Adversarial images present a real threat to the system that relies on machine vision. For example, researchers have shown that they can cheat software in Tesla’s self-driving cars Change lanes without warning Just placing some stickers on the road. Such attacks are a serious threat to many types of AI apps, from medical to military.

But the threat posed by this specific attack is not a concern, at least for now. The OpenAI software in question is an experimental system called CLIP that is not deployed in any commercial product. Indeed, the very nature of CLIP’s unusual machine learning architecture created a weakness that enabled this attack to succeed.

Image: OpenAI

The purpose of CLIP is to find out how AI systems can learn to identify objects without close supervision by training on a large database of image and text pairs. In this case, OpenAI used some 400 million image-text pairs scrapped from the Internet to train CLIP, which was Unveiled in january

.

This month, researchers at OpenAI published a new paper detailing how they open CLIP to see. They came to know what they are calling “multimodal neurons” – individual components in machine learning networks that respond not only to images of objects but also to related text. One of the reasons it is exciting is that it seems from the mirror how the human brain reacts to stimuli, where single brain cells have been seen Feedback on abstract concepts Rather than specific examples. Research from OpenAI suggests that it may be possible for AI systems to internally recover such knowledge as humans do.

In the future, it may lead to more sophisticated vision systems, but right now, such approaches are still in their infancy. While any human can tell you the difference between an apple and a piece of paper with the word “apple” written on it, software like CLIP cannot. The same capability that allows programs to add words and images at an abstraction level creates this unique weakness, which OpenAI describes as a “degradation of abstraction”.

Image: OpenAI

Another example given by the lab is the neuron in the CLIP that identifies the piggy bank. This component reacts not only to pictures of piggy banks, but also dollar signs. As in the example above, this means that if you overlay it with “$ $ $” strings, assume that it is half the price at your local hardware store.

Researchers also found that CLIP’s multimodal neurons encoded exactly the way you can find your data while sourcing it from the Internet. They note that the neuron for the “Middle East” is also associated with terrorism and it is revealed that “a neuron that fires for both dark-skinned people and gorillas.” This is a replica of a notorious error in Google’s image recognition system, which tagged whites as gorillas. This is yet another example of different machine intelligence for humans – and why we are pulling the former apart to understand how our lives work before trusting AI.