Facebook’s next big AI project is training its machines on public videos of users

Understanding AI systems as what is happening in the video as a whole is one of the most difficult challenges as a human – and the biggest potential breakthroughs in the world of machine learning. Today, Facebook made an announcement New initiative Hopefully, this will give it an edge in this resulting work: training its AI on public videos of Facebook users.

Access to training data is one of the biggest competitive advantages in AI, and by collecting this resource from millions and millions of users, tech giants such as Facebook, Google and Amazon are able to move forward in various fields. While Facebook has already trained machine vision models on millions of images collected from Instagram, it has not previously announced projects of similar ambition for video understanding.

“By learning from global streams of publicly available video spanning almost every country and hundreds of languages, our AI systems will not only improve accuracy, but adapt to our rapidly growing world, and nuances across different cultures and regions and Will recognize visual cues, “the company in a blog. Project Title Learn from video, Is also part of Facebook’s “comprehensive efforts to build learning machines like humans.”

The resulting machine learning model will be used to create new content recommendation systems and moderation tools, Facebook says, but could do so much more in the future. The AI ​​that can understand the content of a video can give Facebook unprecedented insight into users’ lives, allowing them to analyze their hobbies and interests, preferences in brands and clothing, and countless other personal details. Of course, Facebook already has such information through its current ad-target operations, but being able to parse video via AI is incredibly rich (and aggressive) of data in its stores Source will be added.

Facebook is unclear about its future plans for AI models trained on users’ videos. Company told The Reporter Door Such models can be used in many ways, from capturing video to creating advanced search functions, but did not answer the question of whether they would be used to collect information for ad-targeting. Similarly, when asked whether users would have to agree to use their videos to train Facebook’s AI, or if they could opt out, the company responded by simply noting that Data policy Users’ uploaded content can be used for “product research and development”. Facebook also did not respond to questions about how much video would actually be collected for training its AI systems or how the data would be accessed by the company’s researchers.

In its blog post announcing the project, however, the social network pointed to a futuristic, speculative use: using AI to retrieve “digital memories” captured by smart glasses.

Facebook plans to release a pair of consumer smart glasses sometime this year. Details about the device are unclear, but it is likely that these or future glasses will include integrated cameras to capture the wearer’s vision. If an AI system can be trained to understand the content of a video, it will allow users to search for past recordings, just as many photo apps allow people to search for specific locations, objects, or people. (This is information, incidentally, often trained on user data by AI systems.)

Picture: Facebook

According to Facebook recording videos with Smart Glass “becomes the norm”, people need to be able to recall specific moments from the vast bank of their digital memory as much as they capture them. It gives me an example of a user conducting a search with the phrase “Every time we sing Happy Birthday to Grandma,” before the corresponding clip is served. As the company notes, such a search would require that AI systems establish connections between types of data, calling them “cakes, candles, people singing various birthday songs, and much more” to the phrase ‘birthday. Teach matching with ‘wishes’. As humans do, AI will need to understand rich concepts involving different types of sensory input.

Looking to the future, the combination of smart glass and machine learning can be referred to as “worldcrapping” – capturing granular data about the world by turning smart glass wearers into CCTV cameras. As described in practice Last year’s report Guardian: “Each time someone arrives at the supermarket, their smart glasses will record real-time pricing data, stock levels, and browsing habits; Every time he opened a newspaper, his glasses showed which stories he read, which advertisements he appeared in and which celebrities stared at the beach. “

This is an extreme result and not a part of the research Facebook says it is currently exploring. But it does describe the potential importance of pairing advanced AI video analysis with smart glasses – something the social network is clearly keen to do.

By comparison, the only use of its new AI video analysis tool that Facebook is currently revealing is relatively mundane. With the announcement of learning from video today, Facebook says it has deployed a new content recommendation system based on video work in its Tiktok-clone reels. “Popular videos often have the same music set in which the same dance plays, but are created and acted by different people,” Facebook says. By analyzing the content of the video, Facebook’s AI can suggest similar clips to users.

Such content recommendation algorithms are not without potential problems, however. a Recent report from MIT Technology Review It highlighted how the development of social networks and the emphasis of user engagement has prevented its AI team from fully stating that algorithms can spread misinformation and encourage political polarization. In form of Technology review The article states: “The [machine learning] Models that increase engagement also support controversy, misinformation and extremism. “This creates conflict between the duties of Facebook’s AI ethics researchers and the credit for spurring the company’s growth.

Facebook is not the only large tech company pursuing advanced AI video analytics, nor is it the only one to leverage users’ data to do so. For example, Google maintains over 8 million publicly accessible research datasets YouTube videos are curated and partially labeled “To Help Accelerate Research on Large-Scale Video Understanding.” The search giant’s advertising operations may equally benefit from AI that understands the content of the video, even if the end result is only serving more relevant ads in YouTube.

Facebook, however, thinks it has a particular advantage over its rivals. Not only does it have sufficient training data, but it is also moving more and more resources into an AI method known as self-supervised learning.

Typically, when AI models are trained on data, they are input Label by humans: For example tagging pictures into objects or transcribing audio recordings. If you have ever solved CAPTCHA by identifying fire hydrants or pedestrian crossings, it is likely that the data you have labeled helps to train AI. But self-observable learning does away with the label, speeds up the training process, and, some researchers believe, results in AI systems teaching themselves to engage in dots, making deeper and more meaningful analysis . Facebook is so optimistic about self-supervised learning Called It is a “black matter of intelligence”.

The company says its future work on AI video analysis will focus on quasi and self-supervised learning methods, and such technologies have “already improved our computer vision and speech recognition systems.” With such a plethora of video content available from Facebook’s 2.8 billion users, it certainly makes sense to skip the labeling part of AI training. And if social networks can teach their machine learning models to understand video radically, then who knows what they can learn?