Microsoft’s VASA-1 AI Tool Can Create “Talking Faces” From Photos

Microsoft’s Research arm has just detailed its ongoing work on VASA-1, a new AI framework that can create realistic-looking “talking faces” using just a picture and a voice clip. The AI tool can also synchronize lip movements with the audio with impressive accuracy, and the virtual characters are capable of showing various facial expressions that make them look very human-like.

Microsoft published multiple video examples showing how its VASA-1 model can be used to create virtual talking characters that still may not be able to fool everyone, but it seems that we’re getting close to that reality. And again, all this model needs is a photo and a voice clip to create what could be qualified has a deepfake.

Below is an example of “talking face” created with VASA-1 using an AI-generated picture of a non-existing person. Microsoft has other examples on the blog post, including a clip of a singing Mona Lisa.

Impressive! A few more months and I won't even need to turn on my webcam for meetings! #Microsoft #AIhttps://t.co/RzNDRNr18t pic.twitter.com/XKQiYTYXhl

— Rudy Huyn (@RudyHuyn) April 18, 2024

“Through extensive experiments including evaluation on a set of new metrics, we show that our method significantly outperforms previous methods along various dimensions comprehensively,” the company explained. “It paves the way for real-time engagements with lifelike avatars that emulate human conversational behaviors.”

If the Microsoft Research team acknowledged that its technology could easily be used to impersonate humans, it also pointed out that there could be many positive use cases for virtual AI avatars. “The benefits – such as enhancing educational equity, improving accessibility for individuals with communication challenges, offering companionship or therapeutic support to those in need, among many others – underscore the importance of our research and other related explorations,” the team explained.

For now, however, Microsoft’s research arm made it clear that it has no plans to make its VASA-1 AI model available to the public “until we are certain that the technology will be used responsibly and in accordance with proper regulations.” The company also explained that all clips include identifiable artifacts to prevent the technology from being misused.