MyHeritage and D-ID partner to bring photos to life with both animations and voice - TechCrunch

Last year, genealogy service MyHeritage went viral after introducing a new "deepfake" feature that allowed users to animate the faces of loved ones in still photos. TikTok users posted videos reacting to the technology, called "Deep Nostalgia," as they brought back relatives they never got to meet or those whose loss they still grieved. To date, more than 100 million photos have been animated with the feature. Now comes the next iteration. Today, MyHeritage along with technology partner D-ID is expanding upon "Deep Nostalgia," with the launch of "LiveStory," a feature that doesn't just bring the people in photos to life with movement, but actually has them speak.

MyHeritage licensed the new technology from D-ID, a Tel Aviv-based startup that works to create patented video reenactment technology powered by AI and deep learning techniques.

D-ID's technologies, available to developers through APIs, have been used by a range of licensees, across media, education, marketing and more. Warner Bros., for instance, worked with D-ID to allow users to personalize a movie trailer with animated photos and for a Harry Potter exhibition. Mondelēz International, advertising agency Publicis and Digitas Vietnam partnered with D-ID on marketing efforts for a local festival. India's short-form video app Josh integrated the facial animation tech as a creative tool. Nonprofits and governments have also used the technology in various public awareness campaigns.

With the debut of LiveStory, MyHeritage is bringing the latest AI tech from D-ID directly to consumers.

To use the feature, consumers can create a free MyHeritage account, which allows them to try out the tech for free several times. Beyond that, they'll need to upgrade to a paid subscription plan for unlimited use of LiveStory.

The technology works to create an animated video narrative of an ancestor's life, so they can tell their own story. This is powered by D-ID's patent-pending Speaking Portrait Technology, which generates a narrated video based on an uploaded photo, then combines it with a synthetic voice generator. The story's narration comes from text input by the user.

To make the lips match the words, D-ID trained a neural network on a database of videos of people speaking. Its technology is able to work with any language, the company says. The MyHeritage implementation, however, supports 31 languages, including dozens of dialects, with both male and female voice options.

"The technology is so good that you don't need a driver video," notes D-ID's co-founder and CEO Gil Perry — meaning, the tech doesn't require a video of a real person's movements which are then mapped to a still image. "You input only text and the photo and then you get the person to speak," he says. The technology is not perfect, Perry admits, likening it instead to "really good lip-syncing."

After the LiveStory is created, users can watch it, share it with friends or post it to social media. They also can customize the story further by editing the text, choosing a different voice or even uploading their own audio recording.

Image Credits: D-ID

Longer-term, D-ID envisions how this technology could be used in a metaverse environment, where digital avatars of people could be animated with AI — and not just faces, but full-body movement in 3D worlds. Perry imagines how users could upload their own childhood photos, those of family members or even historical figures, then animate them in the metaverse, and have conversations.

"Our children will want to converse ourselves with Albert Einstein and learn and hear him and ask him questions," he says. "And he will answer us." (And when universal translation is available, he could even answer in the user's own language.)

This technology, of course, is several years down the road, but it will be built on concepts being developed today — like Deep Nostalgia and LiveStory — when and if it eventually comes to pass.

In the meantime, there's MyHeritage and, soon, D-ID's own consumer app that will help to demonstrate its technology in a different way. The app will arrive in the "coming weeks," D-ID says.

The MyHeritage LiveStory feature was announced today at the RootsTech family-history tech conference. It will work on desktop, mobile web and in the MyHeritage mobile app.

"LiveStory takes storytelling to the next level," said Gilad Japhet, founder and CEO of MyHeritage, in a statement about the launch. "With this latest viral feature, MyHeritage continues to lead the world of online family history in both vision and innovation. Our use of AI to breathe new life into historical photos is unique and is helping millions of people cultivate a renewed emotional connection with their ancestors and deceased loved ones. Genealogy is all about telling and preserving our family stories. We keep showing the world how fun and compelling genealogy can be."

D-ID, which was also co-founded by Sella Blondheim and Eliran Kuta, is a team of 32 and plans to double by hiring across positions in the U.S., U.K., Singapore and Israel.