Meta announces new AI model that can generate video with sound

Facebook owner Meta announced on Friday it h built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leing media generation startups like OpenAI and ElevenLabs. Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people's real photos to depict m performing actions like painting on a canvas.

Movie Gen also can generate background music and sound effects synced to content of videos, Meta said in a blog post, and use tool to edit existing videos. In one such video, Meta h tool insert pom-poms into hands of a man running by himself in desert, while in anor it changed a parking lot where a man was skateboarding from dry ground into one covered by a splashing puddle.

Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that model performs favorably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling.

announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts.

Technologists in entertainment industry are eager to use such tools to enhance and expedite filmmaking, while ors worry about embracing systems that appear to have been trained on copyright works without permission. Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around world, including in U.S., Pakistan , India and Indonesia.

Meta spokespeople said company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers risks individually for each model. y declined to comment on Meta's assessment for Movie Gen specifically. Inste, y said, Meta was working directly with entertainment community and or content creators on uses of Movie Gen and would incorporate it into Meta's own products sometime next year.

According to blog post and a research paper about tool released by Meta, company used a mix of licensed and publicly available datasets to build Movie Gen.

OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although no deals have been reported to have come out of those talks yet. Anxieties over company's approach increased in May when actor Scarlett Johansson accused ChatGPT maker of imitating her voice without permission for its chatbot.

Lions Gate Entertainment, company behind " Hunger Games" and "Twilight," announced in September that it was giving AI startup Runway access to its film and television library to train an AI model. In return, it said, studio and its filmmakers can use model to augment ir work.