This video’s A.I. prompt: “The camera is behind a man. The man is shirtless, wearing a green cloth around his waist. He is barefoot. With a fiery object in each hand, he creates wide circular motions. A calm sea is in the background. The atmosphere is mesmerizing, with the fire dance.”

Meta Unveils Instant A.I. Video Generator That Adds Sounds

The tech giant is among the many companies building technology that could remake Hollywood — or help spread disinformation.

by · NY Times

In February, the artificial intelligence start-up OpenAI unveiled technology called Sora that let people generate photorealistic videos — like woolly mammoths trotting through a snowy meadow — simply by typing a sentence into a box on a computer screen.

Because of concerns over how the technology might be misused, and perhaps the high cost of operating the technology, OpenAI has not yet released Sora beyond a small group of testers. But other companies are racing to release similar technology.

On Friday, the tech giant Meta unveiled a set of A.I. tools, called Meta Movie Gen, for automatically generating videos, instantly editing them and synchronizing them with A.I.-generated sound effects, ambient noise and background music.

This is the prompt used for the audio: “Rustling leaves and snapping twigs, with an orchestral music track.”

“Video is nothing without audio,” said Ahmad Al-Dahle, Meta’s vice president of generative A.I., said in an interview.

Given a short text description of an image, one tool creates a video. Then, given another description of some sounds, a second tool adds the audio.

A demonstration included short videos — created in minutes — of a man in a poncho standing over a waterfall, a snake slithering through a forest and a person riding an all-terrain vehicle across the desert. Each included music as well as sound effects.

The new system also let people upload photos of themselves and instantly weave these images to moving videos.

It generates 16-frame-per-second videos that last for up to 16 seconds. In some cases, these videos are flawed. During one demonstration for The New York Times, when asked to generate a video of a dog in a park talking on a cellphone, it mistakenly grafted a human hand onto the phone.

This video’s A.I. prompt: “A fluffy koala bear surfs. It has a gray and white coat and a round nose. The surfboard is yellow. The koala bear is holding onto the surfboard with its paws. The koala bear gets on the surfboard, its facial expression is focused. The sun is shining.”
CreditCredit...Video by Meta

Meta, the owner of Facebook and Instagram, sees the technology as a way to accelerate the work of Hollywood moviemakers and online creators. Like OpenAI, it has started testing the technology with a small group of professionals.

(The Times sued OpenAI and its partner, Microsoft, in December, claiming copyright infringement of news content related to A.I. systems.)

The two companies are among many developing this kind of tool; others include start-ups like Runway and Pika and tech giants like Google and Microsoft. Though many believe the technology could speed the work of seasoned moviemakers, it could also replace less experienced digital artists.

Others experts worry that the technology could become a quick and inexpensive way of creating online disinformation, making it even harder to tell what’s real on the internet.

While OpenAI and other companies have been reluctant to release some A.I. technologies amid concerns about disinformation and other potential risks, Meta has been quicker to share them freely, arguing that the risks are not as great as they may seem.

Mr. Al-Dahle said Meta would tag videos produced by the system with watermarks that identified them as being generated by A.I. But such tags can be removed. (The Times added “Generated by A.I.” watermarks to the videos with this article.)

He also said the company was still testing the technology to better understand what the risks might be and how they could be mitigated.

Meta’s demonstration did not include spoken words. But many companies, including OpenAI, are developing A.I. technologies that can instantly recreate human voices.

Explore Our Coverage of Artificial Intelligence

News and Analysis

  • Cerebras, a chip company with bold ambitions to take on Nvidia, filed for an I.P.O., taking a key step toward being among the first A.I. companies to go public since the release of ChatGPT.
  • Gov. Gavin Newsom vetoed a California A.I. safety bill, blocking the most ambitious proposal in the nation aimed at curtailing the growth of the technology.
  • At an event in Silicon Valley, Meta exhibited a range of products, including new smart glasses, meant to blend the real world and virtual reality with a healthy dose of A.I.

The Age of A.I.