An update on Sora
New AI-generated videos, but still no public release
In February OpenAI announced they were working on an AI-powered video editor, Sora. It wasn’t available to the public or anyone, other than OpenAI staff and an unidentified red team. (My first post; my second)
Sora still isn’t available to consumers, but a group of video and filmmakers did get a chance to explore it. OpenAI blogged about the results this week, and I’d like to share them here, including published video samples, along with some speculation.
Overall, the results fascinate me. There’s a clear sense of objects arranged in a volume of space. There’s definitely a strong flavor of surrealism or fantasy.
Examples:
“air head” is a fun short about a person with an, er, unusual head:
While some visuals in the opening part struck me as odd (a human leg looked off), I found myself gradually assuming all but the balloon character were live actors in real sets, and wondering how they got the balloon to work in cramped environments.
The studio then released some reflections on Twitter/X.
“Somme Requiem” is an impressionistic meditation on WWI’s famous Christmas Truce, and is the longest clip of the lot, at over two minutes:
On the one hand I admired the use of depth of field, along with some attention to detail. On the other, there was more buildup to the event, rather than depiction of the thing itself.
“Adventurous Puppies” is a silent, very short (20 seconds) fantasy about heroic dogs:
This one did as promised.
Another one depicted a spaghetti monster eating Will Smith, a fun response to the now-classic Will Smith eating spaghetti AI clip:
As a fan of horror and surrealism, I enjoyed this and thought it succeeded in its mission, but know it might not be for everyone.
There’s an untitled video by Paul Trillo, depicting an imaginative fast flight through multiple venues:
This really impressed me by the sheer richness of visuals in a very short time.
Trillo also put several Sora results together into a video fantasizing about the Voyager spacecraft’s golden record:
“Sora First Impressions” feels like a demo reel from a marketer:
Some fine images and microvideo there.
An artist and musician named August Kamp made this short clip:
…then published “Worldweight,” a longer (2:18) video, just yesterday:
Both combine various scenes and objects in an abstract, almost Brothers Quay fashion.
Another designer, Josephine Miller, created a short clip of what looks like people wearing stained glass:
Don Allen Stevenson III created a surreal nature fantasy documentary:
Alexander Reben, OpenAI’s artist in residence, shared this video comparing his own sculptures with what Sora visualized for him, I think:
MIT’s Technology Review interviewed some of the creators.
So what do we make of these examples of AI-created video?
My first Sora post has a lot of ideas in its second half, and I’ll point you to those for starters, because I think they still hold.
Additionally… the artists’ examples above demonstrate creative uses of Sora across a range of styles and settings. There is a historical commemoration, creative fashion, sculpture, cute puppies, slow holds and blazingly fast point of view shifts.
There isn’t much storytelling. Instead some of the clips tend to the abstract or the miscellaneous - which isn’t a criticism, just an attempt to identify what I see. Few characters appear and develop, besides poor Will Smith and the balloon fellow. I’d like to see more experiments with narrative, and would be happy to try my hand at the same once I can access Sora.
Fantasy and surrealism appeared in the first set of Sora examples and we see more of the same here. Perhaps this will shape how we use text-to-video AI, viewing it as unrealistic and playful with reality.
I’m curious about how Hollywood will respond. Sam Altman has been showing Sora to studio leaders, but it looks like many in the film industry are skeptical. Chris O’Falt reports that people he spoke to were unhappy about several things: the lack of copyright on results; variable quality output, a real problem for continuity; having to rely on businesses; uncanny valley results.
[F]ew see a viable path for Gen AI video to make its way to the movies we watch. In my notes from the last few weeks, “I would never use AI in my final project” appears 15 times from conversations with animators, visual effects artists, post-production and virtual production supervisors, and filmmakers who use generative AI tools. These exchanges are usually on background, as using AI is currently the equivalent of showing up on set in a MAGA hat.
I wonder about people using Sora to explore and experiment, especially away from the public eye.
“the equivalent of showing up on set in a MAGA hat” - I’m working on a post about shifting cultural attitudes to AI. More on this coming up.
Somme requiem is NOT Sora generated. I got suspicious with the horrible look of the human faces plus I saw no one else pointing to that particular one as a sora example.
Tail credits claim it was made using Runway.
I can't find the quote/link at the moment... but I read that someone in the video game industry (maybe someone at NVIDIA?) predicted that within 10 years all video game graphics will be produced, on the fly, by AI within the game. That is, no pre-designed graphic assets. There would still be programmatic assets (maps, stats, etc.), but the "thing that makes pretty stuff on my screen" will be an AI generating it (nearly) instantly, rather than having pre-set 3D models, sprites, textures, etc. When that happens, we're not going to be far off from being able to "design" film/video/TV stuff without any human interaction beyond getting any known IP rights.
After that happens? We'll have movies where there is one credit "Prompts created by John J J Smith," and then whichever AI tools were used.