Experimenting with Sora, OpenAI's video generator

This past week OpenAI finally unleashed Sora on the public. Nearly a year ago they made their video generator available to a handful of artists. I shared some examples and reflections back in March. Now everyone can use the thing.

Today I’ll share my impressions, along with some examples, walking you through using Sora.

I started small, based on a story a friend told about visiting Detroit for the first time and encountering cats on a street. I had already had DALL-E and Midjourney make some static images, like this:

Now I turned to Sora. If you haven’t used it, it’s not built into OpenAI’s ChatGPT suite of offerings yet. Instead, it’s a different website (https://sora.com/), albeit with similar design to ChatGPT’s, and you need to create an account from scratch. (This is where the problems occurred for the past few days, as the site kept crashing here, and OpenAI just blocked creation of new accounts.)

The screen will fill with video clips created by other people. A small, quiet text entry box appears at the bottom, like so:

Note the options. You can: upload an image or video to start with; pick some preset themes (stop motion, balloon world, etc); alter the aspect ratio; alter the resolution; select length (5-20 seconds); choose how many different videos will appear (1-4). There’s also a storyboard option, which lets you chain together a series of clips.

To start with, I used my friend in Detroit with cats prompt. It took about one minute to generate:

We now have a few seconds of a woman hanging out with cats. I’m not sure what signals “Detroit” but it is a city and it’s not not Detroit. Sora added a title: “Woman Befriends Alley Cats.”

Inspired, I turned to my professional work on higher education. I asked Sora to imagine a future university campus after climate change. Two clips appeared:

Not bad. I like the advanced technology and the camera motion. But they seemed to emphasize “future university” more than “climate change,” so I revised the language from “climate change” to “climate disaster.” The results were quite different:

Now the grounds are ruined and the skies are grey. “University of the future” seems a cruel title when it appears.

Next, I decided to get more imaginative and ambitious. My text prompt was one which has haunted my dreams since I was a kid: a cosmonaut on a decaying starship near a red giant star. Inputting that prompt, I extended the video length to 10 seconds. Sora chewed on it, then produced two videos. Here’s one, which it titled “Cosmonaut's Silent Stargaze”:

I liked some of this, but thought the cosmonaut was too active for my thinking and the “star” looked like a moon. So I revised it with a tweaked prompt: “A cosmonaut is thinking on a large, decaying starship. The starship is in orbit around a vast red giant star.” Sora dutifully revised, offering a new title (“Solitary Cosmic Contemplation”) and two new video clips:

I think the second one does a better job with the star, but don’t like the way the human just pops into existence.

One more example for today. Thinking of a classroom assignment, I asked Sora to depict a World War One battle on the eastern front. A pair of videos arrived, and I’ll share one:

I started thinking that it was interesting that Sora picked black and white, that there wasn’t a lot of human detail, and asked for revisions thereby:

Hm. Still no color, but at least there are more people. It looks more like WWII footage.

I can imagine assignments around this, asking students to get Sora to generate clips on certain topics, then the students determine how well the AI did.

---

Let’s step back a bit.

I exported each of these videos as mp4 files, which is easy to do. Sora also lets us download videos as animated gifs. The website keeps copies of each video in a user’s library. Here’s a snapshot of part of mine as of now:

Doing these various tests cost me Sora credits. I started with 1,000 and am now down to 350. Sora tells me a credit refresh arrives in mid-January, so I should be careful with my use for the next month.

Overall, Sora is easy to use. Text inputs generally yield plausible video outcomes, much like text to image generators do. Downloading is quick and easy. Storing copies of videos is useful.

That credit cap, though, is fierce. Clearly it’s aimed at nudging us to the $200/month tier.

More experiments to come. And I’ll have more to say about OpenAI’s December releases in following posts.