Sora's "Air Head": plenty of human post-processing

OpenAI recently presented several videos made by artists who were allowed to test the Sora video AI. One of the videos was particularly impressive: "Air Head". And what was already suspected when it was released is now certain: the video has been heavily edited, especially in human terms. An entire team worked on the effects and filtering out artifacts.

In "Air Head", a yellow balloon floats above the body of a man. However, the balloon was obviously not even yellow originally, but red. And what's more, it changed color various times during the attempts to generate one or more consistent videos with Sora. This is according to Patrick Cederburg from shy kid, who was responsible for the post-production of "Air Head", while there was also a producer in Sidney Leeder and a director in Walter Woodman. Now you might wonder why so many people worked on an AI-generated video? Did they write the prompt together? Hardly. Accordingly, numerous questions about the video arose shortly after its publication. For example, it is longer than OpenAI states how long the AI videos can be, it has sound, which Sora cannot do yet, and much more. OpenAI said that the published videos were created with Sora – there was no post-production or further information.

Classic post-production: from artifacts to resolution

Cederburg tells industry magazine FXGuide that it was really interesting to be able to play with Sora: “It’s a very, very powerful tool that we’re already dreaming up all the ways it can slot into our existing process. But I think with any generative AI tool; control is still the thing that is the most desirable and also the most elusive at this point.” To generate videos with Sora, you can really only enter one prompt, there is no help to create multiple videos that build on each other. They therefore took a hyper-descriptive approach, inserting every detail, such as clothing, into new prompts to create new scenes that were then edited together. However, a lot of post-production was required.

As an example, Cederburg cites a banana, of which the video AI does not have a direct image or a kind of stock photo, but only an idea: yellow, curved, dark ends. However, this results in a new or different-looking banana every time. As a result, the balloon not only came out different in color, it even had a scary face that had to be removed afterwards. "Air Head' only uses footage generated by SORA, but most of it has been graded, edited and stabilized as well as upscaled or high-resolution." The image sections were also changed, although Sora offers certain options for this, camera settings and angles were difficult to enter. However, the filmmaker considers this to be essential for filmmakers.

Cederburg and his colleagues are obviously also testing other AI video tools. He says they all had similar problems, for example with the camera settings, whereby he emphasizes Rundway AI, which generated worse and shorter videos than Sora. For "Air Head", they ended up with a lot of material that had to be edited and put together. There were hundreds of generated videos, each 10 to 20 seconds long. He assumes a ratio of 300:1. And even the speed had to be adjusted - most of the clips that Sora generated looked like they were shot in slow motion. The voice of the person with the balloon head, called Sonny, is Cederburg himself. Everything is classically dubbed, without any AI.

Nevertheless, Cederburg still thinks Sora is impressive and obviously enjoyed the work so much that they are already working on a second part.

(emw)