THIS IS (NOT) A TURTLE

Experimenting with artificial documentaries

Hello again /promptcollective subscribers,

Today, I will share something with you that I have been tinkering with for the last week or so whenever I had a minute… It is a video I created, and it is what I choose to call an “artificial documentary.” This newsletter will offer more on what that is, why I made it, and what I learned.

Created with Midjourney

It all started with this “photo” of a “sea turtle” that I created - or rather even one stept before that, with me thinking about the aesthetic of most AI-generated images:

Screenshot of the Midjourney explore page

When I see AI-generated images and videos, there are often a lot of colors, a lot of photographic realism, a lot of wild surreal stuff happening, and a lot of young sexy women. Most of that is probably down to the target group, mostly working with these tools.

So I experimented with creating a rough black and white, print-inspired style. And because I had recently finished reading “The People in the Trees” by Hanya Yanagihara (huge recommendation by the way) I had turtles on my mind…

While creating the turtle I kept thinking about AI aesthetics in images and videos and how most examples out there are animation or science fiction or fantasy. This brought me to the idea of creating an artificial documentary, a film that uses the aesthetics of a documentary but is completely AI-generated.

Before I tell you a little about how I made it and some reflections here is my little piece:

The process

Now let me tell you a little bit about my process:

I started by using Claude 3.5 Sonnet from Anthropic to generate the text. Here is my prompt:

As you can see I did give some specifics, but left the storyline to the AI. The generated text was actually really nice right off the bat. Interestingly, the mentioned Wayuu people and their reverence of the sea turtle as well as La Guajira beach are legit, no hallucinations there. So the facts are checked, just of course my protagonist is purely a fantasy. The next step was generating the voice with elevenlabs.

Then I started playing with Midjourney to generate different Images and Runway to generate video clips. In generating video clips I really enjoyed the new possibilities of defining both start and end frames to control the output. The “lipsync” was also done in Runway. The music was done with Udio, a tool that blows my mind every time.

I put everything together in Adobe Premiere and also did some very basic color correction to make the video look more like documentary shots.

My analysis

I really enjoyed playing around with the tools as it had been a while:

Paw-some Warfare

With new models and tools being released constantly, I feel the only way to reality-check the hype is to get hands-on and explore the current state of generative AI. For me, some elements like the video generation from images with the newest models, the image generation and the level of control have really come a very long way and are production-ready. One caveat though is that working with generative AI still means a lot of looking at progress bars:

The lipsync functionality is still quite rough and the generated talking heads are not super convincing, especially as the backgrounds are completely static. I will try experimenting with Heygen and D-ID and see if those deliver better results.

Browsing through the voices on Elevenlabs it was clear, that the most common use cases are either narration or speak and not just speaking. But I found a voice that I was rather happy with and after playing a bit with the settings the generation was super quick and the result quite convincing.

Will artificial docs be a thing?

Using generative AI to tell stories is happening all around and at an accelerating pace. As I mentioned in the intro I feel, that a lot if it defaults to certain genres and aesthetics, leaving creative potential untouched. The same way that “People in the trees” used a fictional memoir with footnotes and an overall factual style to create an eerie feeling, this can be applied for AI-generated films and content in general. I am really interested in this genre of “artificial docs”, that blur the lines between fiction and fact and I hope that there will be a lot of experimentation with these types of narratives. In literature these types of genre bending experiments are quite popular and of course we have also seen some amazing mocumentaries. I hope that a lot of creatives will use generative AI tools to explore these aesthetics and possibilities.

What do you think?

Who Are We?

The /promptcollective was founded by Jes Brandhøj (Denmark) and Hannes Jakobsen (Germany). We're on the lookout for like-minded enthusiasts. If you're passionate about the AI-creative nexus, reach out!