Writing the perfect AI prompts

neural frames offers tons of features to create stunning AI animations and AI music videos, but one single aspect of the video generation stands out in its significance: The prompting.

The prompt is at the heart of the current generative AI excitement - we can finally control algorithms with human language 🥳. But with it comes a new daunting realization: We have to control algorithms with human language 🤯.

Good prompting could very well be the major differentiator between a beautiful and a disappointing project. We do offer the "Pimp my Prompt" button, which sends your prompt to another AI that tries to optimize it for you, but you are the director, and therefore have to take control over your prompting skills.

Prompting is a very personal task and there is no scientific rule to it - at least there has not been one developed so far. So we thought it could be helpful to ask some of our power users what tips they can give about prompting. Let's go!

Update: We have a personalized AI assistant to improve your AI prompts. You can find the Stable Diffusion Prompt Generator here!

Tichi Techa

Starting with Music:

My process begins with the music. It's not just the background; it's the compass that guides me. Whether it's a soft melody or a dynamic beat, the music influences the level of movement in my neural frame creations. As a musician I have started to tailor my sound with Neural frames in mind as the end goal.

Visual Style - There Are No Rules:

In neural framing, your imagination is the only limit. I blend my personal style with the flexibility of the medium, often using custom models. This allows me to focus less on the style and more on the actions and events I want to depict. When creating custom models I personally take great care in not using copyrighted imagery. I create my own images with tools such as Dall E or find images that are not under copyright such as archives or similar library's.

Embracing a Personal Aesthetic:

My preference leans towards a vintage look, akin to old cameras or classic video games. I achieve this by emphasizing high flicker and selecting prompts that degrade image quality, a deliberate choice to create a unique visual texture. This is my style, but clear, high-definition images are just as achievable for those who prefer them.

The Art of Negative Prompting:

Negative prompts are vital. They're not just about avoiding unwanted elements but about sharpening the focus on your desired theme. For a black and white scene, I'd counterbalance with vibrant colors in the negative prompts to enhance the monochromatic effect.

Experimenting with Settings:

When working with a new model or prompt, I begin with three short test runs, each lasting 15-30 seconds, across the basic settings. This helps me understand how the model reacts to different levels of smoothness and flicker. Then, I refine my approach in pro mode, continually adjusting as the render evolves. The key is to keep adding and tweaking prompts - the more you put in, the more you'll like the outcome, compared to setting a long prompt and letting it run unchecked.

Deep Surface

When I start a project, everything I thought I wanted to do is not even a memory. The screen is like an empty page or a blank canvas. So I stare at the screen for a little while, and sometimes give a few words as a prompt to see what image I get, or throw every saved prompt with weird words and settings at the robot. I’m always amazed at how the process gets inspired by what I watch being made.

Here are the six things I believe can help you succeed and exceed what you thought was possible:

• Embrace Experimentation: Learn through doing. Try new stuff. You can always back up and do it again. Refer Back to Older Projects: Refresh your memory on what did what. Watch your videos. You know you want to anyways.
• Evolution: Rethink your approach to AI art in general. Is it a tool, a crutch, or a toy? There is no right answer.
• Know the Basics: Understand the language/syntax and format of the medium and delivery platforms.
• Learn How to Drive the Camera: Understand it’s not just a static image but a world you can move through.
• Embrace the Strengths & the Flaws: The robot will do things we don’t want... or do we??? Just because you didn’t like it for this video, does not mean you won’t want to do that very thing the next time you set out to make your masterpiece.

After considering these points, remember the importance of understanding the basics of video/image creation – like 25fps meaning 25 pictures in a row. Also, knowing how to communicate effectively with the AI is crucial: what does it expect, and what can you expect from your input? In short, I try everything, research constantly, and never feel like I know what I am doing.

We are all newbies in this game; it hasn’t been around long enough for it to be solved.

First syndrome

I like rendering things that are very human, but I also love a vintage look. So, I think my prompting is best when it's literal, but with an emphasis on describing decay, noise, and some distortion. This means adding things like, "bleached film" or "polaroid look" can help. A great starting point for me would be, "bowl of cereal in a mid-century kitchen, bleached film, polaroid aesthetic, filmed in 1976" and then I can let the AI drive.

If I'm looking for more, I'd want to think about specifics in the kitchen and add more decay references - so, "bowl of cereal in mid-century kitchen, with a retro style clock hanging in the background with checkerboard floors, bleached film, polaroid aesthetic, filmed in 1976 on toy camera", and continue to make changes as needed.

LEGIO_X

My artistic approach with Neural Frames is different than most other users as I like to generate tons of footage using long renders, and then edit it down later - so there's less emphasis on micro-managing each particular "scene" - this is just a stylistic choice however - I have seen some of the prompt tips submitted by other artists and its all very useful advice covering a wide range of approaches.

1) Don't use the Negative Prompt - blank it out!

2) Embrace the esoteric and absurd - don't be afraid to prompt for scenarios that don't make any sense

3) Simpler is sometimes better - some of my best results have been from prompts that were as short as two words

the take-away here should be that there are no set rules - you can tear the prompting process down to a zero-point and then build it back with pieces from the wreckage - and in doing this you might land on a style that's completely your own.

Stormcrow

Think of prompting like sketching out a scene:

- Broad overview
- General subject
- Specifics of subject
- Finishing touches

First describe what type of “shot” the “camera” is viewing. Then who’s in the shot, what’s in the shot. What are they wearing? What colors do you see? Finally describe the art style or medium that it’s “captured” on.

S. Chimaera

CURATE A TOOLBOX OF WORDS

1. Find the word for every pattern, color, texture, visible form, shape, style, perspective, etc. that fascinates you. Ask an AI to help you come up with these descriptors if necessary.
2. Repeat with every object, animal, plant, feature, etc. that interests you.
3. Same thing for actions, scenarios, situations, settings.
4. Now spreadsheet all of these into distinct categories and combine them, at random or with intent. This is a great way to produce personally engaging concepts that are prompt-ready. It’s also a great help in situations where you might feel stuck!
5. It is extremely useful to have the building blocks of your personal universe organized in one modular space.

Scott Haynes

I start with an overall goal in mind. Then, I try to break down the different components of the imaginary image I'm envisioning.

Begin by defining the object or subject of the image, followed by the action they are undertaking, and then add the environment or background.

Next, ensure the inclusion of the type of medium you want to see, whether it's a photograph, digital illustration, painting, etc.

Afterward, define the styles. Instead of using artist names, I prefer using specific art styles. To discover these styles, I look up a list on Wikipedia and select ones I like, given my lack of formal art training.

The last consideration is adding elements like album cover art or a cinematic feel, action shot, low show, high shot depending on the desired outcome. In my experience, the most crucial aspect is trial and error.

Don't hesitate to experiment with random words and combinations to see what works. Once you find something you like, fine-tune it by removing, adding, or tweaking the prompts.

Now that you've established the initial prompt, I prefer to keep the entire structure the same for subsequent prompts but switch out the objects. This approach maintains the style you've worked to achieve.

Twelv

You can change your prompt mid-way through. Meaning you can stop the render before that box is complete and add to or adjust the prompt, then continue the render of that box. Interrupt the render and adjust the prompt then contine from that point, you can do this as many times as you want rather than restarting the entire box over to get changes halfway through.