Join My Newsletter
AI Image to Video Tutorial: Create Cinematic AI Films
AI Image to Video Tutorial: Step-by-Step Guide to Create Cinematic AI Films
In this AI image to video tutorial, you’ll learn how to generate cinematic films using powerful AI tools like Luma AI, MidJourney, and Stable Diffusion. These tools, when paired with the right techniques, can turn static images into dynamic, visually stunning videos. Throughout this guide, we’ll cover everything from setting the cinematic tone and enhancing mood to using keyword strategies that will elevate your AI-generated videos to a professional level.
Key Takeaways:
- Learn how to use AI tools to create AI video from text.
- Explore how to turn text prompts into AI-generated images & turn them into cinematic films.
- Master the process of adding keywords to emphasize the narrative and shape the mood of your videos.
- Discover advanced keyword techniques like Keyword Influence and Negative Power Keywords for even more control over your results.
- Learn how to upscale and finalize your images and turn them into videos using Luma AI.
- Then transform your images into stunning videos using Luma AI.
- Compare the results of text-to-image generation between MidJourney vs Stable Diffusion.
The Tools I Use
To create cinematic AI films, we rely on a combination of advanced AI tools that allow for precision and creative flexibility:
- Luma AI for video generation.
- MidJourney & Stable Diffusion for creating high-quality images using text-to-image prompts.
* As these techniques apply to both MidJourney and Stable Diffusion, you’ll find AI-generated image examples from both platforms in the video and throughout this blog post.
While these are the tools I personally use, the techniques I’m about to show you can work with virtually any AI filmmaking platform. In this tutorial, we will be using the image-to-video option, as this generally produces better results than text-to-video options on most current AI video platforms.
Step 1: Set the Cinematic & Artistic Tone
* We recommend watching the video for detailed guidance, starting at 02:58 when Step 1 begins.
To begin creating your AI film, it’s essential to set the right cinematic and artistic tone for your images. This first step involves carefully selecting keywords that define the style and aesthetic of your video. A small prompt can go a long way in shaping the overall feel of your video.
- Use small prompts like “low key photography” or cinematic film still, to set the artistic tone.
- Adjust the seed to maintain visual consistency across multiple frames.
*Prompt 01: lowkey photography, of an attractive, 30 year old, blonde woman
*Prompt 02: cinematic film still, of an attractive, 30 year old, blonde woman,
Step 2: Enhance the Mood with Power Keywords
* We recommend watching the video for detailed guidance, starting at 03:57 when Step 2 begins.
Once you’ve set the cinematic tone, the next step is to enhance the mood of your video by leveraging power keywords. These keywords carry more weight in a prompt and have a stronger influence on the final image. For example, adding a keyword like “dramatic lighting” can make a huge impact on the mood and atmosphere.
- Mood modifiers (e.g., “intense,” “gritty,” “dramatic”).
- Color components to adjust the overall tone and contrast of the image.
Step 3: Invoke Specific Emotions with Color Tones
* We recommend watching the video for detailed guidance, starting at 10:35 when Step 3 begins.
In this step, we focus on evoking specific emotions through color tones. The use of color is a powerful tool in filmmaking, and it can dramatically change how a scene is perceived.
- Warm Tones: Use colors like reds, oranges, and yellows to evoke feelings of warmth, comfort, and happiness.
- Cool Tones: Blues and greens can create a sense of detachment, sadness, or even calm and serenity.
- Dark Tones: Darker shades like deep blues, purples, or blacks often convey a sense of mystery, danger, or fear.
- Bright Tones: Vibrant and bright colors, such as yellows and whites, can emphasize joy, optimism, and positivity.
- Deep Colors: Adding deep and rich colors can increase intensity and suspense, perfect for creating a more dramatic or tense mood.
- Dull Colors: Muted, dull tones can bring a sense of seriousness, harsh reality, or even melancholy to an image.
Step 4: Adding Keywords to Emphasize the Narrative
* We recommend watching the video for detailed guidance, starting at 12:25 when Step 4 begins.
This is where your video truly comes to life. Emphasizing the narrative requires carefully chosen keywords that reflect the story you want to tell. Using these keywords strategically in the prompt can help guide the AI model to create visuals that fit the mood and theme of your video.
4.1 Keyword Influence
AI models prioritize certain keywords more than others based on how they are positioned in the prompt and the number of words used.
Example:
A short prompt like “intense gritty woman” will give more influence to “intense” and “gritty” as they share weight with fewer words. Adding more keywords will distribute that weight, which can dilute the effect of your most important keywords.
4.2 Prompt Weight
-
In Stable Diffusion, you can select a keyword and adjust the prompt weight by. For instance, if you’re creating an intense and gritty scene in a hospital corridor, you might set the keyword weight for “intense” to 1.4, making it more dominant in the prompt. This will give the image a much grittier and intense mood compared to a lower weight like 1.0, where “intense” would have less influence on the final result.
Instructions:
In Stable Diffusion, first select the keyword you want to emphasize, like “intense.” Then, hold the control key and press the up arrow to increase its weight, for example, to 1.4. This gives “intense” more prominence in the final image, making it grittier and more dramatic. If you want to reduce its influence, hold the control key and press the down arrow to lower the weight.
This technique allows you to:
- Maintain control over the mood, intensity, and atmosphere of the video.
- Ensure that the narrative is clear and that the most important elements are emphasized.
Another great techniques you can use is Negative Power Keywords.
* Prompt 01: cinematic film still,( intense gritty atmosphere:1.4), of an attractive, 30 year old, blonde woman, in an intense gritty hospital corridor
* Prompt 02 : same but with NK (underexposed:0.1)
4.3 The Negative Power Keyword and the Truth About Negative Keywords
Negative keywords are just as important as positive ones, especially when you want to control the elements the AI includes in the video. Instead of relying on generic negative keywords like “low resolution” or “bad quality,” which often have no effect, it’s better to use specific terms like “underexposed” to adjust the overall brightness or “vibrant” to tone down overly bright colors.
These negative power keywords allow you to:
- Remove unwanted elements from the image.
- Refine the final output for a more polished result.
For example, if you want a darker image, adding “underexposed” in the negative prompt box will reduce the brightness without sacrificing detail.
4.4 Incorporating Other Styles: Joyful, Dramatic, Mysterious, Depressing
Depending on the story you want to tell, you can use power keywords to create different styles, such as joyful, dramatic, mysterious, or depressing. For example, using “bright colors” and “festive decorations” creates a joyful atmosphere, while “gritty” and “intense” keywords emphasize a more dramatic tone.
Example Scenarios:
- Joyful: Add “glamorous photography” and “festive decorations” to create a vibrant, cheerful video.
- Dramatic: Use “dark colors” and “sinister” mood modifiers to enhance a dramatic, tension-filled scene.
- Mysterious: Choose words like “eerie” and “hazy” to build suspense and mystery.
- Depressing: Use “cool colors” and “gloomy” modifiers to evoke sadness or melancholy.
These keywords work in MidJourney and Stable Diffusion, ensuring consistent results across platforms.
Now that we know all this we can incorporate the Other Styles
*Prompt Top Left (Midjourney): glamour photography, of an attractive, 20 year old, mexican woman, wearing a crimson ruffled dress and slingback heels, dancing in a mexican town square with local people dancing, festive decorations, sunset, chromatic bokeh –style raw –ar 16:9 –stylize 0
*Prompt Top Right (Midjourney): horror, close-up, of a scholar of magic, in an enchanted ancient forest with gnarled trees, shallow depth of field –style raw –ar 16:9 –stylize 0
*Prompt Bottom Left (Stable Diffusion): candid film still, wide shot, eerie hazy atmosphere, of a 30 year old, blonde woman, wearing a tank top, with a confused expression, sitting on an old couch, in an abandoned (hazy:1.2) hall, shallow depth of field, dull colors
*Prompt Bottom Right (Stable Diffusion): cinematic film still, wide shot, of an attractive, 30 year old, blonde woman, with a determined expression, wearing a poncho, in a countryside, a single ancient gnarled bare tree in the far distance
Step 5: Prepare the Images for Luma AI (Upscaling & Inpainting)
* We recommend watching the video for detailed guidance, starting at 18:42 when Step 5 begins.
Once you’ve created your initial images in MidJourney or Stable Diffusion, the next step is to prepare them for video creation by upscaling and inpainting.
I always start with upscaling before inpainting the face because upscaling enhances the overall resolution of the image. If you were to inpaint first and then upscale, the upscaling process could cause the image to lose quality again. By upscaling first, you maintain the integrity of the entire image. After upscaling, I use inpainting to focus on smaller areas, like faces, where higher pixel density is needed. Since inpainting works on a small area with a lot of pixels, it guarantees high-quality results without affecting the resolution gained from upscaling.
Upscaling Instructions for Stable Diffusion’s Fooocus:
- Drag the image into the upscale box in Fooocus.
- Select the appropriate upscale option (e.g., 1.5 or 2x).
- Generate the upscaled image for better resolution.
Inpainting Instructions:
- After upscaling, use the inpaint option to refine any areas that need fixing.
- Paint over the parts you want to improve, such as the face or background details.
- Generate the inpainted image to finalize your preparation.
This step ensures that your images are of the highest quality before you convert them into video.
Step 6: Use Luma AI to Turn Cinematic Images into Video
* We recommend watching the video for detailed guidance, starting at 19:35 when Step 6 begins.
Now that your images are ready, it’s time to turn them into AI-generated videos using Luma AI.
To turn your cinematic images into a video using Luma AI, start by uploading your upscaled and inpainted images into Luma AI’s image-to-video option. In the prompt box, you can add specific camera actions like “camera zooms out” or “woman running through corridor” to bring dynamic motion into your video.
I typically use camera prompts to guide the AI and ensure smooth transitions throughout the video. If the movement becomes too exaggerated, you can disable the enhanced prompt feature to maintain control over the final video’s flow.
Once the video is generated, you can use frame interpolation or post-processing tools to enhance the final output.
Conclusion
By following this guide, you can create visually stunning AI-generated videos using tools like Luma AI, MidJourney, and Stable Diffusion. From setting the cinematic tone to refining the final video, each step offers creative freedom while ensuring a professional outcome.
If you found this tutorial helpful, subscribe to my YouTube channel for more AI filmmaking tutorials, and feel free to leave a comment or share your work with me!
Links Mentioned in the Video:
- Craft Cinematic Images with Our Prompt Toolkit: Fast & Easy
- The Free PDF files: https://digitalmagic.gumroad.com
- Rundiffusion – Fooocus in the cloud: https://bit.ly/rundiffusion-digitalmagic
- Get 15% DISCOUNT on your first month for Creators Club Promo Code: digitalmagic15
- Jump Into AI YouTube Channel: https://www.youtube.com/@JumpIntoAI/videos
- A link to the Github page for Fooocus: https://github.com/lllyasviel/Fooocus