AI Lab Session | Video Generation with Runway, Act-one
Thank you for attending the AI Lab session! Here, you’ll find all the topics we covered.
📹 Watch the recording here:
⭐️ We’d love to hear your thoughts.
Please take a moment to fill out the form below and share your ideas for future topics.
Links to the tools we used:
Learning Goals:
After this session, you will master everything needed to independently create a high-quality AI-generated video, from concept to final product.
Writing a Script: Learn how to create an effective script for an AI video.
Recording a Video: Understand how to record an AI video template for Runway.
Creating Characters: Design unique characters in MidJourney for your video.
Voice Customization: Master voice adjustments using ElevenLabs.
B-Roll Animation: Add extra transition shots with B-roll animations in Hailuo.
After this tutorial you will have all the information you need to create a video like this:
Step 1: Brainstorm Ideas & Create a Script
We used ChatGPT for scripting and brainstorming
Brainstorming Ideas:
Use ChatGPT to generate creative ideas with specific prompts. Focus on storylines that feature characters combining humor, warmth, and festive elements. Ensure the storyline has a dialogue between characters and is well-developed.
Prompt Example:
We want to create a Christmas-themed video featuring a dialogue among three characters. The video should convey a warm, festive message and could include humor, heartfelt moments, or a creative twist. Please provide 3 unique and engaging ideas for the storyline.
Creating a Script:
It is important that it is suitable for portrait view and with a dialogue of people talking to each other.
Prompt Example:
Prompt: Create a detailed script for [specify the idea] with a dialogue-centric scene between the characters. Include rich dialogue, a minimalistic setting, and subtle actions to convey mood and depth.
The scene should focus on character interaction and emotions, with little movement, making it suitable for filming in portrait view.
Step 2: Create a storyboard
We used Midjourney for creating images
Engaging Visuals: Brings the story to life better than rough sketches.
Aligned Vision: Ensures everyone understands the creative direction.
Pitch-Ready: Polished visuals are ideal for presentations.
Time-Efficient: Saves time while enhancing workflow.
Step 3: Create a Video Template
Record template videos for later use in Runway and for voiceover.
Key Tips:
Ensure a clear and steady frame while recording.
Minimize head movements.
Avoid blinking too often for smoother animation in later steps.
Step 4: Creating Characters
We used Midjourney for creating images
Use MidJourney to create unique character designs. These custom designs provide a distinct look compared to Runway templates.
Tips for Best Results:
Characters should have realistic, human-like features. Misaligned facial features can cause animation issues.
Use a portrait view with the character facing the camera.
Prompt Example:
Create a Disney/Pixar-style portrait of Santa Claus sitting comfortably in his cozy home. The portrait focuses on Santa from the chest up, highlighting his cheerful expression, round glasses, and traditional red hat with white trim. Behind him, a warm and softly lit background shows hints of a fireplace and a decorated Christmas tree, adding to the festive ambiance.
For Animal Characters:
Add “anthropomorphic style” to the prompt to make animal characters appear more human-like.
Use the Moodboard Option in MidJourney to maintain consistency across character designs.
Step 4: Using Act-One to Create the Main Video
We used Runway Act-one to create the main video
Upload your template video and character portrait to Runway Video Gen-3-Alpha and select Act-One.
Tips:
If the template video has too much movement, zoom in slightly for better results.
If the character portrait does not work, zoom in or try a different design.
Step 5: Changing the Voice
To change the voices we used Elevenlabs
Steps:
Go to voice changer
Select a preferred voice using the voice-changer feature.
Ensure the new voice fits the tone and enhances the video’s quality.
Step 6: Adding B-Roll Scenes
We used Hailuo to create the B-roll scenes
Generate Visuals in MidJourney:
Describe the desired shots to create consistent styles using the same mood board from character creation.
Pro Tip: Upload your script to ChatGPT and ask it to generate multiple prompts describing your scenes.
Example: Write five detailed prompts based on this script, specifying the setting, actions, and emotional atmosphere.
Use Hailuo to bring B-roll, intro, and outro images to life:
1. Upload your images.
2. Add a prompt describing the desired movement.
3. Define the main subject in the first frame and describe their motion or transformation.
Precise Prompt Formula= Main Subject in the first frame + Motion/Change +Camera Movement + Aesthetic Atmosphere
Example Prompt:
A cozy, sunlit workshop with a snow globe on a wooden table. Inside the globe, Santa rides a sleigh pulled by reindeer in a sparkling snowy scene. Surrounding the globe are hand-crafted toys, festive decorations, and warm golden light streaming through a frosty window.
Step 7: Finishing Touches
Add Custom Background Music:
Use Suno to generate music that matches the desired style.
Example: Joyful Christmas music with soft strings and bells. Warm and festive.
Enable the Instrumental option unless vocals are desired.
Final Edits:
Use a video editing software to seamlessly combine videos.
Add sound effects using ElevenLabs.
Design an engaging intro and outro banner to complete the project.
We hope you now have the knowledge to bring your creative ideas to life with AI videos