Author | Yang Ruidian
Edit | Desert Shadow
Imagine what kind of sparks bloom when your script meets the magic of AI.
On September 27, Maoyan Entertainment released the first dynamic storyboard AI generation tool for long script analysis, "Magic Brush Ma Liang". It is understood that "Maleonn" can carry out one-click intelligent analysis, intelligent character creation, intelligent storyboard creation, and intelligent line reading of the script uploaded by users, presenting the content of the script in an all-round audio-visual way.
In the traditional creative model, storyboarding often takes a long time to create, ranging from weeks to months. According to Maoyan Entertainment, "Magic Brush Ma Liang" can generate dynamic storyboards with one click in a short time, greatly shortening the film and television creation cycle. At the same time, its intelligent character creation function can quickly write character biographies and generate AI portraits.
"Maleon" provides users with flexible options, allowing users to choose different visual styles according to the content of the script, including 8 genres such as national style costumes, suspenseful crimes, future sci-fi, and 3D animation.
When using it, users only need to upload a script in .doc or .txt format, and "Maleon" can quickly generate a coherent dynamic storyboard, so that the script can be completed with one click. Users can fine-tune the details at any time to ensure that the final look is exactly what they want.
At present, the dynamic storyboard is still in beta, and its functions such as "partial redrawing" and "image to video" have not yet been opened.
In actual use, can the "magic pen Ma Liang" be as good as the introduction? With this question, the editor of Zhidong applied for the internal test experience of "Magic Pen Ma Liang".
Experience address: https://shenbi.maoyan.com/create
1. Little expert in plot refining: "Magic Pen Ma Liang" dismantles the story in 1 minute, and there are shortcomings in character recognition
According to Maoyan Entertainment, "Magic Pen Ma Liang" can conduct AI intelligent analysis of the content of the script. According to the content uploaded by users, "Magic Brush Ma Liang" can carry out expert-level plot refinement, deeply understand the core of the story, and AI generates dynamic storyboards.
▲ "Magic Pen Ma Liang" can intelligently analyze the script (Source: China Film News)
With a strong curiosity about the effect of the intelligent analysis of "Maleonn", the editor decisively took out one of his short stories and started the experience journey. Surprisingly, in less than 1 minute, "Ma Liang" completed the intelligent analysis of the original content, refined the plot, and generated a dynamic storyboard.
In order to further explore the ability of "Magic Brush Maleon" to analyze different types of stories, the editor immediately uploaded the classic mythological story "Magic Brush Maleon" and the novel "The Name of Rose" with the same name as the product. Although the prompt shows that the script analysis takes 5 minutes, the initial analysis process of the two scripts still does not take 1 minute.
After the start of creation, the short story "The Magic Brush of Ma Liang" was disassembled into 3 scenes and 68 sub-scenes, and the long story "The Story of the Rose" was disassembled into 128 scenes and hundreds of sub-scenes.
When looking at several dynamic storyboards generated by "Maleon", I noticed that this AI tool can grasp the main context of the story and grasp the details of the content more accurately.
For example, in the story of "Maleon", Maleonn draws birds on the sand with a tree branch, and the scenes and details generated by "Maleon" are consistent with the original text.
However, although "Magic Brush Ma Liang" is generally more accurate in refining the plot and accurately grasping the details, after the editor's observation, it is found that it still has room for improvement in the face of more complex plot trends and intricate character relationships.
This may be due to the fact that in some cases, the AI does not have a sufficient understanding of the more subtle expressions in the script or the specific cultural context, which may lead to misunderstanding some key parts of the script, and ultimately causing the generated storyboard to deviate from the original intent of the script in some images.
For example, in the editor's work, the main character in the first storyboard of the first scene is supposed to be a woman, but the AI-generated image is a male character. When the camera pans to the third shot, the figure changes back to female.
This inconsistency shows that there is still a lot of room for improvement in the accurate identification and deep understanding of the character identity, and the technical optimization may be carried out from strengthening the learning of the character characteristics and improving the logical judgment ability.
Fortunately, this recognition error can be corrected by the editorial storyboard of "Maleonn". By tweaking the prompts on the screen, we were able to flexibly adjust the settings of the storyboard and scene to make the story more in line with expectations. Editing prompts can be done manually or generated using AI.
In addition to the occasional confusion of the gender of the characters, Maoyan's "magic pen Ma Liang" sometimes has an inaccurate grasp of the changes in the age of the characters when dealing with the script.
For example, when making the storyboard of "Maleon", the generated images showed inconsistent age of the characters. The previous shot was still a high-spirited young man Ma Liang, and the next shot changed back to his innocent childhood.
This subtle error requires the creator to check the character's characteristics and carefully adjust the details of the picture in post-production to ensure the coherence of the story.
Overall, "Magic Brush Ma Liang" can quickly sort out the story outline and key plots according to the script uploaded by users, laying a good foundation for further film and television creation.
However, in order to more accurately capture the deep meaning and core points of the script, there is still room for improvement in lexical analysis, syntactic analysis and semantic understanding.
Second, the character images are all captured, but the consistency of the dynamic storyboard needs to be adjusted
After the intelligent analysis of the script, "Ma Liang" will quickly write a character biography and generate an AI portrait according to the content of the script.
According to reports, this function can restore the voice of the character in the creator's mind, so that the character image is three-dimensional and full, and it is clear at a glance. Users can adjust the details of the characters in real-time according to their own understanding and the specific needs of the story to ensure that the image and aesthetic qualities of each character can be accurately set according to the script.
▲ "Magic Brush Ma Liang" can intelligently customize the character (Source: China Film News)
After the actual operation, the editor found that after uploading the content of the script, "Magic Brush Ma Liang" can indeed generate a series of AI portraits that match the characters in the script according to the text information.
Almost every character in the novel has been captured by "Maleonn". Hundreds of characters that appear in The Name of the Rose have been generated, and the image below shows just a few of them.
Maleonn goes a step further than other similar tools in character building, allowing users to refine the character's image by adjusting the character's descriptors.
If the user already has a ready-made character concept map in their hands, they can also directly upload the character mat map, so that the AI can use it as a reference to create a character visual image that is more in line with the user's imagination.
However, although "Maleon" can comprehensively generate AI intelligent characters in storyboards, it still has certain limitations in the depth of character portrayal.
At present, most AI-generated characters rely on data-driven pattern recognition, and cannot fully capture the inner emotions and story background of the characters, so it is difficult to create characters with unique charm that can be deeply remembered and loved by the audience.
Although "Maleonn" can provide a complete visual framework for a character, the generated character image is relatively flat, and there is no way to make the character really "live".
In addition, the dynamic storyboard also has shortcomings in character consistency. Between different shots, there may be situations where the characters look different.
Taking the editor's original script as an example, the heroine looks different in different storyboards.
To ensure a consistent character image, creators may need to do additional editing work, such as readjusting the prompts, or manually selecting and adjusting AI-generated images to ensure that the characters maintain a consistent look and characteristics across the storyboards.
3. "Magic Brush Ma Liang" can be animated storyboard in seconds: fun to change the scene, props can be added as you like
After the content is parsed and the characters are generated, it is time to enter the storyboard creation interface. Compared with the single-function general model that only provides "Wensheng Diagram", "Magic Brush Maleonn" can generate audible and visible dynamic storyboards with one click through intelligent analysis, and can intelligently divide the number of scenes, design the lens language, reduce the manpower and time cost of storyboard production, and improve the user's creative efficiency.
▲ "Magic Brush Ma Liang" can intelligently split the lens (Source: China Film News)
"Maleon" will automatically divide the scenes and storyboards, providing creators with a basic storyboard framework. If adjustments are needed, such as adjusting the angle of view of a shot from a long shot to a close view, creators can directly modify the storyboard and session configuration of a single video.
What's even more convenient is that, in addition to manual editing, creators can also choose to let AI assist in these modifications.
What's more, interestingly, you can also modify the configuration of the scene and add scenes or props. For example, the editor added the prop "red wine glass" to a certain storyboard.
After the session configuration is applied, add the corresponding information to the prompt of the storyboard configuration to regenerate the video screen, and the added props will appear in the newly generated screen.
As shown in the picture below, the heroine has an extra wine glass in her hand.
It should be noted that the prompt words written after the application scenario props need to be referenced with special symbols, such as @character, #场景, and & props.
However, each creator has their own unique creative intentions and artistic pursuits, and it is difficult for AI to truly understand these deeper intentions. "Maleonn" can only create storyboards based on the prompts of the text, and cannot fully comprehend the emotions and atmosphere that the creator wants to convey through the storyboards.
Fourth, the four major problems of intelligent audio need to be solved: flat emotion, inconsistent timbre, poor adaptation, and abrupt transition
In addition to the video footage, the "Magic Brush Maleon" dynamic storyboard can also automatically generate character voiceovers simultaneously through the automatically generated storyboard prompts. Users can also intelligently add background music according to the storyline, making the story richer content, more emotional, and more specific picture information.
▲ "Magic Brush Ma Liang" can intelligently match audio (Source: China Film News)
Music and sound play a crucial role in the expression of emotions and the creation of atmosphere in film and television works. The intelligent line reading function of "Maleonn" can automatically generate character dubbing, and can add background music according to the storyline, so that the content of the script can be presented in an audio-visual form.
However, in terms of audio generation, the "Magic Pen Maleon" dynamic storyboard AI has the following shortcomings.
1. The voice is more mechanical and has no emotion. Although "Maleonn" can recite lines according to the text, it lacks in the delicacy and layering of emotions. Through their own experience, understanding and performance skills, actors can express the complex emotions in the script, such as joys, sorrows, joys and sorrows, etc., in a more delicate and vivid way. However, it may be difficult for AI to fully capture these subtle changes in emotion, resulting in the recitation sounding relatively flat and unable to make the audience deeply feel the inner emotional fluctuations of the characters.
2. There is a situation where the dubbing timbre is not suitable for the character. For example, in the animated story of "Maleon", the voice of the white-bearded old man is a young man's voice, which is obviously not in line with the character. From the small to the big, in some special script scenes, there may be some lines that require special processing, such as dialects, accents, special language expressions, etc. It may be difficult for AI to accurately imitate these special situations, which can affect the perfect presentation of the content of the script.
3. The AI-generated background music is not well adapted to the story. For example, the music accompanying the story of "Maleon" is a piano piece. Due to a lack of understanding of the cultural connotation, the AI may choose a background music that does not match the style, destroying the overall style and artistic conception of the story.
4. The musical transition and connection are unnatural. In the process of switching between different scenes and developing the plot of the story, the background music needs to be naturally transitioned and connected to maintain the coherence and integrity of the story. However, through experiments, the editor found that "Maleonn" was not delicate and ingenious enough in dealing with the transition and connection of music, resulting in the transition between music being blunt and abrupt, which affected the audience's auditory experience and immersion in the story.
The video below illustrates the limitations of Maleon's dynamic storyboard AI in terms of audio creation.
Loading...
Conclusion: "Partial Redrawing" and "Picture to Video" have not yet been opened, and the effect of "Magic Brush Ma Liang" remains to be seen
As an AI film and television tool, the advantage of "Magic Brush Ma Liang" is that it can quickly generate corresponding dynamic storyboards based on text content. However, at present, it mainly relies on preset algorithms and modes to create, and cannot fully capture the delicate emotions that the creators expect to express.
In addition, since the dynamic storyboard is still in the beta testing stage, its "local redrawing" and "image to video" functions have not yet been opened, so the generated video is still composed of static images, and the final generation effect of the video remains to be further observed.
Perhaps in the future, AI may indeed become a "magic pen" in the hands of creators, giving everyone the opportunity to become a director and realize their film and television dreams.