In 1895, the Lumière brothers found a café in Paris, set up chairs, and invited people to come in and sit down. The unsure spectator watched as a black-and-white train staggered into the Lasiota station in south France, as if it were real. This 50-second, one-shot film is considered by many to be the first film in the history of world cinema.
Compared with the fleeting memory of human beings, the film works attached to film are almost eternal, which is also the excitement brought by the beginning of the appearance of "Train Station".
But the long history of cinema has long since surpassed the average lifespan of individual human beings. Throughout the 20th century, film and even all video content production, including animation, was built on film. On this longer timeline, film suffers irreversible losses such as mold, distortion or physical wear for various reasons. It wasn't until more and more movies became mottled "old movies", some of which became increasingly unable to meet the technical standards of playback, that people discovered that movies were just "old" and slower, not eternal.
It wasn't until 2002's Star Wars Episode II: Attack of the Clones that it was completely freed from film during filming and became the first film to be shot entirely in digital video format, that the film industry entered a new era.
Time has passed, and now the IP of Star Wars has shifted from Lucasfilm to Disney, becoming one of the most crowded Disney parks in the world. In the development of the film industry, animation, which broke away from documentary roles and began to carry the human imagination, became one of the most important branches.
In front of Disneyland and Universal Studios, which are popular all over the world, we always have some regrets about Chinese animation. Chinese animation represented by "The Haunting of the Heavenly Palace" and "Hulu Brothers" stood at a very high starting point, but in the end it did not bring about the overall development of Chinese animation.
In recent years, more and more classic domestic films are being rescued through 4K repair, and even get the opportunity to be re-released. In contrast, the classic domestic old animation has received much less attention, in fact, most of these works are recorded on film, and in the process of digitization again and again, the loss is deepened, and the story of Kit Kat and the unique oriental rhythm are covered up by rough pictures, and now children are not interested.
Now gradually someone has noticed this blind spot.
Recently, Watermelon Video and Volcano Engine opened a "Watermelon Video Classic Video 4K Restoration Conference", which means to say - "Dust off the old animation, this matter is left to us".
<h2>Use algorithms to achieve "rejuvenation"</h2>
What does the "4K" version of Hulu Brothers look like?
The high-definition picture details and brighter colors filter out the blinding snowflakes and black lines, while completely retaining the animation style of the paper-cut characters and ink painting background of the animation itself - you can even see the lines of the leaves on the head of the gourd. The 4K version of "Hulu Brothers" was broadcast at the press conference, which is the most direct impression given to viewers.
In 1985, the animation team at the Shanghai Fine Arts Film Studio probably could not have imagined that the upcoming "Hulu Brothers" would become the memory of two generations in the future, and return 36 years later with a more refined and full of vitality.
In the era of pure hand-made animation, every frame in the picture was created by the animation team. By fifteen frames, the 19-episode 10-minute Hulu Brothers would mean 170,000 stop-motion animations, and a one-second animation clip could take more than a day to complete. In an interview video about "Hulu Brothers" director Zhou Keqin, the director recalled that sometimes it took three days to shoot, and it was difficult to finish a shot, and once it was seen through, it was necessary to start again, and the animation of several seconds may take three days.
After such painstaking animation works are transcribed in the form of film, on the one hand, they are physically or chemically damaged over time, and on the other hand, they further lose their picture quality and color in the process of post-digitization, and gradually distort colors and mottle pictures, which cannot adapt to the needs of modern movies.
"Such a work is worth our efforts to repair," said Ren Lifeng, president of Watermelon Video.
The restoration of old animations is not much different from the more common classic film restoration in terms of process, which is not a simple project. And the precious artistic elements in the old animation also mean steeper difficulties in restoration. Volcano Engine's machine learning technology and AI algorithms are behind the support of this classic Video 4K restoration plan.
"Clarity, fluidity, color and imperfections are the four main dimensions of old animation restoration." Zhao Shijie, a researcher at the Multimedia Laboratory of Volcano Engine Technology, said.
Taking the optimization of sharpness as an example, the old animation is limited by the simple equipment in the production process, the sharpness is often below 540P, and due to the blurring of the lens during shooting, the immaturity of post-digitization and the problem of storage compression, the problem of loss of picture texture will be further highlighted. The details of the disappeared picture cannot be recovered with the loss of the film, in a sense, the repair of clarity is tantamount to "making something out of nothing".
After the rise of machine learning and AI technology, this kind of reasonable "out of nothing" can be achieved.
The Volcano Engine team introduced machine learning into the repair of old animations, first analyzing massive high-definition information, and after learning some characteristics of high-definition video, the algorithm will get some conversion direction from low-definition video to high-definition video, so as to automatically generate some missing picture details in the middle.
Similar principles can also be applied to the improvement of the smoothness of old animations. Through an intelligent interpolation algorithm, the restoration team can rely on AI technology to analyze the front and back frames of the picture and generate intermediate content, through which the original 15-frame animation can be supplemented by a high frame rate of 60 frames.
However, in the process of repairing "I Am Crazy for Song", the action of making up frames was rejected by the Shanghai Film Art Studio. The explanation given by the other party was, "A musical animation, it is the kind of stun that has a sense of rhythm."
This made the restoration team of Watermelon Video and Volcano Engine realize that this different artistic part of each old animation was the most difficult part of the restoration process to grasp, and that in this small operating space, art and technology needed to be balanced.
<h2>The balance between technology and art</h2>
The sense of stuttering caused by technical limitations in "The Legend of Nezha" has become a wonderful rhythm in "Hulu Brothers" and "I Am Crazy for Song", and the repair situation faced by each animation is different, which makes it difficult for the repair of old animation to be quickly realized with a batch repair method.
"We will first do an intelligent analysis of the film, and finally form their own personalized repair plan," Zhao Shijie said.
And animation is the art of people after all, in order to maximize the preservation of the artistic sense in animation works, the use of algorithms also needs some restraint.
For example, in the process of repairing noise and snowflake defects, 95% of the work is handed over to the algorithm, and the other 5% of the imperfections require designers with artistic backgrounds to make artificial judgments.
"It's not that it can't be repaired 100%, but if the algorithm is required to do complete identification and repair, it is easy to erase some of the original artistic effects or texture information in the picture as flaws, which destroys the film," Zhao Shijie said. Through manual identification, the restoration team can mark out the corresponding "false defects" and inform the algorithm in advance to retain the original artistic effect of the old animation.
Another example is that many of the classic old animations in China contain elements of ink painting, and this hazy feeling needs to be preserved. In the restoration of "Hulu Brothers", the restoration team needs to first distinguish the picture into two parts: the background of the ink painting and the paper-cut character, while maintaining the hazy feeling of the ink painting part, the accuracy of the paper-cut part is improved.
But overall, the benefits of introducing machine learning and AI techniques for the restoration of old animations or movies are obvious. In the past, the restoration method of old movies often relied heavily on senior restorers, which required frame-by-frame color grading and comprehensive restoration of the picture, and the restoration of a film took months or more. However, with the introduction of the above new technologies, some manual repair processes that need to rely on repair software can now be replaced by powerful computing resources, and this repair process will be compressed to a very short time.
At present, Watermelon Video and Volcano Engine have repaired six old animations including "Hulu Brothers" and "Little Tadpole Finds Mother" and launched on the platform, and said that the number of old animations that will be repaired in the next year will reach 100.
"But it is undeniable that the important role of restorers in film restoration cannot be replaced," Zhao Shijie added.
<h2>The boundaries of the video</h2>
Every move in the business world has a cause. Watermelon Video chose to announce the restoration plan of the old animation at this point in time out of feelings, not only feelings. Ren Lifeng gave the definition of China Video for the first time at the Curiosity Conference in October last year, and a year later, the content of China Video has been paid more attention to.
When this idea first came out, many people felt inexplicable. Unlike short videos that are extremely condensed in time, nor are they like long videos with obvious content directions, the reference to "medium video" seems to come from the exclusion method - in addition to short videos and long videos, others are medium videos, and their own characteristics are not distinct.
But whether it is Douyin or Aiyouteng, now all content platforms are moving closer to the middle video, giving people a feeling that people in the circle are full of people who want to go out of the circle. Compared with short videos that are too short and fast and long videos that test concentration, the middle video that stands "outside the circle" of the two is a more natural product. In other words, the reason why people may be unaware of the concept of video as a medium is precisely that it is too common and not in any small circle.
"Tell a complete story in ten minutes, with ups and downs, climaxes, and deep meanings — the old animation is the classic Medium Video." Ren Lifeng said at the press conference.
The classic national comic represented by "Hulu Brothers" has become the next clear content category of the concept of Chinese video, and has been in the blind spot of public awareness before.
The layout of watermelon video for the platform's 4K picture quality since last year makes the "resurrection" of this old animation seem logical.
At the beginning of June last year, Watermelon Video announced that it officially opened 4K picture quality to all users and video creators for free, earlier than iQiyi, and unlike the B station that needed to cross the membership threshold, it became one of the few players in the video content platform camp at that time who fully supported 4K for free. This decision is perceived by the increasingly stringent requirements of users for the audiovisual experience, but at the implementation level, 4K content maintenance and the resulting huge bandwidth costs have become the burden of watermelon video. Watermelon Video intends to provide confidence for more professional content producers through this move, and many head creators, including "Food Writer Wang Gang" and "Unicorn SHOW", have released their first 4K videos on Watermelon Video.
In addition to the attributes of PUGC, the video copyright IP content layout from movies to TV series is still a distinct attribute of Watermelon Video.
In the general direction of the overall content to the medium video in the past year, a two-way encirclement line around "4K" is very clear. On the one hand, Watermelon Video hopes to further catalyze the prosperity of the original Chinese video content ecology with high-quality creative conditions; on the other hand, it takes 4K as an opportunity to gradually gather more high-quality Chinese video content on the platform.
China Video is far from a new thing, and its boundaries are not limited to watermelon platforms or other video platforms. Decades ago, "Hulu Brothers", a graduation party video from a few years ago, or a Vlog for the next holiday can be counted as a medium video. Ren Lifeng said that the video platform that Watermelon Video wants to do is a content platform that includes the past, the present and the future.
"Not even limited to mobile phones".
At last month's watermelon PLAY conference, Ren Lifeng said that he would increase investment in smart TVs, TV boxes and other "big screens". The data support behind this is that in 2020, the number of 955 million Internet TV users and 295 million terminal activations in China reached a new high, while the proportion of TV playback of video on watermelon video increased from 21% in the fourth quarter of 2020 to 35% in the second quarter of 2021.
The mobile Internet era has developed to the present, the public has begun to return to television, and at the same time as the watermelon video turned to the middle video, some users are leaving the mobile phone.
But no matter what stage of technology and smart devices are developed, what remains unchanged is that people will not be far from life, and have always had the willingness to record the moments of life, which has been transformed into a steady stream of original videos on the platform of Watermelon Video.
To this end, Watermelon Video and Volcano Engine also said that some of the basic capabilities in video restoration technology have also been opened to all ordinary users through the watermelon creation platform. The Watermelon Video team receives a large number of fresh and original videos every day, many of which are unsatisfactory due to limited shooting conditions. This is actually similar to the problems existing in the old film, and each user can make a high-definition fix of their own video. This will also improve the user's creative and viewing experience on the other hand.
"Many, many years ago, blurred images carry the brightest moments in the lives of generations of people, and I hope this feature can bring those faded precious videos back to life." Ren Lifeng finally said.