In the 2024 Spring Festival Gala, accompanied by Ren Suxi's warm singing, people across the country watched the first AI video of the Spring Festival Gala. "Looking at a fairy tale through the window, under the clouds of light, the evening breeze gently blows through her silver hair, he smiles and waits for her to walk home slowly", in the music, a pair of men and women dancing to the music go from youth to twilight.
The 2024 China Central Radio and Television Spring Festival Gala
What few people know is that behind such a touching program is a difficult commission with a time limit of less than a month and no room for error.
And AI does what seems like an "impossible task."
The rapid development of generative AI technology has brought unprecedented depth and breadth of cooperation between AI and people. When we no longer just talk about AI in science fiction movies in the abstract, then AI has already quietly begun to change our work and life. In view of this, KOOP China has launched a series of dialogues to talk about everything related to AIGC with industry insiders.
Are you curious about how the AI pas de deux of the Spring Festival Gala work "She with the Light" is realized? How does AI empower designers? The creator behind this video, PPT designer Arwen, shared his experience in a conversation with us.
He said that when he first used AI, he had no idea that he would be where he is today – "I basically can't live without AI anymore."
The following is a summary of the conversation with Arwen.
The stock copyright picture, reprinting and using may cause copyright disputes
How AI is part of my job
Q: Could you please briefly introduce your work?
Awen: My job is a PPT designer for press conferences, and I have a design studio in Beijing that specializes in PPT. I'm also an AI artist.
Q: When did you first become interested in AIGC?
Arvin: I've been focusing on AI since April 2022. Because I surf a lot on Weibo, in April 2022, a tool called Disco Diffusion became popular on Weibo, and I and many artists and good friends around me used it.
Q: When you first started experimenting with AIGC, what did you think of the maximum capabilities of AI painting?
Arwen: The first impression is definitely shocking, who has ever seen this kind of tool in 2022 that can generate images with just a few words? So I was very excited, but at the time, the quality of the AI-generated images was very mediocre and not high-definition, and it wasn't until DALLE-2 came out that I had a completely different view of the tool, and I thought "maybe it could be used in our work", but I didn't expect it to be like this at all - I basically can't do without AI in my daily work. At least in the realm of still images, AI is a fully usable state.
Q: What has AI helped you do?
Arwen: One of the most important aspects of my job right now is finding design materials. Presentations often do some ultra-widescreen designs, but there are very few screens in the gallery that are more than ten meters long and need to use very high-definition large-scale picture materials. We used to spend a lot of manual time compositing large assets, but now we just have to tell the AI what size I want.
In fact, in the workflow of my main business to do press conference PPT, AI currently does not account for a high proportion, about 25%~30%. Recently, I've started experimenting with some AI transpainting, where AI accounts for 80%~90% of the overall workflow.
The stock copyright picture, reprinting and using may cause copyright disputes
Q: Will your creative thinking change because of the addition of AI?
Arwen: I seem to be getting lazy. In the past, when a creative demand came, I would think about it myself, but now I may subconsciously type a few keywords to feed to the AI. Equivalent to, I have an extra super assistant.
Q: Do I need to manually change the image obtained from AI?
Awen: At the beginning, we still needed to import the AI materials into PS to "sew and mend", but now we can basically do it all at once, and we almost don't change it.
Q: Have you tried the tool for AI to make PPT?
Awen: Actually, our professional PPT designers don't really see the effect generated by this tool, and they can't use it. It's more like a template for office workers to cope with their leaders' work reports.
Q: When did you start doing AI transfer?
Awen: At the beginning of the year, Mr. Hai Xin and I received a commission from the Spring Festival Gala program team, when Ren Suxi sang the song "She with Pillow Light", the big screen wanted to use a pas de deux as the background video projected on the stage, wanting to present a pair of men and women dancing to the music, from youth to old age. Because the production cycle is very short, less than a month, if you do it with the traditional path, it takes a lot of time, whether it is to do a mocap scan or model two dancers. At that time, the Spring Festival Gala program team thought of using AI to see if it could make a "not bad" effect in a very short construction period. In the end, we did.
Q: How does it work?
Arwen: We faced a lot of challenges during the execution of the project.
For example, character stability issues. The design of the show is a pas de deux with three stages of cross-dressing, including three stages of marriage, post-marriage, and old age. Leveraging AI for silky smooth cross-dressing is an important need. The reason why pas de deux is difficult is that the AI confuses the characteristics of the two characters, so there are often problems such as gender swapping. We tried a lot of things and finally solved this problem by using the ControlNet tile model to fix the character's features.
Another example is the realization of porcelain materials. With the support of SDXL and Civitai open source models and LoRa, we quickly decided to choose the dancing figure made of white porcelain. But we encountered a lot of problems in the middle, just when we thought we had to train SD1.5 on the porcelain LoRa, we found that we could solve the material problem with a single "keyword". In addition to keywords, we also found a plugin called IP-Adapter, which can be used to guide the AI to generate the effect of a specific material with a reference image.
Another challenge was the stability test of the dressing, we first aligned the clips in the PR, and then achieved the satisfactory effect of the program team through prompt travel (different keyframes describe different content) during generation.
The stock copyright picture, reprinting and using may cause copyright disputes
Imagination of AI
Q: What is the room for upgrading AI-generated images?
Arwen: I think it's the end of AI-generated images.
Q: Have you reached the boundaries you imagined?
Arwen: Anyway, if you put two images in front of me at random, I might not be able to tell which one is generated by AI and which one is created by a real person.
The more AI develops, the more I can't tell the difference, even for designers in our professional field, then for the public, the quality of AI pictures is completely sufficient, and AI painting has been next level. In fact, we are the group that is most proud of AI development, and most of us looked down on AI-generated images at the beginning. I thought, "How can AI be as good as something we design or draw ourselves?" "But the more you get further on, the more you see that the quality of the AI generation is getting better and better, and when we try it, it's just 'addictive', and it really makes you less workload and more efficient. Then I slowly shut up.
However, if I have to say it, it needs to be combined with the ability of ChatGPT, a large text model, so that the Wensheng graph model can better understand "human words".
Now I'm going to do more AI transfer to make a style that is more oil painty. For example, the street scene of Shanghai is turned into a scene from a famous Van Gogh painting.
Q: What jobs can AI replace, and what can't be replaced?
Arwen: Repetitive work will definitely be replaced. For example, if you used to work every day, and every time you did a job that had nothing to do with creativity, you would definitely be replaced. If it can't be replaced, it must be some softer abilities, such as creativity. I think there is no way to replace this kind of work, and the more you go to the back, the more you fight for your personal aesthetics, your personal content aesthetics or design aesthetics, which will affect your final image works. At present, AI can only bring some random inspiration, but human beings can output their own aesthetics very subjectively, which cannot be replaced.
One of the more interesting things I observed was that two years ago, some laymen went to provoke artists, those game concept art, and designers, saying that you were going to lose your job soon. BUT NOW, TWO YEARS LATER, YOU'LL FIND THAT MOST OF THE TOP 10 PLAYERS IN THE OPENART COMMUNITY ARE GAME CONCEPT ARTISTS AND DESIGNERS. AI painting finally climbed to the top of the pyramid, and it was the group of professionals who stood at the top.
Q: What advice do you have for AIGC practitioners?
Arwen: Don't be too anxious about being left behind. New technologies appear every day in this world, and according to my observations for most of the past six months, it is an efficient way to find out the leaks and test the tool that works best after everyone tries it out.
Q: Do you think AI can create new jobs?
Arwen: Absolutely. But at present, more people who use AI are transferred by traditional designers.
Q: Do you think the ceiling of AI in the future will be human-like? Or will AI have another direction to go?
Arwen: I think he will surpass people, because AI has far surpassed every human in terms of knowledge alone, and it may even be the sum of human knowledge. The key comes back to how AI uses knowledge, and I think it's probably only a matter of time before AI surpasses humans.
Q: Can you recommend some interesting ways to play AIGC that you have found?
Awen: The best AI translation plug-in I've ever used is Immersive Translation, which can turn all foreign language web content into bilingual translations with one click, and there is a large language model behind it, and the translation is particularly accurate. It is also recommended that all programming novices try cursor, and they can write websites and applications with zero code foundation! In addition, I would like to share a friend who wants to play with AIGC in depth One of the most important AI tools to get started: comfyUI, after getting started, all open source technologies are your plugins.
Q: Is there anything else you'd like to share?
Awen: I'd like to say that China's achievements in the field of AI are actually very scary, very powerful. Around the world, the media has played up the modeling capabilities of large foreign companies too much, ignoring the low-key but glittering Chinese teams.
In fact, in the open source community, at least in the field of AI painting and AI video, 90% of the components are written by Chinese people or Chinese teams: LCM, AnimateDiff, instantID, IPadapter, LivePortrait, etc., not to mention Colin. In fact, foreign open source communities are very in awe of Chinese teams, but Chinese teams have always been very low-key and rarely out of the circle in China, so many people always think that China's AI technology is not good and cannot beat foreign countries, but in my opinion, it is not at all!
Planning and production
Author丨Frozen top oolong popular science creator
Interviewee丨Simon Arwen, co-founder of AbleSlide, AI artist
Audit丨Yu Yang, head of Tencent Xuanwu Laboratory
Planning丨Lin Lin
Editor-in-charge丨He Tong
Reviewer丨Xu Lai Linlin