Recently, Zhang Zhiwei, Audio Director of Qujia Interactive Entertainment, attended the Game Audio and Video Technology Development Conference hosted by CRIWare and delivered a keynote speech entitled "Game Sound Design and Music Creation".
Zhang Zhiwei's original "King of Avalon" won the Top Ten Music Awards of Chinese Games issued by the China Music and Data Association
Zhang Zhiwei is an artist with high popularity and honor in music, games, entertainment, education and other fields. As a teacher, he is not only the director of the Sound Design Department of the School of Music Science and Technology of Beijing Contemporary Conservatory of Music, the lecturer of the Recording Master Class of the Communication University of China, and the first educational pioneer in The University of China to establish a game sound design major. In the field of game audio, he is also the official instructor of Wwise Engine and CRIWare Engine;
"State of Survival" new theme song "Flowers and Flowers"
"A Thousand Flowers" has been recommended by Apple Music, Mymusic and other global music platforms
"A Thousand Flowers" has been recommended by Apple Music, Mymusic and other global music platforms
For the understanding of games and game audio design, Zhang Zhiwei said: "Doing game development is actually a great thing. The game is essentially a comprehensive art category, it integrates the latest research results of various arts and science disciplines, including technical engine technology, graphic technology, digital media technology, artistic aspects such as art style, game play design, map and level design, etc. To make a game, no matter what the size of the team is, he needs to work together from art to technology Practitioners in various disciplines to create products that make players scream.
Game audio is the same, every game audio designer needs to go through in-depth learning in music, sound effects, voice, logic and other aspects in the early stage, before having the opportunity to enter the game industry and become a "game audio intern". After that, you still need to improve yourself by constantly learning the knowledge of the game industry, more in-depth understanding of the design of game audio, and eventually become a professional game audio designer. ”
The following is a transcript of the speech:
Hello everyone, I am Zhang Zhiwei, the title of my speech today is "Game Sound Design and Music Creation", with this topic, I would like to share with you the creative experience of the past year. Before that, I want to introduce myself.
I am a practitioner who has worked in the music industry for many years and I have 3 main identities.
The first identity is as a teacher. Since graduating in 2006, I have been teaching at the university. At first, he mainly taught the arrangement and composition of popular music. In 2010, I co-founded the game sound design major with Mr. Yang Jie and some industry seniors at the School of Modern Music.
When we first started this major, we were more distressed that students did not know enough about this major, and it was easy to define it as "to do game audio is to do sound effects", and many people will impose the concept of film and television sound design here. So, we have also made a lot of efforts from 2010 to the present. Among the students I have taught, post-00s students will have a better understanding of this major. Because many students are exposed to computer games, home console games and mobile games from an early age. Compared to before, the students' understanding of game audio has changed a lot, which is also a very gratifying point for me.
The second identity is the audio designer, which is also an identity that I am very proud of. In the early days of work, the industry actually called audio designers sound designers. However, with the maturity of the industry, everyone felt that sound designers could not cover our daily work content, so the concept of Audio was introduced, collectively known as audio designers.
Zhang Zhiwei participated in the "Spartan" physical training organized by Qujia Interactive Entertainment
I work as an audio designer at Qujia Interactive Entertainment. In our day-to-day work, we are responsible for all aspects of the product that involve audio, including design, production, mixing, interaction, and so on. I'll talk about this concept in more detail later.
The third identity is the founder of the Good People Orchestra. Many people have asked me what this orchestra of good people means. Let me give you a brief introduction. In ancient China, people called couples "good people", so this orchestra was actually an orchestra that my wife and I made together. The core purpose of doing this orchestra is to make some music that you like.
My wife and I were very fond of traditional Chinese music in the early days, including ethnic music, etc., so when we founded the orchestra, we walked through most of the ethnic minority areas in China, and in the process of visiting and learning, we integrated traditional elements and modern music to create our own music. That's what we expressed in our early music albums.
Later, we also did some stage plays at the Shanghai International Arts Festival. In recent years, the biggest project we have done is China's "Twenty-Four Solar Terms" series of music. It can be heard on all major music platforms. Overall, I made my favorite music with my wife and was very happy.
After the introduction of these three identities, we will talk about the formal content.
I know that many of today's events are senior experts in the industry, but I think there may also be some students who are new to the industry and want to learn about the game industry. So, I'll start with a short introduction to the concept of game audio design.
From the classification of game audio design, there are four main contents: music, sound effects, voice, and logic.
Music, in fact, is the concept of OST and BGM that we usually mention. Most of the music content in the game is pure music, and songs will also be played in the case of some special gameplay and special system embedding. For example, title BGM, which is common in online games and home console games, or BGM that promotes and brand directions. (Title BGM with full song performances can be found in Wonderland 2, Monster Hunter: Rises)
Sound effects are actually the sounds of physics, magic, and world environments in the game world. Because the sound effects tend to be triggered by the interaction of characters, environments and the world ecology at that time, the sound effects in the game are different from the traditional linear sound effects.
Voice, in fact, is the sound that can express and transmit a certain information in the game, it is not only the dialogue voice between the characters, but also a way of information transmission. For example, there is a prairie in the game with a tree in the center of the prairie that is humming every time you approach it. At this time, the tree may be a soft guide or a key clue in the game, and it hopes to attract the player's attention through voice performance. Similar content will appear frequently in many genres such as fantasy and epic. Of course, animals will also have such a performance, such as the form of a bird talking, which actually contains a lot of voice transmission information.
Logic is the triggering of the above types of sounds and the rules of change. We mentioned earlier that music, sound effects, and speech are all based on logic and rely on code. For students who come from an art background, it is a bit difficult to learn this. I'm very aware of this, so I think it's a place to spend time learning.
These are the core foundations you need to master for game audio creation.
Once you have this foundation, let's talk about how to become a qualified game audio designer.
We know that no matter what industry you're in, if you want to be a veteran of the industry, you need to take the time to reserve knowledge and start learning from the basics. So, let's talk about how to enter the game audio industry and become a qualified game audio intern.
I prepared a diagram showing the learning and growth path of the "game audio intern". This learning system consists of instrumental music learning, music theory fundamentals, composition and music production learning, recording and mixing, interactive sound engine technology learning, and some game design knowledge.
First, you need to learn instrumental music for 4-8 years. I think students who have studied musical instruments know that four years of an instrument is actually not a long time. In the process of learning musical instruments, many students also need to take into account their daily studies, most of them are difficult to focus on practicing for 8 to 10 hours a day, so if you want to master an instrumental music solidly, it takes at least a few years to learn. In this learning process, as the difficulty of the exercise continues to increase, it is also necessary to have a good music theory foundation to support. Therefore, I recommend at least 4-8 years of instrumental practice experience before going to college. Of course, there are also some special cases here, such as some people who are very talented and can maintain hard training, they may be able to get some municipal and national awards after 2 years of study, for this type of person, this time can also be used as a reference.
Secondly, in terms of music theory, I personally feel that it is necessary to have at least one year of study experience, at least in the "small three doors" of the college entrance examination to achieve the proficiency level (small three subjects, that is, basic music theory, sight singing, ear training). This way, it will be a lot easier when you enter the university stage to study composition and music production for 4 years. In terms of subject selection, I recommend that you make up for each other. Students who learn traditional composition, because the things output by traditional composition and the things we need to apply at the moment are not exactly matched, so there are many students who will learn DAW music production software from the B station, etc.; if you are learning game audio or digital composition at the beginning, then you can go back to learn traditional composition, it will make up for each other, there is no difference between advantages and disadvantages. While learning composition, I also recommend that you learn the three things of recording, mixing, and foley in the past four years. Because each subject is actually very fine, take the traditional recording, she is divided into music recording, film and television recording and so on. So recording and mixing is best if you can learn and get in touch.
Subsequently, foley also needs to be learned. This is an important means of serving the creation of game audio. I recommend that you learn this skill when you go to school, and you may use it at any time. In my teaching, there are two sets of series of foleys, one called cold weapons and one set called hot weapons. Cold weapons are all kinds of weapons in martial arts novels, knives, swords, guns, etc., and I would recommend my students to look at the history of cold weapons first. Hot weapons are finger guns, from realistic guns in World War I and World War II to sci-fi, animated, and magical gun designs. For these audio designs, my request is 100% original sound, and I will not go into the details.
Immediately after that, we need to learn the knowledge of composition and recording, and we must learn it solidly. I know a lot of people will come up and say that we use a lot of advanced technology and engines, but in the early stages of learning, I still recommend that you learn the creation and production parts solidly.
Finally, we need to learn interactive sound creation. As I mentioned earlier, game audio relies on a trigger mechanism, and without a trigger mechanism, there is no way to talk about sound playback. So this interaction engine needs to be learned over time. After learning the interaction engine, we can learn more about the game, including the engine, graphics knowledge, and some game gameplay and content design, which are conducive to improving the game audio creation, as well as the efficiency of communication in future work.
Once we've mastered these abilities, we'll be honored to be a game interaction sound design intern. Many people will ask, why did I learn so much and just be an intern?
That's because game development is a great thing, a comprehensive art showcase that incorporates the latest research in various arts and science disciplines. Technical aspects such as engine technology, graphics technology, digital media technology, artistic aspects such as new art styles, audio design, gameplay design, map and level design, and so on. To make a game, no matter what the size of the team, it requires practitioners from art to technology to work together to create a product that others like. Therefore, I think that the previous years of learning and preparation can be colleagues who have the opportunity to enter this industry, through further learning the knowledge of the game industry, more in-depth understanding of the design of game audio, and finally slowly become a professional game audio designer.
Well, by this point, we've become a professional game audio designer. After a few years on the job, you might start thinking about how to make game audio better. My approach to this is to solve it with some strategies. The examples we show next are the experiences that have been summed up in the real battlefield, not the theoretical teaching.
In this PPT, I wrote two core keywords, one is the record, and the other is film and television.
Many people will ask, why do I make game audio, but I have to learn from these two seemingly unrelated industries?
So let me share a point of view: for ordinary users, watching TV, brushing network dramas, playing games, brushing vibrato, listening to songs are all entertainment behaviors, everyone likes entertainment, the core is because entertainment can make people happy, which is one of its characteristics. In addition, the logic of the audio experience is that the two industries of film and television and records are more mature than the game industry. So, what they do in terms of experience, the game industry can learn from and learn.
Let me tell you in detail about what we learned from the film, television and recording industries through a series of examples.
First, we should learn from film and television about their audio technology and narrative methods. Take the CG movie of "State of Survival" produced by Fun Plus Interactive Entertainment, which uses theatrical panoramic sound mixing technology. So what is theatrical panoramic sound? Here I give you a brief science popularization, theatrical level atmos is our last output of the file into the standard Dolby Atmos theater can be broadcast. The output of Dolby Atmos itself is hierarchical and has to do with his mixing process. That's in our video, we're using the highest standards to do the audio output.
Next, we will show you a CG audio production that is coordinated by two countries across borders. Because the film took a long time, we first asked the European production team to help us make a DEMO version. Due to jet lag and geography, it was difficult for our audio team and the European team to stay at the same time, so we first got some basic sound from the DEMO version produced by each other, and after this version, we designed the music and sound effects based on the final picture.
In the production stage, the European team also provided us with some design ideas, because the European team's creative ideas are not the same as ours, so we can work together to produce this piece of sound effects design. This includes the sound effects of the character's footsteps, guns, etc., as well as the effects of the final battle part, the movement of the enemy, and so on. The place where these sounds are foley is in China is a digital film production base.
Everyone knows that The recording studio of China Film is used to make big movies, so their technology, props, including foley artists have a high level. When I was composing CG music, I would think about what kind of atmosphere I needed to create around the worldview of State of Survival, and at this time, I was reminded of mozart requiems that I used to like, so I wrote a piece of music that felt this way. In addition, the whole battle music in the film, I personally think has a bit of a cyberpunk feel. The original song of this music was used in the battle tracks of the previous game, which was well received by many players at that time, so we thought of rearranging the original engineering files, and the composer students should know that if you directly use the original audio engineering, its cue point is not right. So we modified the parts including tempo, barts, etc., but the overall melody and timbre are the same. It is equivalent to re-customizing a user's favorite music for CG.
When we finished the production and entered the mixing stage, we found a theatrical mixing studio to mix, this mixing studio is actually a movie theater, at that time I worked on this project with the mixer, the whole project in the production process, including audiovisual effects, mixing effects and so on, we applied a lot of techniques and techniques in it.
Last year, we did 3 things around atmos: theatrical-level CG mix, the atmos records, and the interactive attics in the game, which we'll read in detail later.
As many students who follow SLG mobile games will know, last December, State of Survival and DC had a deep connection, and we added the very popular Joker character from DC Comics to the game, while also producing a high-quality CG video. So I want to share with you what we did when we made this film.
Throughout the CG production, the director made a request to the audio crew to simulate Joker's laughter and hissing voice as the characters changed. At that time, after hearing this idea, I felt that it was actually more difficult to achieve this effect, and we repeatedly discussed this need and the difficulty of implementation with the director, but finally decided to make this effect.
In order to achieve this effect, our audio group students also worked very hard, and they tried to record many different multi-layer voice samples. In addition, we would also like to be very grateful to the fish used in foley, which contributes a lot in the process of foley. In order to make a sound texture that caters to the changes in the characters, we first freeze the fish and then thaw, and finally make this special sound effect. When I was writing the Joker theme song, I felt that the image of Joker was a crazy person in a suit, so I wrote a piece of music around this theme. At that time, we also decided to make this music into a panoramic version and launch it on a global music platform. It is worth mentioning that this album is honored to receive global recommendation from Apple Music.
After the CG was officially launched, it was viewed by 470,000 people on the Youtube channel alone. The whole CG has resonated with a lot of players. People think that the audiovisual effect of the whole CG is very good, and the narrative completion is relatively high, they can learn what Joker did in the world of "State of Survival" through this CG video, and what kind of crisis was triggered at the same time.
The above case, in fact, is the content we produce by learning audio technology and narrative methods from film and television, film and television as one of the pioneers of digital media art, it has a relatively deep accumulation of narrative skills, although the game has a relatively long history from the overall point of view, but in terms of interactive digital media, film and television narrative, emotional expression there are still many places worth learning and learning. In contrast, while games are interactive, most of the time you're also growing over time and experiencing new content. So from this point of view, the game can actually be like film and television, giving players a complete world view and story, outputting the emotions of different characters, allowing players to experience the plot of the game with their own feelings over time. While it's not possible to discuss the details directly, we've been very successful in another project, and judging from the test results, this narrative experience not only improves the user's experience, but also performs well in terms of data.
After talking about film and television, I would like to talk to you about the second point of the game's audio strategy - records.
Before I start my lecture, I would like to explain to you that the reason why we make records is not for art, nor for ourselves, but for users.
My company, Qujia Interactive Entertainment, is a company with a strong UA and data analysis, so the project team found through interviews, data analysis, and so on that our users were impressed by the music in the game, and they liked the form of records. So when we saw the results of the survey, we thought that the audio team was going to make a limited record in time for the second anniversary of State of Survival.
After we came up with this idea, the community and operations students in the project team felt that it was very meaningful, so we made a music documentary called "The Birth of Sound", which also talked about the music creation concept of "State of Survival". Here we have taken a short section as a demonstration.
State of Survival Music Documentary: The Birth of Sound
The album shown in the film was originally titled Deep in the Sea of Flowers, and we gave him a very romantic Chinese called "Trail of Flowers". If you've been through last year's version of the game, you know at a glance that it's expressing some important events in the game when you see the cover art, which is one of the core points of our design. I hope that players from getting the album, seeing the cover and listening to music, in the whole process can produce some memories or associations of the game narrative and plot, then at this time to hear the music, the whole atmosphere feels very good.
After this album is listed on major platforms around the world, it has won a lot of platform Banner recommendation and awards, these recommended bits are spontaneously selected and listed by the editors on the platform side, we are not spending the promotion fee, we can achieve such results, I am personally very proud, you can also imagine that after many years of deep cultivation in an industry, what you write can get a certain recognition, in fact, it is a very happy thing.
After finishing this album, we hit the iron while it was hot and made a 5th anniversary record for another of our fist products, King of Avalon. King of Avalon is a very powerful product, and by the time the album was made, it had more than 100 million users worldwide. Students who have done projects may know that this kind of long-term operation of online games needs to be constantly iterated and polished, from the perspective of the game, "King of Avalon" often re-produces the novice guide and the early game experience optimization according to the current user situation. From a musical point of view, his quality is also gradually iterating, because the original product music investment budget is still different from the current one, so it contains some test music, some arrangements are very good, but some are not recorded.
We made the 5th anniversary commemorative record this time, but also hope to make up for these regrets and make a product iteration from a musical point of view. So, in the recording part, we hired two international symphony orchestras to do it. We invited one of the world's top 10 symphony orchestras, butapes' symphony orchestra, and China's very strong principal philharmonic orchestra, to help us re-record the music from this 5th anniversary album. At the same time, in the mixing part, we also invited the domestic front-line OST mixer to help us complete the mixing part. Of course, we also spent a lot of time to communicate and coordinate, so the process is still difficult, but the results we get are very good.
Through the 5th anniversary album of "King of Avalon", we won the award of "Top 10 Games: Excellent Music Design of Games" from last year's Game Industry Committee, and we also received Apple Music global homepage recommendation. From the listing recommendation bit to the end, the recommendation continued for about a week.
In this project, we have top multinational orchestra collaborations, over 100 million user products, and Apple Music that has been recommended for many days and professional awards, which I think are also a point of great pride and pride for me. Both me and our entire team have put a lot of effort into it. In order to produce good music, we also put users and projects first, so that players can establish a connection to the game through music, and at the same time, through music, make the narrative, story and game atmosphere of the project better.
The above is what we have learned from film, television and records in the audio promotion strategy. I would like to continue the topic of transnational writing and tell you about our achievements in transnational art collaboration.
In terms of cross-border cooperation, we not only have orchestras, but we have also co-produced some content with some top overseas musicians, including the part of the cooperation with the teacher of the very well-known Japanese composer Iwatsuyuki (a well-known Japanese game composer whose representative works include Grandia, Reverse Referee, Fantasy Simulation Battle and other well-known home console games).
Last year, because of the needs of the project, we needed to make a Japanese song that integrates European and American styles around the characters. At that time, we wrote a theme song called "Cherry Blossoms Fluttering", which was Chinese by CRI's teacher Zhu and was very romantic.
The music was actually fun when it was made, when we had a new hero named Kaya. The character is a witch in the setting, so naturally we also adopted some Japanese elements. We all know that jazz has a profound influence on Japanese music, whether it is pop music, animation, movies or games, you can often see some Jazz, Funk elements in it. So when we produced Cherry Blossoms Fluttering, we also used jazz as the main tone of music, allowing Western pop music to blend with traditional Japanese japanese instruments.
Zhang Zhiwei life photo
We used instruments such as the flute, shakuhachi, and shamisen in our music, and let professional musicians play them. The musicians who played them all had a very high memory, and they showed a lot of new feelings while performing perfectly, which was very amazing. In this case, guitar, bass, and wind music are played by Chinese first-line musicians. In the composition and arrangement part, Mr. Iwatori Tokuyuki used his superb professional ability to improve the quality of music again.
Therefore, it can be said that this music creation is a transnational in-depth artistic cooperation, in addition, in order to restore the weapons, costumes and action sounds of the new heroes Kaya and Nanami, we have used film-level standard equipment to record in the top voice studios in China. In short, the hard work of the entire audio team for several months is to make players have a new experience in the game and make users play more enjoyable.
In the cover of the album Cherry Blossoms Fluttering, you can also see a lot of content related to the game's plot, because of some of the settings in the game, these new heroes who have just joined the world of State of Survival are having concerts, so this cover is also what I like, it not only reflects the narrative expression we just mentioned, but the atmosphere created by this cover is also very attractive to me.
In addition, we have a project, its whole worldview is an alternate fantasy world, in this project, I also spent a lot of energy to set his music, at that time we also worked with Japanese artists to complete, the composition of the music was written by Iwatsuru Tokuyuki and me. During that time, the work was actually quite hard, I had to go to meetings during the day to do projects, sound effects or build interactive projects, and discuss with my planning colleagues how to do the music part, and at night I needed to calm down to do content creation. Because these projects have been followed up from 0 to 1, for me and my colleagues in the project team, these projects are like our own children, and we all want to do each project very carefully.
At the time, the whole project was about 30 tracks in size, and the two of us wrote about half of it each. These tracks are dominated by symphonic and fantasy world styles. The symphony here refers to the large orchestra, and occasionally there are some small choreography of chamber music, or some solo passages.
Symphonic, fantasy world style music recording site
In this project, when we actually landed, it included details such as the early setting, communication with the project team, consultation, composition, and so on. In this process of cooperation, I did not communicate with the teachers as a party A, but really devoted myself to it, and had in-depth communication with the teachers in terms of music, composition and skills. Here, I am also very grateful to the industry senior Mr. Iwatori Tokuyuki and his team, whether it is at the level of musical score or composition skills, I have learned a lot of new knowledge and insights from everyone.
While recording this project, we went to tokyo's largest recording studio and played with the Tokyo Symphony Orchestra, which is very well-known in Japan. I think it's a pity if you don't let everyone hear a little bit in advance, so I edited a small video in advance for everyone to enjoy, and in the future, after the project is officially launched, we will also make it INTO OST and distribute it on mainstream music platforms around the world.
So in the final epilogue, I would like to give you a summary, today I mainly talk about 3 parts:
The first part is the design concept of game audio. Here are some of the experiences I have learned in the past 15 years of teaching, game audio is really not just about sound effects, in addition to programming, we will integrate modern musical styles, forms of expression and composition techniques. For students who want to enter this industry, the learning growth path map we mentioned earlier can help everyone find their goals faster and be more clear about what skills are required for game audio.
The second part is the game's audio enhancement strategy. Here we share a lot of examples, including the state of survival and DC Joker panoramic sound audio production, the production of new hero music, the listing of albums on major music platforms around the world, and the awards we have won. The core of what I want to tell you here is to improve the game audio strategy, you can learn from film and television and records.
The third part is transnational artistic collaboration. My philosophy of doing game audio is not to go straight to a professional and outsource all the production parts to him. It's that I'm willing to get deeply involved, and I'm also from the record industry to the game industry. I'm familiar with all aspects of production, so I think I can make products that are more attentive. If you are a top artist in the industry, I will try to help you complete your artistic creation better.
Zhang Zhiwei participated in the "Spartan" physical training organized by Qujia Interactive Entertainment
In my opinion, cross-industry learning is very important. Many of the game audio improvement strategies I just mentioned are derived from the technology and narrative of classic industries such as film and television and records, which have a longer history than game audio, and have reached this level through the accumulation of experience of countless people. So, the more you know about these cross-cutting areas, the more it will help you on the track of game audio.
Of course, there is also a small episode here. There are some students who are purely doing film and television or records who want to come to the game industry, then they may encounter a big difficulty, and there are many differences between the work logic of games and film, television and records. Games from the development of the project to the official launch, usually take 2-3 years, so the game audio design is to face the future market and target groups to produce, how to make what you do now can compete with the opponent in a few years, can make people like, I think it is necessary for practitioners to think. For me, I think it is to do every step that can be done in the present carefully, with ingenuity, and there will be achievements in the future.
Zhang Zhiwei life photo
The above is my thinking on "how to do a good job of game audio", my sharing is here, thank you.