A light field acquisition system photographed in the VR laboratory of the School of Optoelectronics of Beijing Institute of Technology. Photo by Xinhua News Agency reporter Ren Chao
Xinhua News Agency, Beijing, November 18 (Reporters Zhao Xu, Qiang Lijing, Ren Chao) With the melodious sound of Jinghu, three spotlights lit in turn, and 26-year-old Mei Lanfang, holding a folding fan, walked towards the center of the stage with a light step... In the avatar lab of Beijing Institute of Technology, "Digital Mei Lanfang" appeared on the computer screen.
"Now on stage is Mr. Mei Lanfang's twin digital person. Based on the 26-year-old Mr. Mei Lanfang, we use information technology and artificial intelligence technology to reproduce this Chinese Peking Opera master. Weng Dongdong, a researcher at the School of Optoelectronics of Beijing Institute of Technology, said that in 2020, the Central Academy of Drama and Beijing Institute of Technology jointly launched the public welfare scientific research project to create a "Peking Opera digital person" that can be interacted in real time.
From the face, posture, voice tone, to dress, props, a smile... By digitizing and visualizing the details of real-life people, the "people" in the digital world are also "everything". However, due to the inability to collect Mr. Mei Lanfang's real three-dimensional data source now, it is not easy to restore his character state to the greatest extent.
In the VR laboratory of the School of Optoelectronics of Beijing Institute of Technology, researchers collect face data in the light field acquisition system. Photo by Xinhua News Agency reporter Ren Chao
"We collected a large number of relevant historical photos, and then asked the teachers and students of the Central Academy of Fine Arts to repeatedly polish and modify the photos, made a physical portrait sculpture, and then scanned the statue with a high-precision laser scanner, so as to obtain the basic facial structure of Mr. Mei." Weng Dongdong said.
The sculpture itself can not visualize the skin details, hair details, eye charm and other microscopic features, the team found a person who looks exactly like Mr. Mei Lanfang, collected his facial data, and captured the basic expression, "transplanted" on the "digital Mei Lanfang", and finally under the joint control of plastic surgery experts in public hospitals, to achieve "accurate reproduction of the face".
Song Zhen, director of the High-tech Research Center for Traditional Drama Digitization of the Central Academy of Drama, introduced that in order to restore Mr. Mei Lanfang's dress that year, the team consulted a large number of documents and visited a number of tailor shops in Beijing. "We found that Mr. Mei's costume at that time was sewn with gold thread, and now this technique has been basically lost. However, we found the cloth samples left in the Republic of China, according to the guidance of experts in the field of Peking Opera, according to the collar and cloth pattern of Mr. Mei's casual clothes at that time, we found the most suitable cloth samples, so that we scanned and obtained the data of the cloth, and first completed the creation of digital casual clothes. ”
"At the beginning of the project, the technical team did not expect that the knowledge was so deep." Weng Dongdong said that many details need to be explored in depth, including the costumes and helmets that Mr. Mei Lanfang used during performances, and at the same time, Peking Opera artists will be invited to imitate Mr. Mei Lanfang's dramatic performances to complete motion capture and make the performance of "Digital Mei Lanfang" more realistic.
"Digital Mei Lanfang" static digital model (left) compared with Mr. Mei Lanfang's photo (right) (courtesy of the interviewee). Photo by Xinhua News Agency reporter Ren Chao
Regarding sound processing, Weng Dongdong introduced that most of the existing mr. Mei Lanfang's audio and video materials are of poor sound quality, and the next step will be to consider restoring Mr. Mei's voice characteristics in the form of real imitation, collecting voice data, and "binding" on "digital Mei Lanfang".
"In the future, we hope to create an immersive interactive, virtual and real fusion of 'digital Mei Lanfang' character scene application, the audience put on the VR helmet, you can enjoy Mr. Mei Lanfang's Peking Opera performance, but also can interact with Mr. Mei in real time, to achieve science, teaching and other tasks." Weng Dongdong said.
Mei Lanfang's grandson Fan Meiqiang said: "My grandfather died in 1961, more than half a century since the present, with the current high-tech techniques, the past artists vividly presented in front of modern people, to introduce to young people, is a very meaningful thing." ”
"Through the collision of science and technology with China's excellent traditional culture, let the Peking Opera masters reproduce, better mobilize the audience's curiosity about Peking Opera, truly bring people closer to the Peking Opera masters, understand the essence of China, and let culture transcend time and space and transcend the country." Weng Dongdong said.