【Global Network Comprehensive Report】In the just-concluded ice and snow event, the AI sign language anchor created by Baidu Intelligent Yunxiling for CCTV News was officially inaugurated, and she conveyed the passion of ice and snow sports in real time with fluent and professional sign language services. However, in daily life, hearing impaired people still face many difficulties in obtaining information easily, and the scarcity of barrier-free windows needs to be solved urgently.
On March 3, Baidu Intelligent Yunxiling officially released the "AI Sign Language Platform", which can realize the ability to generate sign language synthesis videos at the minute level and live broadcast by sign language anchors in real time, and proposes a new scientific and technological solution to the problem of the popularization of sign language services. At the same time, Baidu Intelligent YunxiLing also released the "AI Sign Language Platform All-in-One Machine", which can quickly establish barrier-free windows through large-scale deployment, plug-in and simple operation, suitable for hospitals, banks, stations and other public places, alleviating the shortage of barrier-free windows.
Wu Tian, vice president of Baidu Group, said that artificial intelligence presents new characteristics of integrated innovation and lowering the threshold, the comprehensive use of multiple technologies of digital people to bring new experiences, AI blessing to reduce the production and operating costs and thresholds of digital people, AI sign language platform will allow more hearing impaired people to enjoy the convenience brought by technology.
(Wu Tian, Vice President of Baidu Group)
AI Sign Language Platform + All-in-One Machine, Build "Barrier-free Window" Anytime, Anywhere
At present, there are more than 27.8 million people with hearing language disabilities in mainland China, while there are less than 10,000 professional sign language interpreters, and the huge demand gap needs to be filled urgently.
In the "14th Five-Year Plan for the Construction of Barrier-free Environment" issued by the state, it is clearly proposed to improve the basic barrier-free public service system and strengthen the in-depth application of information barrier-free services. The "AI Sign Language Platform" and "AI Sign Language Platform All-in-One Machine" released by Baidu Intelligent Yunxiling can easily and quickly realize sign language translation services in different online and offline scenarios to help the construction of barrier-free environments.
For online scenarios, the "AI Sign Language Platform" has four major functions: "Video Sign Language Synthesis", "Live Sign Language Synthesis", "Text to Sign Language", and "Speech to Sign Language", which can realize the integration of ordinary video into sign language video, the addition of sign language screens in real-time live broadcasts, text translation into sign language, and real-time voice translation into sign language. The AI sign language platform can be installed in various apps, websites, and mini programs, so that the hearing impaired can easily realize various needs such as online social networking, entertainment and leisure, and course learning.
For offline scenarios such as hospitals, banks, airports, etc., the advantages of integrated products are gradually being amplified. Baidu Intelligent Yunxiling released the full offline all-in-one machine V3 and the end cloud combined with the all-in-one machine P3, equipped with the core functions of the "AI Sign Language Platform", which can realize ASR speech recognition, speech translation, portrait rendering and other multi-functions, so that the "AI Sign Language Platform All-in-One Machine" can be produced quickly and in batches like mobile phones and computers, and penetrate into all corners of life.
In the future, through the "AI sign language platform all-in-one machine", the ordinary service window can be directly converted into a "barrier-free window", and the hearing impaired can easily realize the full range of processes such as registration, consultation, payment, and drug collection in the hospital, and can independently complete check-in, check-in, change, and transfer at stations and airports, and can also communicate with various emergencies in real time and equally in special scenarios such as courts and police stations.
Behind the highly understandable sign language expression is the innovation and precipitation of technology
Compared with traditional translation, the sign language translation process is more complex, mainly because sign language translation is not translated word by word in the order of daily speech, but language refinement and word order adjustment according to the meaning of the sentence. Sign language is a real small language, the amount of data is difficult to support training, the acoustic environment of various scenes is complex and difficult to deploy, and it also needs the cooperation of lips and expressions to assist in understanding, which puts forward layers of challenges for the construction of AI sign language platforms.
In order to achieve highly understandable AI sign language ability, it is necessary to solve three major technical points: the clarity of speech recognition, the accuracy of sign language translation, and the fluency of sign language movements. After many tests such as the opening ceremony and the broadcast of the event, the AI sign language anchor also confirmed that the "AI sign language platform" has successfully solved these three major problems, and achieved sign language services of "hearing clearly", "turning accurately" and "playing well" with industry-leading full-link AI capabilities.
First of all, in terms of speech recognition clarity, Baidu Intelligent Yunxiling-AI Sign Language Platform realizes end-to-end modeling speech recognition through the SMLTA speech recognition algorithm model, and the near-field speech recognition rate on the mobile phone side can reach more than 98%. Gao Liang, director of Baidu's speech technology department, said that based on Baidu's self-developed deep learning algorithm, it can also be targeted training for word accuracy in different fields such as tourism, hospitals, and justice, and the word accuracy rate can reach 96%, making the AI sign language platform all-in-one machine better applied to different scenarios.
(Gao Liang, Director of Baidu Voice Technology Department)
Secondly, in terms of the accuracy and refinement of sign language translation, He Zhongjun, chairman of Baidu Artificial Intelligence Technology Committee, said that Baidu developed the first controllable sign language translation model based on neural networks, and jointly formulated natural sign language labeling specifications with sign language linguistics experts, special education experts and the Deaf Artificial College of Tianjin University of Technology, and built a large-scale natural sign language translation corpus, so that the model can learn sign language translation knowledge from real data, such as word order adjustment, word mapping and length control. This results in natural sign language that conforms to the habits of the hearing impaired.
(He Zhongjun, Chairman of Baidu Artificial Intelligence Technology Committee)
Yuan Tiantian, deputy dean of the School of Deaf Artificial Labor of Tianjin University of Technology, shared at the press conference that deaf students can feel more empathy and do not miss every detail. Due to the tight time and heavy tasks, the students also spontaneously called on nearly 500 deaf students from other colleges and universities to complete the labeling of the corpus together, and make efforts to bridge the social information gap with a sense of responsibility and honor.
(Yuan Tiantian, Vice Dean of School of Deaf Artificial Labor, Tianjin University of Technology)
Finally, in terms of the fluency of sign language movements, the AI sign language platform refines nearly 11,000 sign language movements based on the specifications of the National General Sign Language Dictionary through the "action fusion algorithm", so that the ups and downs of each action are naturally connected, bringing coherent expression like real sign language. Coupled with the training of 4D scanning technology, the EXPRESSION of AI sign language digital people is natural and vivid, and the accuracy of mouth shape generation is as high as 98.5%.
In the experience after the meeting, the reporter saw that only the recorded video needs to be transmitted to the "AI Sign Language Platform", and after processing, it can output the video that has been integrated with sign language translation, and it is very convenient to spread it. In the staff presentation, all that is needed is to speak into the microphone, and later the digital person on the screen can translate the semantics in sign language. However, the staff said that this function is not yet perfect and is still under continuous development.
The relevant person in charge said that Baidu's intelligent Yunxiling's "AI sign language platform" is mainly based on serving the public needs of airports, railway stations, hospitals or media institutions at this stage, and the real-time translation function of language and sign language will continue to be developed, and when there is the latest progress, it will be announced to the public in time, hoping to help the hearing impaired people to communicate more smoothly on a daily basis.
The construction of barrier-free environment directly determines the quality of life of disabled people, and is also a symbol of national and social civilization. The release of Baidu Intelligent Yunxiling's "AI Sign Language Platform" and "AI Sign Language Platform All-in-One Machine" will quickly solve the problem of the scarcity of barrier-free windows in various scenarios and help the sustainable construction of social civilization and a better life.