Author丨Linfeng
Editor丨Sea waist
Source丨SakanaAI official website
On June 14, SakanaAI, founded by David Ha, a well-known researcher at Google, and Llion Jones, one of the authors of the AI seminal paper, announced the completion of a $127 million funding co-led by Khosla Ventures, Lux Capital, and NEA. The company was established in July 2023 and has raised two rounds of financing in the past six months, including NTT (Japanese telecommunications giant) and Sony.
According to the Asahi Shimbun, SakanaAI is valued at about 180 billion yen ($1.142 billion) after a new round of funding, making it Japan's fastest-growing AI unicorn in less than a year.
"Every country needs to own its own intellectual production, which is why there is the concept of 'sovereign artificial intelligence'. It codifies your culture, your social wisdom, your common sense, your history, so you have to get it, refine it, and have your own national intelligence, and you can't allow anyone else to do that. ”
Huang mentioned sovereign AI at the World Government Summit in February. If a small country wants to focus on an artificial intelligence, the first thing to do is to build the "infrastructure" and weave its own culture and language into its own large language model.
Source: YouTube, WGS2024 video screenshot
In 2023, they will use the power of the whole country to compete for the AI track, major famous companies will train AI models, and make huge bets on the "dream team" SakanaAI, aiming to surpass China and the United States. Not only Japan, but also Singapore, India, the United Arab Emirates and other countries are investing heavily in sovereign AI.
In less than 1 year, it is valued at $1.1 billion
SakanaAI, known as the "Tokyo AI Dream Team", is home to top industry talent and innovative start-up ideas. More importantly, they are a 100% registered company in Japan.
SakanaAI's name is taken from the Japanese word for "fish", their logo is a school of fish, and they aim to develop artificial intelligence based on the principles of nature, where many small elements work together to produce big results.
Source: JAFCO
In March, SakanaAI announced the release of three Japanese models: EvoLLM-JP (large language model), EvoVLM-JP (visual language model), and EvoSDXL-JP (image generation model), which can be tested online on Huggingface. Judging from the popularity, none of the three types of models has been downloaded more than 1,000 times.
来源：HuggingFace
From the perspective of effectiveness, the generation ability of the model has also been improved. Using his EvoSDXL-JP demonstration model, the authors found that there were still errors in the species and sex, as well as errors in the display of eyes, legs and feet, using the Japanese word "Yamato Nadeshiko" to run the image. In terms of food generation, the model generation effect is better, and it is more artistic.
Figure 1 inscription: Yamato Nadeshiko, picture 2 inscription: Ukiyo-e style delicious food. Note: The input language is Japanese. Source: EvoSDXL-JP test model generation effect
However, the speed of the model is very fast, about 5s per sheet, which is in line with its reasoning only 4 steps, and the speed is 10 times faster than the existing Japanese model, and the style of the picture is close to the inscription "Japanese ukiyo-e".
In just a few months, the SakanaAI team has achieved results. In June, they began working on the new question: "Can we also use LLMs to come up with better algorithms to train LLMs themselves?" and if so, this is a way to optimize new algorithms with minimal human intervention. They will collaborate on this project with the University of Oxford and the University of Cambridge.
来源:SakanaAI Blog
Founded in Tokyo in July 2023 by David Ha, Llion Jones, and later Ren Ito, all three of whom are well-known names in the industry, is a big hit on the side.
左David Ha，右Llion Jones，来源：Lux Capital
David Ha is a multidisciplinary Ph.D. from the University of Tokyo, having spent 8 years at Goldman Sachs, where he was the head of research at Google Brain Japan, and a 6-year research scientist at Google. He has published several papers on neural networks, deep learning, image generation, project presentations, and papers in top conferences/journals such as ACM, ICCV, ICLR, AAAI, etc. As early as 2018, he had a paper related to deep learning of Japanese classical literature published in NeurIPS.
David was the head of research at Stability AI, but his departure was confirmed less than a year ago, and at the same time, Stability AI's top management changed, and his then-COO Ren Ito also left, and both are currently working at SakanaAI. He revealed that part of the reasons for starting a business in Japan are that Japan has not yet developed a competitive model. Second, it is difficult to achieve differentiation when starting a business in Silicon Valley.
Ren Ito, COO of SakanaAI, is a graduate of Tokyo University, an LL.M. and a Senior Fellow at New York University's School of Law, and a M.A. from Stanford. He began his career as a diplomat in politics as the principal deputy director of Japan's Ministry of Foreign Affairs, and later in the business world, helping Japanese second-hand platform Mercari plan its IPO before moving to Stability.
At Stability, he was exposed to the Western gaze and cultural biases generated by AI models, and Ren Ito argues that every culture, every nation, and every country should build its own foundational model. Running SakanaAI in Japan can be regarded as Ren Ito's wish.
The 2015-2016 World Report, which was first reported by the World Heritage Office in 2014, said it was "very important" that the 18-day-to-date event was being considered a "new and important event" for the 2016 world.
Another Llion Jones, one of the "Transformer Eight", graduated from the University of Birmingham and worked at YouTube and Google before starting the company last year. He is the fifth author of the seminal paper "Attention Is All You Need" (each author is equally important), which introduced the Transformer deep learning architecture that became the basis for the development of products such as ChatGPT. At a time when there are only more than 100 top AI talents, it is of great significance for Llion Jones to be at SakanaAI.
According to IT Media, SakanaAI announced a list of founding members after raising funds in January, and the Google content is extremely high, with 5 out of 10 people coming from Google's main AI research institutions, and 3 from Stability.
David said in an interview that in February this year, they received 300 applications within four days of posting their job postings online, and 70% of them were from non-Japanese regions such as the United Kingdom, the United States, and Australia.
Source: IT Media
The team is gorgeous, so they raised $30 million in funding in the first seven months of their existence, valuing them at $200 million. SakanaAI received a round of funding in January 2024, in addition to Silicon Valley VC Lux Capital and Khosla Ventures, as well as Japan's NTT Group, KDDI, Sony Group, and JAFCO Group, as well as personal investments from Jeff Dean, Google's chief AI officer, Clem DeLang, founder of Hugging Face, and Alex Wang, founder of Scale AI.
At that time, their model was still in the development stage, and it was even a concept.
According to Ren Ito, they are developing an innovative path of generative AI, using the concept of "nature-inspired intelligence" to achieve low cost and strong performance. "Imagine a honey bee colony, where worker bees leave the hive in search of resources, and they can communicate with each other about the location of high-quality resources and optimize the use of resources."
The expert explanation given by the team is that their algorithm can help automatically create AI models that process language or images by mimicking natural selection when combining, testing, and discarding data, reducing the amount of human input. In stark contrast to OpenAI's multibillion-dollar training methodology, David envisions, "We want to reduce the time to days or weeks with less computation, while achieving similar levels of performance." ”
According to the official website, they use "evolutionary model merging" to develop large models with both performance metrics and robustness.
The first benefit is the saving of computing power. According to Bragi Analysis, "Model Merging" is the creation of high-performance models by combining the knowledge and capabilities of multiple small models at a fraction of the computational cost. The resulting unified model can match or exceed the performance of several large models.
Source: Medium
In addition, this process is similar to the fusion of multiple open-source models, similar to natural evolution to reproduce the next generation of excellent models, without human intervention, and can be better than blue.
Evolutionary model merging effectively automates model building, not only for language generation, but also for image generation. For example, in the construction of their EvoSDXL-JP, a Japanese-specific image generation model was combined with the English-based image generation model to obtain a Japanese-compatible image generation model (corresponding to model 6 in the table), and then merged it with the faster image generation model, to obtain the EvoSDXL-JP (corresponding to model 7 in the table), a high-speed image generation model that supports Japanese.
来源:SakanaAI Blog
SakanaAI's methodology of rapid development, cost reduction and efficiency improvement is in line with the speed that Japan desires. After the release of its 3 large models, another new financing was launched quickly. Even in March, Sakana AI's initial merger model did not work well, and the Japanese government and business community chose to continue to invest.
In April, SakanaAI was one of seven labs to receive a grant from the Ministry of Industry's "Generative AI Accelerator Challenge" and won a supercomputer royalty grant from the Japanese government and partnered with giant companies such as NTT and Sony to build Japan's AI ecosystem. In addition, SakanaAI is also targeting government agency work and subcontracting in the defense industry.
"At the very least, [Japanese] companies should have their own base model, even if it doesn't perform particularly well," David jokes, "and they need to have their own fax machine." ”
According to Crunchbase, Sakana AI closed a 20 billion yen (about $143 million) financing co-led by Lux Capital, NEA, and Khosla Ventures in June, with a post-investment valuation of about $1.15 billion. It is precisely because of this financing that Sakana AI has become an emerging unicorn without a clear core product.
SakanaAI is the "star team" chosen by Japan on the way to the promotion of building a local large model, and it has attracted much attention when it develops AI with national efforts, and it has received full expectations and sincerity.
Sovereign AI is expected to "earn" $10 billion
Not only Japan, but countries around the world are investing in sovereign AI.
Chinese and American startups and giants are "rolling up and dying" on the big model. China has developed the second largest number of large models in the world, and it goes without saying that the country is a strong support for the development of artificial intelligence.
According to foreign media reports, some countries that believe that they are lagging behind in the mobile Internet and cloud computing revolution must try to preserve their local culture and national security in this wave of AI.
Pablo Chavez, who is now at the Center for a New American Security, said: "Some governments are extremely anxious about how the generative AI revolution will impact their economies."
This anxiety is hitting the pain points of governments, and only action can overcome it.
In Asia, in April this year, the governments of South Korea and Japan introduced policies to invest in the AI field. South Korea has invested a significant amount of about 9.4 trillion won ($6.94 billion) to support its goal of becoming a major supplier of semiconductors. In addition to the above-mentioned Naver's Hyper CLOVA, LG's Exaone, KT's Mi:DEUM (the first LLM developed by a Korean company), etc., large Korean e-commerce and cloud service companies have cooperated with AI chip companies to participate in government projects.
Singapore is a major investor in sovereign AI and a big buyer of NVIDIA. Nvidia's third-quarter 2023 results noted that about 15% ($2.7 billion) of its revenue came from Singapore, which is related to its own multiple data centers and cloud service providers. According to Crunchbase data, there are about 1,354 AI, machine learning, data and other related business companies in Southeast Asia, of which 866 are located in Singapore.
In terms of policy, Singapore introduced the policy NAIS 2.0 in December last year, and will invest more than US$1 billion in AI computing, talent and industry development in the future. Moreover, Singapore plans to take the lead in establishing an LLM based on language training in Southeast Asia.
Turning to India, where the power infrastructure is relatively backward. According to the data, there are less than 10 companies in the field of AI large models in India, and the self-developed model of unicorn Krutrim only allows more than 400 characters to be entered. The Indian government directly approved the national-level "IndiaAI" project in March, with a $103 billion ($1.24 billion) investment aimed at achieving the goal of "making AI work for India", driven by the concept of sovereign AI in the hope of building a comprehensive ecosystem.
Middle Eastern tycoons choose to spend money on advanced technology. The UAE established the AI and Advanced Technology Council (AIATC), and in March this year, AIATC announced the establishment of an investment company, MGX, which is a joint venture between Mubadala and G42 to deploy cutting-edge technologies, and the investment is concentrated in three areas: artificial intelligence infrastructure, semiconductors and artificial intelligence applications, with assets reported to exceed $100 billion in the next few years.
Saudi Arabia's Public Investment Fund (PIF) plans to invest $40 billion in AI, hoping to support a range of startups, chip companies, and sprawling data centers, and even Saudi Arabia's own AI companies.
Kenya signed a deal with the G42 in May to build a $1 billion data center in the country to train an AI model in Kiswahili and English using Kenya's geothermal energy (power).
Canada, which has no shortage of AI talent, invested 2.4 billion Canadian dollars (about 1.77 billion U.S. dollars) in April, 2 billion yuan to build infrastructure, and the rest of the funds to empower startups and research and development. A package to develop AI was launched to ensure Canada's AI dominance.
On the other hand, Europe's artificial intelligence is also secretly competing. Huang Songyan, an investor in Linear Capital, said in an interview that Europe is the birthplace of many original technologies, and many big names in the field of AI and machine learning are hidden in all corners of Europe. For example, the French version of OpenAI Mistral created Mistral Large in Paris, and the inference task is not inferior to GPT-4.
In April, the European Commission announced that it would invest more than 65 million euros in artificial intelligence, of which 50 million will be used to expand large model capabilities. France, Germany and Italy are all making efforts, with French President Emmanuel Macron calling on Europe to establish a public-private partnership to buy chips and increase Europe's share of global GPU deployment from 3% to 20% by 2030 or 2035.
In general, governments and companies invest in local language models, computing power, and data centers to prevent the development of advanced AI technologies from being constrained by other countries. Countries with weaker models are building foundations, while countries with stronger models are building moats to strengthen their positions.
Among them, the people who make money turn the concept of sovereign AI into the USP of the product. In the past six months, Huang has participated in the UAE's WGS, met with Indian Prime Minister Narendra Modi in September last year, and met with the prime ministers of Japan and Singapore in December to promote investment in AI by the government or government-linked units, computer infrastructure and telecommunications companies.
In fact, when sovereign AI comes out of the mouth of businessman Jensen Huang, it is in a profitable nature. Back in 2019, Nvidia had an "AI Nations" program to help countries build sovereign AI, including ecosystem support and workforce development. Governments are spending money on new data centers and retrofitting old ones with specialized computer chips, most of which come from Nvidia.
According to the Wall Street Journal, Nvidia said in May that its sovereign AI strategy is expected to bring in nearly $10 billion in revenue this year, up from a blank last year. Nvidia reported quarterly revenue of $26 billion, nearly half of which came from large cloud computing companies that use its chips.
Angelo Zino, an analyst at CFRA, concluded: "[Nvidia's] cloud-oriented sales are clearly not going to climb sustainably, but it can profit from the AI boom in other ways." To create higher revenue, sovereign AI is a new revenue growth point. ”