OpenAI cut off supply, Chinese companies: Thank you for the invitation, it has reached the top

Coldplay Labs

2024-07-01 10:33Creators in the field of technology

The full text is 2752 words, and it takes about 8 minutes to read, so help me highlight the key points

Highlight the point

01OpenAI has stopped providing APIs to Chinese mainland, Hong Kong and other regions since July 9, which were once the targets of sanctions.

02Due to OpenAI's supply cut, domestic companies have launched preferential prices to attract customers, such as Alibaba's Tongyi Qianwen main model calling API at only 1/50 of GPT-4.

03 However, domestic large models have performed well in various rankings, such as the top 20 on the C-Eval list are all large models from the mainland, and GPT-4 is at the 21st.

04As the first large-scale technology enterprise in China to do open source, Alibaba Cloud has open-sourced 8 models and 5 models, with parameters ranging from 50 million to 72 billion, and its performance is close to GPT-4, the first closed-source model.

Technical support is provided by Tencent Hybrid Model

OpenAI cut off supply.

Since July 9, countries and regions including Chinese mainland, Hong Kong, Russia, North Korea, and Iran can no longer access their APIs.

Looking at the list, it is all the targets of US sanctions, which is, of course, a political issue.

However, if you continue to pay attention to OpenAI, this decision is not surprising at all.

Not long ago, OpenAI's CEO Altman disbanded the security team, Super Alignment, which was led by Ilya Sutskever, the former chief scientist of OpenAI.

Ilya then left. Soon, Altman formed a new security team, led by former NSA Director Paul M. Nakasone.

When OpenAI becomes CloseAI, what impact will it have on the entire industry?

Let's take a look at what this wave of "supply cuts" may bring.

01

The so-called "API" refers to the "Application Programming Interface".

You can think of API colloquially as a restaurant menu. You can use it to order, but you don't know how it's made.

Do you still remember when ChatGPT first came out, and countless AI companies emerged? They are the customers, and it is OpenAI that stir-fries, and then they package the stir-fried dishes and sell them to us outside the restaurant.

So in essence, they're shell companies, and it's OpenAI that really produces.

It is conceivable that cutting off the supply for these companies is tantamount to taking a salary from the bottom of the kettle and taking away the guys who eat.

But from another point of view, OpenAI, a giant in the AI industry, actually took the initiative to give up the market, and it would be a fool not to make money.

No, all the masters in China immediately rushed up to grab this wealth like a hungry tiger, and various "moving plans" were released, and the service should not be too thoughtful.

For example, in addition to providing exclusive migration services, the price of the main model calling the API is only 1/50 of GPT-4. This is still the price when Tongyi Qianwen is comparable to GPT-4.

Others, including Zhipu, Xunfei, Baidu, Baichuan, etc., all offer quite favorable prices.

Therefore, what is in front of them now is not a matter of life and death to be or not to be, but how to choose the best one from a crowd of "spare tires".

02

What makes a good model?

As Napoleon, the strongest man in European history, said, a model that does not want to take the first place is not a good model.

Just as students compare their scores through exams in various subjects, the ability of large models is also regarded as the results of questions.

Students have a variety of projects such as language and mathematics and foreign science and literature comprehensive, and large models have reasoning ability, mathematical ability, programming ability, language ability, multimodal ability and other projects.

For example, the most commonly used MMLU dataset, which covers 57 disciplines such as STEM, humanities, and social sciences, is a dataset that is commonly used to test model knowledge and reasoning ability.

In addition, there is C-Eval for the Chinese exam, MATH for the Olympiad (good name) and so on.

Naturally, every model will strive for a good score in the exam. But just like student exams, some people are strong, and some people have their own methods.

As of June 28, 2024, the top 20 on the C-Eval list are all large models from the mainland, and GPT-4 is ranked 21st.

This is not in line with my actual experience, although GPT-4's ability is not crushed as before, but it is not ranked 21st. It can be said that this list is somewhat distorted.

There are many reasons for this phenomenon.

The first is that with the continuous upgrading of large models, some topics have become relatively simple. Just like in the past, everyone was a primary school student, and everyone didn't get a high score in the junior high school exam. But after more than a year of study, everyone's level has risen to high school students, and if you do these questions again, you can get more than 90 points, so this test question can't distinguish everyone's level well.

Secondly, the closed book becomes the open book. Although none of these questions are publicly available, I put the exam on hold every day for dozens of days in a row. Then the exam is like brushing the question bank, and the questions of the exam are gradually being made public. Later, the large models directly used these open questions to train, and when they took the test again, it was equivalent to an open-book exam. Even if it is as difficult as a math competition, it is not impossible to get a result.

Of course, the quality of the questions themselves is also important.

This is the highest score on the Open LLM Leaderboard released by Huggingface, a well-known open source community. It can be seen that from September 2023 to May 2024, the performance of the large model in various subjects has been continuously improved, and they are all close to the dotted line, that is, the human level.

Of course, this achievement does not mean that the large model is already the same as a person, but it means that this volume has been made rotten.

所以，Huggingface在6月推出了一套新试题，升级版的Open LLM Leaderboard v2。

This set of test questions is much more difficult than the previous version, for example, the GPQA data is full of graduate-level knowledge, and doctoral students in biology, physics, chemistry and other fields are specially found to write the questions.

Objectively speaking, this wave is very sincere, and it does not leave any loopholes for those big models to drill.

The results of each candidate's performance of this set of papers came out quickly, and the list was very unexpected:

There are many old acquaintances in it, such as the popular fried chicken, "France's OpenAI" Mistral, "the most powerful open source model in history" Llama3, and Tongyi Qianwen Qwen2-72B (72 billion parameters) that defeated Llama3.

I was a little surprised to see our domestic Tongyi Qianwen on this French list.

I went to look at the detailed report card again, and Qwen2's Mathematics (MATH), Specialized Knowledge (GPQA) and Long Context Reasoning (MuSR) were the dominant subjects, especially Mathematics, which was 6 points higher than the second place. Oh sorry, but second place went to Qwen2, which was not fine-tuned.

This achievement was praised by the CEO of Huggingface:

I went to see another list, LiveBench AI, a big model benchmark led by Turing Award winner Yann LeCun, one of the Big Three in AI.

Qwen2-72B排名第8。

But this is an overall list that includes all closed-source models, and Qwen2-72B is the only open-source model in the top 10.

There are other lists, Qwen2 is basically the overlord in open source, so I won't list too much here.

This shows that Qwen2 is very capable of solving problems. But is it a problemalist or is it really powerful? This requires testing by front-line developers.

After testing it on Reddit, the developers gave a positive review:

The initial test was very good, there was a question and the other models were wrong, only Qwen7B was right

Another Italian developer even said that "too strong to be true":

I went to look at Twitter again, and the developers were very satisfied with Qwen2's capabilities after testing, such as Percy Liang, an associate professor of computer science at Stanford:

At this point, I don't think there's anything to question about Qwen2's strength.

03

On the track of open source, the competition is never smaller than that of closed source.

There is a magnetic download link for the French unicorn Mistral to throw out its own large model Mistral 7B for free.

The developers took a look down and found that the performance was not much weaker than the 13 billion parameters of Llama 2, and with a little fine-tuning, it could run on a single graphics card.

后来他们又推出了更大参数量的 Mixtral 8x7B，性能追平了Llama2和ChatGPT3.5。

This has allowed Mistral to raise more than 1.1 billion dollars, counting the money until it is soft.

Two months ago, Meta, which is deeply involved in open source, officially released Llama3 8B and 70B, an open-source model that cost an estimated $15 million to train, to regain the Iron Throne.

Meta officially certified it as "the strongest open-source model so far".

At that time, there was a voice from the outside world that said, "Only GPT-5 can suppress Llama3." ”

But GPT-5 didn't come, Qwen2 came first, is it surprised, is it not surprised?

In fact, Alibaba Cloud is the first large-scale technology enterprise in China to do open source. In August 2023, they open-sourced Qwen7B.

到现在,他们一共开源了Qwen-VL、Qwen-14B、Qwen-72B、Qwen-1.8B、Qwen-Audio、Qwen1.5的8款模型和Qwen2系列的5个模型,参数从5000万到720亿,可谓是全家桶了。

Seeing this, some people may ask, Alibaba Cloud still has its own closed-source model, is it hesitant to open source?

But Qwen2's "getting ahead" in the world this time proves that Alibaba Cloud is serious about doing open source!

With the release of GPT-5 repeatedly delayed, and now the news has been postponed until the end of next year, there is a high probability that OpenAI has not yet found a way to make GPT-5 greatly improve on the basis of GPT-4.

At the same time, the performance of the open-source model represented by Qwen2 is constantly approaching GPT-4, the first closed-source model.

Sam Altman, who turned OpenAI into CloseAI and from a non-profit to a profit, will he have a trace of regret when he sees the performance of Qwen2?

View original image 239K

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

•