AIxiv is a column that publishes academic and technical content in the heart of the machine. In the past few years, the AI xiv column has received more than 2,000 reports, covering the top laboratories of major universities and enterprises around the world, effectively promoting academic exchanges and dissemination. If you have a great job to share, please feel free to submit or contact us. Submission mailbox:[email protected];[email protected]
Since OpenAI's Q* project was exposed, it has sparked a lot of discussion in the industry. According to the available information, the Q* project is regarded as a major attempt by OpenAI to explore artificial general intelligence (AGI), which is expected to bring revolutionary breakthroughs to AI technology in multiple aspects, including mathematical problem-solving ability, self-learning and self-improvement.
Nvidia scientist Jim Fan, Turing Award winner Yann LeCun, and others discuss OpenAI's Q* implementation
Meta scientist Tian Yuandong believes that Q* is a combination of Q-learning and A*, and is naturally suitable for reasoning tasks, especially in mathematical reasoning
However, OpenAI has not disclosed specific details about the Q* algorithm so far, and we don't know exactly how effective it will be.
Kunlun Wanwei has been paying close attention to the trends of Q* since the Q* project was exposed, and set up a research team to try to develop its own Q* algorithm at the first time, hoping to break the blockade of OpenAI and improve the inference ability of existing open-source models.
After months of experimentation, Kunlun and Nanyang Technological University in Singapore have successfully developed an algorithm called Q*, which can significantly improve the inference ability of existing large models. On the GSM8K dataset, Q* helped Llama-2-7b achieve an accuracy rate of 80.8%, surpassing ChatGPT. On the MATH dataset, Q* helped DeepSeek-Math-7b achieve an accuracy rate of 55.4%, surpassing Gemini Ultra. On the MBPP dataset, Q* helped CodeQwen1.5-7b-Chat achieve an accuracy rate of 77.0%, narrowing the programming gap with GPT-4.
- 论文：Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
- Link to paper: https://arxiv.org/abs/2406.14283
Q* can help small models achieve the inference ability of models with tens or even hundreds of times the number of parameters larger than them, which not only greatly improves the performance of small models, but also significantly reduces the demand for computing resources, bringing new possibilities for the wide application of artificial intelligence and creating a new era of efficient intelligence.
Research has proved that Q* can help a small model with only 7b parameters to achieve the inference ability of a model with a parameter number that is tens or even hundreds of times larger than it, greatly improving the performance of the model and significantly reducing the demand for computing resources. At present, the research of Q* is still in its infancy, and there is still room for further improvement in all aspects of the algorithm. In the future, Kunlun Wanwei will continue to deepen this research, continuously improve the inference ability of domestic open-source models, break the closed-source blockade of OpenAI, and bring new possibilities for the development of cutting-edge artificial intelligence technologies.