【Abstract】The field of AI is undergoing a new paradigm shift, which is achieved by integrating multiple large language models (LLMs) and other complex component systems. Therefore, the development of efficient automated optimization methods for composite AI systems has become one of the primary challenges. SIMILAR TO THE BACKPROPAGATION AND AUTOMATIC DIFFERENTIATION TECHNIQUES THAT NEURAL NETWORKS RELIED ON IN THE EARLY DAYS, THIS PAPER INTRODUCES A NEW FRAMEWORK CALLED TEXTGRAD TO ACHIEVE AUTOMATIC "DIFFERENTIATION" THROUGH TEXT.
TEXTGRAD leverages text feedback generated by LLMs for backpropagation, optimizing the various components of the composite AI system. It operates in a similar way to PyTorch and is highly flexible and easy to use. TEXTGRAD IS AUTOMATICALLY ADAPTED TO A WIDE RANGE OF TASKS WITHOUT THE NEED FOR ADJUSTMENTS TO FRAME COMPONENTS AND PROMPTS. Our study demonstrates the effectiveness of TEXTGRAD in solving problems, molecular optimization, and radiotherapy planning. After experimental verification, TEXTGRAD has improved the zero-shot accuracy of GPT-4o in Google-Proof Q&A from 51% to 55%, achieved a 20% relative performance improvement in the optimization of LeetCode-Hard code questions, and successfully designed targeted treatment regimens. TEXTGRAD for развитию следующего поколения ИИ систем laid the foundation for the next generation of AI system development.
原文：Automatic "Differentiation" via Text
Address: https://arxiv.org/abs/2406.07496
Code: https://github.com/zou-group/textgrad
Publisher: arXiv
Institution: Stanford University
1. Research the problem
The core problem of this paper is how to design a common and automated framework to optimize composite AI systems composed of multiple large language models and complex components.
Consider a two-tier question answering system: the first part is a large language model that generates answers based on a given prompt; The second part is another large language model that evaluates the answers and scores them. The challenge now is how to automatically adjust the prompts so that the answers output in the first part get the highest possible rating.
The study faced the following challenges:
- The components of composite AI systems are often non-differentiable black boxes, and traditional gradient-based optimization methods are difficult to achieve.
- Intermediate variables and outputs of these components often exist in an unstructured form, such as natural language text, which makes optimization difficult.
- The optimized objective function can be complex and indifferentiable, sometimes requiring another AI system to compute.
- Existing methodologies often require extensive custom development and lack a common, automated optimization framework for universal use.
Based on the above challenges, this paper proposes a powerful automatic differentiation framework based on text feedback, TEXTGRAD. TEXTGRAD REPRESENTS A COMPLEX AI SYSTEM AS A COMPUTATIONAL GRAPH, WHERE A VARIABLE CAN BE ARBITRARY UNSTRUCTURED DATA AND A FUNCTION ON AN EDGE CAN BE A COMPLEX OPERATOR THAT IS NOT DIFFERENTIABLE. Backpropagation is achieved through text feedback generated by LLMs, and each variable is optimized. TEXTGRAD works in a similar way to PyTorch, allowing you to use tasks in different fields right out of the box.
2. Research Methodology
WHEN IT COMES TO THE TEXTGRAD FRAMEWORK, IT PRESENTS THE AI SYSTEM AS A COMPUTATIONAL GRAPH, AND THE VARIABLES CAN BE ANY UNSTRUCTURED DATA. In the case of a system consisting of two LLM calls, optimization is achieved in two steps: first, the variable generates a prediction through prompts, and second, the prediction is evaluated by the instructions of another model. TEXTGRAD GENERATES GRADIENT FEEDBACK THROUGH NATURAL LANGUAGE AND USES TEXT GRADIENT DESCENT (TGD) TO GUIDE VARIABLE OPTIMIZATION.
The framework supports multiple optimization iterations until the optimal solution is obtained. IN TERMS OF OPTIMIZATION METHODS, TEXTGRAD FLEXIBLY USES MANY TECHNOLOGIES, COMBINING BATCH OPTIMIZATION AND CONSTRAINT OPTIMIZATION, TO ACHIEVE EFFICIENT AUTOMATIC OPTIMIZATION.
3. Experiment
In this paper, the effectiveness of TEXTGRAD is verified in multiple fields and experimental scenarios, such as code optimization, scientific question answering, molecular design, and radiotherapy plan optimization. THE EXPERIMENTAL RESULTS ARE REMARKABLE, SHOWING THAT TEXTGRAD CAN ACHIEVE EXCELLENT PERFORMANCE IN COMPLEX SYSTEMS WITHOUT FRAME ADJUSTMENT.
- Instance optimization: Optimizing the code on the LeetCode Hard dataset, TEXTGRAD increased the pass rate of GPT-4o from 23% to 36%.
- Science Question Answering: In the science question dataset, TEXTGRAD improved GPT-4o's zero-shot accuracy from 51% to 55%.
- Inference task prompt optimization: GPT-3.5's performance on multiple inference tasks has been improved by optimizing the prompt.
- Drug molecule optimization: Optimize the binding energy and drug-like properties of molecules at the same time, and continuously improve the binding energy and drug-like properties on 58 targets.
- Radiotherapy Program Optimization: Optimize dose distribution in radiotherapy for prostate cancer, taking into account tumor dose and organ protection.
4. Summary
THE TEXTGRAD FRAMEWORK PROPOSED IN THIS PAPER DEMONSTRATES ITS POTENTIAL IN COMPLEX AI SYSTEMS BY GUIDING OPTIMIZATION THROUGH NATURAL LANGUAGE FEEDBACK GENERATED BY LLM. TEXTGRAD HAS LAID THE FOUNDATION FOR THE DEVELOPMENT AND POPULARIZATION OF AI SYSTEMS, AND HAS DEMONSTRATED EXCELLENT PERFORMANCE IN A VARIETY OF AREAS SUCH AS CODE OPTIMIZATION, SCIENTIFIC QUESTION SOLVING, MOLECULAR DESIGN, AND RADIATION THERAPY PLANNING. Possible future research directions include combining TEXTGRAD with other optimization techniques to expand its range of applications.