summary
The interdisciplinary evaluation framework is mainly based on the transferability of interdisciplinary knowledge and the changes of students' subject knowledge literacy and ability literacy in the learning process. The framework of interdisciplinary learning evaluation in foreign countries mainly includes the contextualization of learning tasks, the visualization of learning processes, the technicalization of evaluation methods, and the evidence of evaluation results, which have a certain enlightening effect on the further construction of interdisciplinary learning evaluation framework in mainland China, which is helpful to further improve the scientificity of interdisciplinary learning evaluation and better play the value of interdisciplinary practice.
Keywords: interdisciplinary learning assessment; innovation ability; thinking skills; Discipline literacy
The concept of interdisciplinary learning was first proposed by the American psychologist Woodworth in the 20s of the 20th century, and is used to refer to "a learning model that takes a discipline as the center, uses the knowledge of different disciplines, and develops the processing and design of the common problems it points to". [1] Subsequently, other countries such as the United Kingdom, Germany, and Japan also carried out interdisciplinary construction. After 100 years of development, countries already have some experience in interdisciplinary education and teaching. The first interdisciplinary symposium was held in Beijing in 1985, marking the official beginning of the continent's exploration of interdisciplinary learning.
There is a lot of controversy about the implementation of interdisciplinary education, and the interdisciplinary teaching activities in most schools are still in the exploratory stage, so the evaluation of the implementation of interdisciplinary learning programs is very important. Combined with the current trend of international interdisciplinary evaluation education reform, this paper analyzes the representative interdisciplinary learning evaluation frameworks abroad, and deepens the understanding of the main interdisciplinary evaluation frameworks used abroad through two parts: the interdisciplinary learning comprehensive ability evaluation framework and the interdisciplinary learning discipline literacy evaluation framework, in order to bring some useful suggestions for improving the interdisciplinary evaluation system in mainland China.
1. The content of the comprehensive ability evaluation framework for interdisciplinary learning
In order to transform students' interdisciplinary ability into evaluable indicators and achieve the goal of evaluating students' interdisciplinary ability, after combing the evaluation framework of interdisciplinary learning ability at home and abroad, according to the perspective of different types of interdisciplinary ability training, the content of the evaluation framework is roughly divided into evaluation of interdisciplinary innovation ability, evaluation of interdisciplinary thinking ability and evaluation of interdisciplinary problem-solving ability from the perspective of different types of interdisciplinary ability training, from the perspective of highlighting the transferability of interdisciplinary knowledge and the change of students' ability and literacy in the learning process.
1. Evaluate interdisciplinary innovation capabilities
Interdisciplinary innovation refers to the ability of students to think independently, propose (or improve or innovate on the original basis) new, unique and valuable ideas or solutions in the process of interdisciplinary learning. The commonly used interdisciplinary innovation ability evaluation models include the TTCT innovation model and the TRIZ innovation model.
(1) TTCT innovation model
The Torrance Test of Creative Thinking (TTCT) was officially issued by Torrance at the University of Georgia in 1966 to assess students' creative behavior, creative thinking, and creative potential. The TTCT is divided into two parts, a verbal test and a graphic test, and consists of 6 items, each of which sets different tasks to test students' creative thinking skills. During the test, the TTCT score is mainly judged based on originality, originality, flexibility and feasibility, and through these sub-tests, students' creative thinking and creative expression ability can be understood. The TTCT innovation model is an effective way to assess students' interdisciplinary creativity. Through this test, teachers can better understand students' creative abilities and provide them with appropriate multidisciplinary development and development support.
(2) TRIZ Innovation Model
TRIZ stands for TheoryofInventiveProblemSolving, which can be translated as Innovative Problem Solving. Based on the analysis of patents in various countries, TRIZ proposes a theoretical framework composed of various methods and tools for solving problems and realizing innovation and development, and believes that the core of a large number of innovation problems is the same, but the fields are different, and if the core content of these problems is summarized, a systematic theoretical knowledge can be formed for later innovation practice activities.
As a system for solving problems in real situations, interdisciplinarity is one of the most important characteristics of TRIZ. Zhong Baichang's team used the literature analysis method to sort out the core evaluation elements of innovation ability, and when formulating the evaluation scale, with TRIZ as the core, the interdisciplinary characteristics of the evaluation elements were emphatically added, and finally the interdisciplinary innovation ability evaluation index system composed of 5 first-level indicators and 12 second-level indicators of innovative personality, innovative thinking, innovative learning, innovative skills and innovation achievements was determined. That is, the ultimate goal is to find the best solution to achieve the final solution. When designing the questions, the teacher first gives a demonstration case, and then encourages students to design a variety of solutions to achieve the goal based on their personal life and learning experiences, and find the best solution from them, so as to cultivate students' interdisciplinary innovation ability. The use of this learning mode can not only assess students' self-directed learning ability, but also evaluate students' ability to transfer multidisciplinary knowledge and innovation ability.
Both TTCT and TRIZ give the concept of product model before learning, encourage students to design or transform new models in the process, and carry out exploratory and innovative teaching activities in the learning process, so that students can experience and explore problems first-hand, so that students can actively think about multiple ways to solve problems, so as to cultivate their interdisciplinary innovation ability. The whole learning process promotes the emergence of new ideas, new technologies, and new products.
2. Evaluate interdisciplinary thinking skills
As an interdisciplinary and comprehensive education model, STEAM education emphasizes the integration of knowledge and skills in the five fields of science, technology, engineering, art and mathematics, which not only focuses on the knowledge and skills of a single discipline, but can focus on the intersection and integration between different disciplines, and guides students to cultivate their multidisciplinary thinking skills through inquiry, creativity, cooperation and reflection.
In view of the extensive research on interdisciplinary, STEAM, and thinking education, Wilson attempts to increase the understanding of the impact of an interdisciplinary approach on student thinking by analyzing the perspectives and thoughts of student participants in STEAM projects, recording their vocabulary, information synthesis, and changes in thinking. Wilson divides the STEAM thinking process into creative thinking and critical thinking: creative thinking refers to the ability of students to think fluently and uniquely, connecting ideas in their own way; Critical thinking refers to a student's ability to apply what they have learned to a new environment (Table 1). ［3］
Table 1 Definition of open themes for STEAM interdisciplinary courses
The results of the experiment showed that after participating in the STEAM interdisciplinary course, the students' thinking ability improved: they paid more attention to their imagination and new ideas in the process of participation, and also generated new ideas that combined their personal experience and learning interests, and the results showed that the STEAM interdisciplinary course had a positive impact on students' thinking ability.
Shouxin et al. divided interdisciplinary thinking into creative thinking and critical thinking: creative thinking is mainly embodied in making suggestions and solutions to problems; Critical thinking is mainly reflected in analyzing problems and verifying problems. [4] Based on this, the STEAM interdisciplinary content questions are designed to reflect the advanced process of creative thinking and critical thinking. Ronghui Que uses STEAM interdisciplinary thinking level to evaluate the course, and divides different interdisciplinary thinking levels according to students' degree of integration of interdisciplinary knowledge based on SOLO classification theory. [5] Different levels of interdisciplinary thinking have different degrees of knowledge integration, and the evaluation of the degree of interdisciplinary knowledge integration can understand the current situation of students' interdisciplinary thinking development, as shown in Table 2. The higher the level of interdisciplinary thinking, the higher the ability of knowledge integration, and the higher the ability of comprehensive thinking, until students can creatively build a new subject knowledge network by themselves.
Table 2 Evaluation framework of interdisciplinary thinking in STEAM education
The core of STEAM education is to cultivate students' interdisciplinary thinking, cultivate students' ability to focus on the topic of the essay when facing complex problems, apply logic ability and comprehensive analysis ability, and find ways to solve the problem. The evaluation of interdisciplinary thinking can effectively feedback students' interdisciplinary learning ability and improve teaching. In the future of STEAM education, teachers should find strategies to promote the high-level development of students' interdisciplinary thinking and enhance students' understanding of the nature of interdisciplinary learning.
3. Evaluate the ability to solve interdisciplinary problems
The ability to solve interdisciplinary learning problems mainly refers to the ability of students to combine the knowledge and skills of different disciplines to flexibly solve practical problems in life. Represented by ATC21S and project-based pedagogy, both models emphasize that students need to have practical skills and the ability to deal with complex situations, and be able to apply the knowledge and skills they have learned to solve practical problems in practice.
(1) ATC21S model
Assessment & Teaching of 21st Century Skills (ATC21S) combines critical thinking, problem-solving, decision-making, and cooperation skills into collaborative problem-solving. The ATC21S test uses a "person-to-person" approach, with two students completing a common task. During the test, the system captures and records students' key operational behaviors and conversations in real time, and then forms detailed process data. Subsequently, through an in-depth analysis of this data, key behaviors can be extracted as the basis for assessment, and then accurately inferred the actual ability of students in problem solving.
(2) Project-based pedagogy
The project-based pedagogy can be traced back to the "learning by doing" advocated by educator John Dewey, and later proposed by Keberw. The project teaching method focuses on combining students' multi-disciplinary content to solve practical problems, attaches importance to students' independent design, investigation and development of projects, and improves students' multidisciplinary knowledge application and comprehensive ability through the production of real project works. In the process of problem solving, because problem solving involves multi-faceted needs, which requires the integration of multidisciplinary and multi-domain content participation, the project teaching method plays a positive role in cultivating and promoting students' interdisciplinary ability to solve problems. The ATC21S and Project-based approach is student-centered, problem-based, problem-solving, and interdisciplinary knowledge as a clue to develop students' real problem-solving skills. In the problem-solving-oriented interdisciplinary teaching method, students need to reasonably select and link interdisciplinary knowledge to solve problems after analyzing the problems, which helps to improve students' proficiency in using interdisciplinary knowledge and problem-solving efficiency.
2. The content of the evaluation framework for interdisciplinary learning discipline literacy
From the perspective of highlighting the changes of students in the interdisciplinary learning process, interdisciplinary learning literacy focuses on the intersection and integration between different disciplines, emphasizing the integration and comprehensive application of students' knowledge, and is represented by the five most widely used interdisciplinary learning discipline literacy evaluation models, which are respectively described in detail.
1. Australia's National Literacy and Mathematics Assessment Program
The National Assessment Program Literacy and Numeracy (NAPLAN) is an evaluation activity of the Australian government at the national level in accordance with unified evaluation standards for the quality of basic education, which enables the Australian government to monitor the development of national basic education quality and determine the learning effectiveness of students. [6] NAPLAN is aimed at Australian students in Years 3, 5, 7 and 9 and covers the four areas of reading, writing, language (spelling, grammar and punctuation) and mathematics. In 2022, NAPLAN has fully shifted to digital testing, reforming its methods, tools, technology and organization.
First of all, in terms of evaluation mode, NAPLAN mainly adopts the internationally mature computer adaptive measurement technology. The adaptive test process of the reading and math tests is relatively simple, and the difficulty of the test questions at different stages is different, and students choose the test questions that are suitable for them according to their test level. The adaptive mechanism of the language test is a little more complex, and the adaptive mechanism of the language test includes two parts: spelling, grammar and punctuation, and the test process is the same as that of the reading and math tests.
Secondly, NAPLAN uses a vertically equivalent personalized assessment report. [7] The assessment results are divided into 1-10 levels, and each grade has 6 different grading levels, and the grades increase with the increase of students' grades, which can effectively assess students' interdisciplinary comprehensive ability.
2. Pan-Canadian Assessment Program
The Pan-Canadian Assessment Program (PCAP) is a national assessment of learning that covers three domains: reading, math, and science, with each assessment focusing on one of the areas and the remaining two as sub-areas. The core thinking of PCAP is mainly embodied in the connection between students' learning and life experience, emphasizing the mastery of core knowledge and skills of the subject. PCAP2013 emphasizes students' ability to use relevant scientific knowledge and skills to reason scientifically in terms of scientific literacy, so that students can understand scientific problems. In terms of mathematical literacy, it emphasizes that "mathematics can be studied and used in personal life and in the workplace, and at the same time, students need a new set of mathematical theories that enable them to be proficient in calculations and creative problem solving". [8] Based on this, PCAP2016 requires individuals to be ICT literate, as well as interdisciplinary competencies in mathematics, science, languages, social studies, geography and the arts. In terms of reading literacy, PCAP divides it into four elements: reader, text, purpose, and context. This means that students need to consider factors such as their own reading background, the purpose of reading, and the context in which they are reading. At the same time, PCAP also emphasizes students' knowledge construction ability in the reading process, that is, to improve knowledge comprehension ability through continuous reading and thinking. In addition, PCAP also focuses on students' ability to comprehend, interpret, and respond critically based on personal experience, so as to develop students' real reading skills.
PCAP focuses on the ability to make "evidence-based" inferences, focusing on allowing students to identify patterns, make connections, make interpretations, etc., from the context of the test questions. This ability will be shown when students are faced with real situations, and can be evaluated at the intersection of other abilities, which has a certain guiding role for the further reform of the evaluation system in the mainland.
3. Assessment of International Trends in Mathematics and Science Education
The International Assessment of Trends in Mathematics and Science Education (TIMSS) is an international assessment project initiated by the International Association for the Evaluation of Educational Achievement (IEA). Since 1995, the program has assessed the mathematics and science proficiency of fourth- and eighth-graders in the participating countries every four years.
TIMSS places great emphasis on evaluating interdisciplinary learning in life sciences and human biology. In the field of physical science, TIMSS focuses more on the combination of physics and chemistry, and effectively realizes the integration and application of physics and chemistry knowledge. [9] The key points of TIMSS assessment in various fields gradually tend to the direction of combining science and human life, gradually shifting from scientific inquiry to scientific practice, and more emphasis is placed on students' scientific literacy ability.
In terms of evaluation technology, two sets of adaptive digital assessment questions are set up in TIMSS2023 to make the assessment method more scientifically reflect students' abilities. At the same time, in order to better evaluate students' problem-solving and scientific inquiry skills, TIMSS will form an enhanced version of the international evaluation report, which will be included in the TIMSS2023 report, so as to have a deeper understanding of students' learning background and interdisciplinary learning ability, and make more scientific educational decisions.
4. U.S. National Assessment of Educational Progress
The National Assessment of Educational Progress (NAEP) is the only program in the U.S. that continuously assesses students' literacy in multidisciplinary domains on a national scale. It surveys a sample of public or private school students in grades 4, 8, and 12 in subject areas such as math, reading, science, writing, art, civics, economics, geography, history, technology, and engineering literacy.
Since 2009, NAEP has gradually begun to broaden the scope of assessment, transforming the original testing framework: transforming the field of science into two areas: scientific content and scientific practice, requiring students to master knowledge in three major areas: physical science, life science, and earth and space science. In addition, students are also expected to follow the rules of scientific activities, "using appropriate tools and techniques to conduct scientific investigations, conduct technical designs, etc." [10] This change examines scientific literacy ability, emphasizes the integration of subject knowledge, and also cultivates students' social science cognition. Although NAEP has different content requirements for students at different grade levels, the core goal is to develop students' scientific literacy skills.
In terms of assessment tools, since 2017, NAEP has fully adopted computer-based evaluation methods to improve the accuracy and flexibility of testing.
5. Programme for International Student Assessment
Initiated by the Organisation for Economic Co-operation and Development (OECD) in 1997, the Programme for International Student Assessment (PISA) assesses 15-year-olds in participating countries or regions every three years, with one of the three domains of science, reading and mathematics as the focus area for each assessment. [11] The purpose of PISA is to determine whether students are well-equipped to cope with the challenges posed by the modern social situation. PISA reflects the level of basic education through the comprehensive scientific literacy of students.
Table 3 Comparison of PISA2015 and PISA2025 science literacy assessment frameworks
Comparing PISA2025 with PISA2015, it can be found that the dimension of scientific knowledge is more prominent in the PISA2025 scientific literacy assessment framework, and the application process of scientific knowledge in the social environment is emphasized (Table 3). ［12］
The evaluation of interdisciplinary learning subject literacy pays more attention to the mastery of students' core knowledge and abilities in related disciplines, as well as students' literacy level and emotional attitude towards the course. PCAP pays more attention to students' interdisciplinary learning ability to combine reading and mathematics with life. TIMSS, NAEP and PISA pay more attention to the interdisciplinary content learning of life science and human biology, and through the cross-learning between physics, chemistry and biology, it not only strengthens the integration between disciplinary knowledge points, but also improves students' scientific cognition.
3. The promotion path of interdisciplinary learning evaluation
Through the further combing and analysis of the above-mentioned foreign interdisciplinary learning evaluation frameworks, it is found that different types of interdisciplinary learning evaluation frameworks have different characteristics and trends in terms of evaluation dimensions, and the mainland is actively exploring the interdisciplinary learning evaluation framework in line with the characteristics of the mainland, and the international interdisciplinary learning evaluation framework can bring the following reference opinions for the implementation of interdisciplinary learning evaluation in the mainland.
1. Contextualize learning tasks
In recent years, more and more attention has been paid to interdisciplinary education in all aspects, and various subjects have paid more attention to creating a good atmosphere and environment for students. Diverse scenarios can help students become aware of the connections and intersections between different disciplines, and promote the development of interdisciplinary awareness. For example, ATC21S adopts a real-life interactive scenario assessment model, which guides students to examine problems from multiple perspectives, discover the internal connections and rules between different disciplines, and propose novel, unique and valuable ideas or solutions, which helps to improve students' interdisciplinary thinking ability and innovation ability.
2. Visualize the learning process
In interdisciplinary learning activities, the learning process is a process of exercising thinking, improving innovation and problem solving. The visualization of Observable Learning Outcome Structure (SOLO) evaluation can visualize students' learning outcomes and ability levels, helping educators better understand student learning. Through the visualization of the learning process, educators can intuitively understand the learning level of students, which helps educators to make teaching decisions more accurately and quickly, and the SOLO classification evaluation focuses on the students' thinking level, which can visualize the students' thinking process and help students sort out their own learning ideas, which helps to improve the efficiency and learning experience of students' interdisciplinary learning, and enhance students' interest in learning interdisciplinary activities.
3. Technicalize evaluation methods
With the development of intelligent technology, computers, as an efficient evaluation tool, provide us with a more convenient evaluation method: the scientific and accurate scoring of artificial intelligence is far beyond the traditional manual scoring method. More importantly, the information evaluation based on big data can not only provide massive data support, but also achieve rapid data processing, and the larger the sample size, the more reliable the evaluation results. For example, many international evaluation frameworks have adopted adaptive assessment models based on human-computer interaction models, and have gradually become the trend of learning and teaching, computer-based online evaluation systems have been able to realize the transformation from outcome tests to personalized tests, and the mainland can combine the development of technology to improve the existing evaluation methods.
4. Evidence the results of the evaluation
The core idea of "evidence-centered evaluation" is that "evaluation is evidence-based reasoning", which is an evidence-centered evaluation and reasoning process, which completes learning evaluation by obtaining multi-channel data types. For example, ATC21S, PISA and other evaluation projects use an "evidence-centered" design model to record students' conversations and behaviors in the process of collaboration, and form process data for scoring based on evidence. PCAP focuses on getting students to identify patterns, make connections, make interpretations, and more from the context of the test questions. The application of "evidence-centered design" to learning evaluation breaks through the limitations of subjective perception in the past, promotes the evidence-driven teaching evaluation cycle, and promotes the cohesion and symbiosis of all links in the interdisciplinary learning evaluation process.
Based on the analysis of the main interdisciplinary learning evaluation framework models at home and abroad, this paper sorts out two types of interdisciplinary evaluation from the perspective of highlighting the transferability of interdisciplinary knowledge and the changes in students' subject knowledge literacy and ability literacy in the learning process, hoping to bring some reference value to the construction of interdisciplinary learning evaluation framework in mainland China, so as to further improve teachers' interdisciplinary learning evaluation ability and better play the value of interdisciplinary practice.
bibliography
[1] Du Huijie, Schultz. Global Education Outlook,2005,34(8):28-32.)
DU Huijie， Dieter Scholz. Analysis on Ideal and Design of Cross-Discipline Instruction in Germany［J］.Global Education，2005，34（8）:28-32.
[2] Zhong Baichang, Gong Jiaxin. Evaluation of interdisciplinary innovation ability based on TRIZ:Preparation and verification of test questions[J].Research on Modern Distance Education,2023,35(4):75-82.)
ZHONG Baichang， GONG Jiaxin. Evaluation of Interdisciplinary Innovation Ability Based on TRIZ: Test Compilation and Demonstration ［J］. Modern Distance Education Research，2023，35（4）:75-82.
［3］ WILSON H E， SONG H H，JOHNSON J， et al. Effects of Transdisciplinary STEAM Lessons on Student Critical and Creative Thinking ［J］. The Journal of Educational Research，2021，114（5）:445-457.
[4] Shou Xin, Huang Xiuli, Li Jian, et al. Evaluation of higher-order thinking based on STEM learning goals[J].Modern Educational Technology,2021,31(3):20-27.)
SHOU Xin， HUANG Xiuli， LI Jian， et al. Evaluation of Higher Order Thinking Based on STEM Learning Goals［J］. Modern Educational Technology，2021，31（3）:20-27.
[5] Que Ronghui, Zhou Lifang. Research on the evaluation of interdisciplinary thinking in STEAM education: An empirical analysis based on chemistry normal students[J].Chemical Education,2023,44(6):59-66.)
QUE Ronghui， ZHOU Lifang. Research on Evaluation of Interdisciplinary Thinking in STEAM Education: Empirical Analysis Based on Chemistry Normal Students［J］. Chinese Journal of Chemical Education，2023，44（6）:59-66.
［6］ National Assessment Program: Literacy and Numeracy.2016 Information for Parents and Carers［EB/OL］.（2022-3-16）［2023-11-27］. http://www.nap.edu.au/_resources/NAPLAN_2016_information_for_parents_web.
［7］ ACARA. Results and Reports［EB/OL］. ［2023-11-29］.
https://www.nap.edu.au/naplan/results-and-reports.
［8］ CMEC. PCAP-2016-Asesment-Framework ［EB/OL］.（2018-1-2）［2023-11-29］. https://www. cmec. ca/docs/pcap/pcap2016/PCAP-2016-Assessment-Framework.
[9] Gu Naijing. Overview and Implications of TIMSS2023 Scientific Assessment Framework: A Comparison with the TIMSS1995-2019 Scientific Assessment Framework[J]. Examination Research,2023(1):75-84.
GU Naijing. Overview and Enlightenment of TIMSS 2023 Scientific Assessment Framework-and Comparison With TIMSS 1995-2019 Scientific Assessment Framework［J］. Examinations Research，2023（1）:75-84.
［10］ National Assessment Governing Board， U. S. Department of Education. Science Framework for the 2015 National Assessment of Educational Progress［EB/OL］.（2008-09-30）［2023-11-29］.https://nces.ed.gov/nation-sreportcard/science/.
[11] Fataramanatsrutrakul(吴美灵).泰国Pisa测评研究[d].上海:华东师范大学,2019.
PHATTHARAMANATSRITRAKUL （WU Meiling）. Research on PISA Assessment in Thailand ［D］.Shanghai: East China Normal University，2019.
[12] Li Chuan. PISA2025 New trends and enlightenment of scientific literacy assessment framework[J].Popular Science Research,2022,17(1):52-58.)
LI Chuan. New Trends and Implications of PISA 2025 Science Literacy Assessment［J］. Studies on Science Popularization，2022，17（1）:52-58.
Shanghai Educational Evaluation Study
Founded in 2012, Shanghai Educational Evaluation Research is supervised by the Shanghai Municipal Education Commission and sponsored by the Shanghai Educational Evaluation Institute, and is publicly distributed at home and abroad with the postal code 4-886. It is an extended journal of Chinese humanities and social sciences journals AMI Comprehensive Evaluation (A journal), and has been rated as the most popular journal in the education discipline of the National Philosophy and Social Sciences Documentation Center for three consecutive years.
From Wei Zhou, Minjun Cai, Shanghai Educational Evaluation Research