DEVELOPMENT OF A HYBRID HUMAN-AI SYSTEM FOR EDUCATIONAL PROGRAM DESIGN

Abstract

The use of artificial intelligence on the educational program design allows you to automate the learning process and align with labor market demands. This will be an important step aimed at improving the quality of education and improving the learning experience. We introduce a novel hybrid human-AI system for educational program design. The traditional answers often can’t keep up with fast-changing job markets or meet diverse learning needs. Our method is aimed at improving the flexibility of the educational framework design using Llama 3.  In this work, we discuss all primary components, which includes 1) educational program objectives, 2) relevant job positions, 3) requisite skills, and 4) corresponding learning courses, in order refine suggestion and recommendation services. The large language model itself already gives fairly high-quality answers, although it lacks deep knowledge of university disciplines. We tried to use not only Llama 3, but also other language models, such as GPT-3.5 and GPT-4. Our model achieves a high performance (F1 score of 77.6%) in skill extraction and outperforms GPT-3.5 and GPT-4 in educational content generation, with F1 score of 0.35±0.10, precision of 0.37±0.12, and semantic similarity of 0.84±0.10.  Llama 3, due to a large number of parameters and a more extensive knowledge base, better took into account the context of the university database and generated a program structure with a large number of educational entities, which made work easier. Even with such promise, these three powerful limitations, technological infrastructure, biases in datasets and resistance of educational institutions to AI adoption, create challenges. Our hope is that this study will contribute further research to the human-AI collaboration educational applications.