PhD Project – Zhaoxing Li

Generative Adversarial Imitation Learning For Student Simulation

In this project, we are exploring the application of deep learning (DL) and reinforcement learning (RL) algorithms for student simulation technology in intelligent algorithm teaching systems (ITS). The aim is to improve the efficiency, robustness, and interpretability of the ITS system.

Research Outline and Goals With the rapid development of Artificial Intelligence (AI) technology, more and more Machine Learning (ML) technologies are used in every aspect of life. In the field of education, Intelligence Tutorials Systems (ITS) have also made significant developments. Developing different teaching strategies according to students’ personalities and learning styles could significantly improve students’ learning efficiency and performances. This requires the ITS system to generate different learning strategies and learning trajectories for different students. However, there are not enough interactive datasets between students and ITS that could be used to train the systems. The lack of interactive data sets severely limits the development of ITS systems. Therefore, building a student model or using an algorithm to simulate student behaviour to train the ITS is extremely important for improving the system. This project aims to develop a deep reinforcement learning-based student simulation algorithm to generate interactive data between students and ITS. These data could improve the training efficiency of ITS. Moreover, it could give teachers a different perspective and a deeper understanding of the learning process of students.

Pilot Study and Feasibility Testing We have developed a student simulation method based on generative adversarial imitation learning (GAIL) to generate interactive data between students and ITS. The interactive data generated based on GAIL performs better than other generation algorithms (such as FEM and GTAL), and its authenticity is almost always achieved at least 85%. We have tried to apply these data into the training of the ITS system and found that these data can effectively improve the generation of ITS for different student learning trajectories. It demonstrates the feasibility and effectiveness of the GAIL-based student simulation method.

Methodology and Future Work Currently, we have developed a student simulation method based on generative adversarial imitation learning (GAIL) to generate interactive data between students and ITS. The initial training data comes from the EdNet dataset. In future research, we plan to improve the efficiency of our algorithms, experiment with the generated data in ITS projects, and then use these data to improve the efficiency of ITS. In addition, Because GAIL is based on GAN, it also inherits the shortcomings of GAN-mode collapse. During training, the model tends to cover only specific modes in the distribution, not to generate sufficiently diverse samples. The advantage of VAE is that it could generate diverse behaviours, so in the next step, we will develop a generation method based on VAE to obtain robust strategies that cover the diverse behaviour of students. Also, we will develop other interactive data generation models, such as algorithms based on MGAIL and other Deep Reinforcement Learning methods.