(How) Can Machine Learning Support Teaching Staff in a Virtual Collaborative Learning Environment?

Purpose: Supervising roles within virtual collaborative learning (VCL) environments face many different challenges and are often having difficulties keeping an eye on everything and comprehending every participant’s or group’s workflow. To help supervisors with problems like that we developed a software prototype to support their daily workflow and overcome the mentioned challenges. Study design/methodology/approach: This study used a design science research approach to investigate how learning analytics might be able to support teaching staff in VCL settings. Previous studies demonstrated that this approach is very well suited to derive design guidelines for such software artifacts. Initially, a qualitative interview series with four experienced tutors was carried out to get an in-depth understanding of the challenges and tasks teaching staff typically faces in VCL environments. The interview material was systematically analysed to acquire the underlying software requirements. These were then combined with existing knowledge about support software of similar use cases to create a first prototype, primarily based on a supervised machine learning model that classifies online messages sent within the teams. Findings: The software tool we have developed has shown that machine learning processes can indeed be used to support teaching staff in VCL environments. The tool achieves satisfactory classification performance in categorizing chat messages. It could therefore be demonstrated that it is possible to classify chat messages using a software tool. The interviews conducted revealed a particular interest in the statistical evaluations of group activities in Microsoft Teams. This was said to save a lot of time in the subsequent evaluation of the groups. The interviewed eTutors also expressed an interest in receiving private information on the development of possible conflicts within groups. The interest of the interviewed supervisors was very high and additional interviews could lead to further possible feature ideas. In general, the evaluative interviews resulted in positive feedback regarding how our prototype supported eTutors during their work. Originality/value: In current times, spatially distributed and asynchronous education plays a bigger role than ever before. VCL environments are useful tools to overcome these challenges of time and space. However, unlike research on conversational agents or automated feedback systems, the focus of this work is not on the communication interface between software and users or the exclusively quantitative analysis of messages. The goal is to evaluate the content of sent messages within VCL environments using the six defined categories to gain a better qualitative understanding of the teams’ collaboration and its workflow.