E-learning University Evaluation Through Sentiment Analysis Centered On User Experience Dimensions

The COVID-19 crisis increased the number of users of university online teaching, enhancing the importance of this learning format. Additionally, ISO 9241-210:2019 standard sets the international standards for the design of products, services and interaction systems from usability, accessibility, and user experience (User eXperience - UX) perspective. Then, in order to design interfaces and learning experiences that include motivations, feelings and needs of end users, it is necessary to previously evaluate the UX of these environments, with less general and/or laborious methods than those that currently exist. Therefore, this work aims to establish the basis of a method that allows to automatically evaluate the UX of online teaching platforms by analyzing the users' sentiment about specific aspects of their virtual learning experience. To do this, 2,035 users were surveyed about their online learning experience with a questionnaire and an open text field to give their opinion. The population surveyed were online postgraduate students of the Universitat de València and the Universidad Rey Juan Carlos, and university students of massive open online courses of the Universitat Politècnica de València. The opinions collected in Spanish from 476 students were processed with the commercial sentiment analysis and natural language processing tool MeaningCloud, to analyze the sentiment (positive, negative, or neutral) about aspects of their experience. The results present a new model that, on the one hand, ontologically classifies categories and aspects of online education with sentiment analysis techniques, and on the other hand, the model groups these categories according to UX criteria, presenting its own classification to facilitate the evaluation of online learning experiences in a concrete and automatic way.

Keywords: user experience, UX, e-learning, virtual learning, sentiment analysis, data mining, MeaningCloud, natural language processing, university online learning, user centered design, UCD, NLP, VLE