Granular Linguistic Model Based Multimodal Data Integration for Automated Evaluation of Core Soft Skills

Abstract

This research introduces an innovative hybrid intelligence framework leveraging multimodal data to automate the evaluation of core soft skills, including decision-making, conflict resolution, and creativity. The model applies the principles of the Granular Linguistic Model of Phenomena (GLMP), a sophisticated method that delineates phenomena at varying granularity levels, ensuring a detailed analysis of the exhibited skills. The process involves mining significant behavioural features from diverse data sources, specifically video, audio, and text, employing deep learning algorithms. The extracted features are then subjected to the GLMP, representing the students’ behaviour in a structured, interpretable format across multiple granularities. The GLMP application yields an exhaustive set of granular linguistic prompts that encapsulate the complexity of the identified soft skills. This multimodal information feeds into a fuzzy logic-based detector that evaluates the defined soft skills. This integrative approach merges granular linguistic modelling with multimodal data, enabling a comprehensive and accessible understanding of the students’ soft skills. The implications of this approach extend beyond the academic sphere, finding utility in broader contexts such as college admissions and job recruitment, where objective skill evaluation is crucial. This research underscores the value of multimodal integration within the GLMP framework, highlighting its critical role in translating raw data into actionable insights. It further illuminates the potential of such methods in enhancing real-world decision-making processes and outcomes.