HyEWCos: A Comparative Study of Hybrid Embedding and Weighting Techniques for Text Similarity in Short Subjective Educational Text

Abstract

This study is intended to evaluate and contrast the performance of varying combinations of embedding algorithms and weighting systems in measuring perception-based text similarity using the Cosine Similarity approach. Within a structured experiment design, a hybrid model referred to as HyEWCos (Hybrid Embedding and Weighting for Cosine Similarity) was built incorporating conventional embedding models (Word2Vec, FastText), transformer-based models (BERT, GPT), and statistical and linguistic word weighting schemes (TFIDF, BM25, POS-weighting, and N-weighting). The test results indicate that Word2Vec merged with the CBOW architecture and TFIDF weighting always returned the most reliable performance, with lowest error values (RMSE and MAE of 0.9868) and the highest rating correlation with expert judgment (Pearson’s, 0.524; Spearman’s, 0.543). These results show that contextually conditioned distributional representation approaches perform better in maintaining the semantic subtlety of short and subjective texts than transformer models that are not fine-tuned. This work is unique in terms of its evaluation framework because it integrates embedding and weighting approaches that have hitherto been examined mostly in separation. The main contribution of the study is the development of an experimental framework that serves as a foundation for building more stable and accurate text-based assessment systems. The research also proves the need for making decisions on representation methods based on the data type and domain and opens a door for continuing research in adaptive hybrid models and how their potential can be achieved through combining the best of various approaches.