Development of an assessment questionnaire for on-line teaching competency

Abstract

Evaluating teaching to improve the quality of education implies to recognize its complexity and to count with valid assessment instruments that lead to improvement. The objective of this article is to report the research process that allowed the development of a formative assessment questionnaire of on-line teaching based on the opinion of students. The participants were six experts on the design and implementation of on-line courses, and a total of 2821 university students. The method consisted of two stages: 1) Development of the instrument through theoretical background regarding the elements of the on-line teaching competencies that are susceptible of being evaluated by students; the making of the questionnaire specification matrix; matrix validation by experts judgment and pilot study; 2) Contribution of confidence evidence and validity included the application of the questionnaire; descriptive statistical calculations; unidimensional analysis through the Rasch model; establishment of the scores reliability; exploratory and confirmatory factorial analysis. It was demonstrated that 28 items evaluate the same construct and 2 factors were confirmed: forecast of the teaching-learning process and the conduction and evaluation of the teaching and learning experience; a total variance of 0.01% and alpha ordinal of 0.998; and adjustment index RMSEA=0.08; SRMR= 0.03, CF1=0.91. It was concluded that the results corroborated the instrument’s theoretical background with acceptable psychometric characteristics so that its application is recommended.