An Exploratory Sequential Mixed Methods Design to Identify Criteria for Continuous Performance Evaluation

Abstract

An alternative to the ubiquitous annual performance evaluation is continuous performance evaluation that is posed to give more objective, frequent and constructive feedbacks. Despite researchers recommending following a continuous performance evaluation for employees, there are hardly any progress in this regard. One of the hindrances is in identifying the evaluation criteria. Identifying the relevant criteria - both from employers and employee's perspective - is important for the success of a continuous performance evaluation process. Given its novelty, it calls for an exploration to understand a holistic set of criteria. For this purpose, an exploratory sequential mixed method was adopted in the Information Technology Industry in India. In the first phase, an unstructured interview was conducted among 11 software engineers (employees). Based on the responses, a semi-structured interview was conducted among 52 software engineers. A qualitative content analysis of the interview transcripts resulted in 33 criteria. Criteria that were similar, synonymous and complementary were combined. An acceptable interrater reliability was obtained in the content coding and categorization process (Krippendorff's a = 0.822). T his w as followed by a focus group discussion among employers (represented by 15 project managers) and 11 criteria were eliminated and seven were added. In the second phase, the importance of 28 criteria were sought using a questionnaire based survey. A simple random sampling was adopted and 498 responses were received. (Cronbach's a = 0.786). After elimination for incompleteness and erratic information, responses from 443 participants were considered for quantitative analysis. An exploratory factor analysis was conducted on the responses with 28 criteria subjected to exploratory factor analysis. Two criteria were not included for analysis due to a low mean sampling error value. The 26 most relevant criteria were categorized into six factors based on their factor loading. The main criteria factors were titled: Diagnostic, Dynamic, Proactive, Prompt, Resourceful and Responsible. These factors and the criteria within these factors can be used for continuously evaluating the performance of employees.