Development and Evaluation of a Framework for the Performance-Based Testing of ICT Skills

3Citations
Citations of this article
27Readers
Mendeley users who have this article in their library.

Abstract

This paper addresses the development of performance-based assessment items for ICT skills, skills in dealing with information and communication technologies, a construct which is rather broadly and only operationally defined. Item development followed a construct-driven approach to ensure that test scores could be interpreted as intended. Specifically, ICT-specific knowledge as well as problem-solving and the comprehension of text and graphics were defined as components of ICT skills and cognitive ICT tasks (i.e., accessing, managing, integrating, evaluating, creating). In order to capture the construct in a valid way, design principles for constructing the simulation environment and response format were formulated. To empirically evaluate the very heterogeneous items and detect malfunctioning items, item difficulties were analyzed and behavior-related indicators with item-specific thresholds were developed and applied. The 69 item’s difficulty scores from the Rasch model fell within a comparable range for each cognitive task. Process indicators addressing time use and test-taker interactions were used to analyze whether most test-takers executed the intended processes, exhibited disengagement, or got lost among the items. Most items were capable of eliciting the intended behavior; for the few exceptions, conclusions for item revisions were drawn. The results affirm the utility of the proposed framework for developing and implementing performance-based items to assess ICT skills.

References Powered by Scopus

Validating the Interpretations and Uses of Test Scores

1308Citations
N/AReaders
Get full text

The Interplay of Evidence and Consequences in the Validation of Performance Assessments

824Citations
N/AReaders
Get full text

Making sense of credibility on the web: Models for evaluating online information and recommendations for future research

811Citations
N/AReaders
Get full text

Cited by Powered by Scopus

Developing Performance Tests to Measure Digital Skills: Lessons Learned From a Cross-National Perspective

1Citations
N/AReaders
Get full text

When work bothers your mind: a diary study on the relationship of German teachers’ work-related rumination with fatigue

0Citations
N/AReaders
Get full text

The Development and Validation of the Planning to Make Meals Performance Measure

0Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Engelhardt, L., Naumann, J., Goldhammer, F., Frey, A., Horz, H., Hartig, K., & Wenzel, S. F. C. (2021). Development and Evaluation of a Framework for the Performance-Based Testing of ICT Skills. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.668860

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 6

60%

Lecturer / Post doc 2

20%

Researcher 2

20%

Readers' Discipline

Tooltip

Social Sciences 5

63%

Physics and Astronomy 1

13%

Neuroscience 1

13%

Psychology 1

13%

Save time finding and organizing research with Mendeley

Sign up for free