E-assessment: an update on research , policy and practice

Abstract

Assessment is central to educational practice. High-stakes assessments exemplify curriculum ambitions, define what is worth knowing, and drive classroom practices. It is essential to develop systems for assessment which reflect our core educational goals, and which reward students for developing skills and attributes which will be of long-term benefit to them and to society. There is good research evidence to show that well designed assessment systems lead to improved student performance. In contrast, the USA provides some spectacular examples of systems where narrowly focused high-stakes assessment systems produce illusory student gains; this friendly fire results at best in lost opportunities, and at worst in damaged students, teachers and communities. ICT provides a link between learning, teaching and assessment. In school, ICT is used to support learning. Currently, we have bizarre assessment practices where students use ICT tools such as word processors and graphics calculators as an integral part of learning, and are then restricted to paper and pencil when their knowledge is assessed. Assessment systems drive education, but are themselves driven by a number of factors, which sometimes are in conflict. To understand likely developments in assessment, we need to examine some of these drivers of change. Implications of technology, globalisation, the EU, multinational companies, and the need to defend democracy are discussed. All of these influences are drivers for increased uses of ICT in assessment. Many of the developments require the assessment of higher-order thinking. However, there is a constant danger that assessment systems are driven in undesirable ways, where things that are easy to measure are valued more highly than things that are more important to learn (but harder to assess). In order to satisfy educational goals, we need to develop ways to make important things easier to measure - and ICT can help. All is not well with education. The Tomlinson Report (2004) identifies major problems with current educational provision at ages 14-19 years: there is a plethora of qualifications; too few students engage with education; the drop-out rate is scandalously high; and the most able students are not stretched by their studies. Young people are not being equipped with the generic skills, knowledge and personal attributes they will need in the future. A radical approach to qualifications is suggested which (in our view) can only be introduced if there is a widespread adoption of e-assessment. The UK government is committed to a bold e-assessment strategy. Components include: ICT support for current paperbased assessment systems; some online, on-demand testing; and the development of radical, ICT-set and assessed tests of ICT capability. Some good progress has been made with these developments. E-assessment can be justified in a number of ways. It can help avoid the meltdown of current paper-based systems; it can assess valuable life skills; it can be better for users for example by providing on-demand tests with immediate feedback, and perhaps diagnostic feedback, and more accurate results via adaptive testing; it can help improve the technical quality of tests by improving the reliability of scoring. E-assessment can support current educational goals. Paper and pencil tests can be made more authentic by allowing students to word process essays, or to use spreadsheets, calculators or computer algebra systems in paper-based examinations. It can support current UK examination processes by using Electronic Data Exchange to smooth communications between schools and examinations authorities; current processes of training markers and recording scores can be improved. Systems where student work is scanned then distributed have advantages over conventional systems in terms of logistics (posting and tracking large volumes of paper, for example), and continuous monitoring can ensure high marker reliability. Current work is pushing boundaries in areas such as text comprehension, and automated analysis of student processes and strategies. E-assessment can be used to assess new educational goals. Interactive displays which show changes in variables over time, microworlds and simulations, interfaces that present complex data in ways that are easy to control, all facilitate the assessment of problem-solving and process skills such as understanding and representing problems, controlling variables, generating and testing hypotheses, and finding rules and relationships. ICT facilitates new representations, which can be powerful aids to learning. Little is known about the cognitive implications of these representations; however, it seems likely that complex ideas (notably in reasoning from evidence of various sorts) will be acquired better and earlier than they are at present, and that the standards of performance demanded of students will rise dramatically. Here, we also explore ways to assess important but ill-defined goals such as the development of metacognitive skills, creativity, communication skills, and the ability to work productively in groups. A major problem with education policy and practice in England is the separation of academic and practical subjects. In the worst case, to be able to invent and create something of value is taken to be a sure sign of feeble-mindedness; where as to opine on the work of others shows towering intellectual power. A diet of academic subjects with no opportunities to act upon the world fails to equip students with ways to deal with their environments; a diet of practical subjects which do not engage higher-order thinking throughout the creative process equip students only to become workers for others. Both streams produce one-handed people, and polarised societies. E-portfolios can provide working environments and assessment frameworks which support project-based work across the curriculum, and can offer an escape from one of the most pernicious historical legacies in education. E-portfolios solve problems of storing student work, and make the activity of documenting the process of creation and reflection relatively easy. Reliable teacher assessment is enabled. There is likely to be extensive use of teacher assessment of those aspects of performance best judged by humans (including extended pieces of work assembled into portfolios), and more extensive use made of on-demand tests of those aspects of performance which can be done easily by computer, or which are done best by computer. The issue for e-assessment is not if it will happen, but rather, what, when and how it will happen. E-assessment is a stimulus for rethinking the whole curriculum, as well as all current assessment systems. New educational goals continue to emerge, and the process of critical reflection on what is important to learn, and how this might be assessed authentically, needs to be institutionalised into curriculum planning. E-assessment is certain to play a major role in defining and implementing curriculum change in the UK. There is a strong government commitment to high quality e-assessment, and good initial progress has been made; nevertheless, there is a need to be vigilant that the design of assessment systems is not driven by considerations of cost. Major challenges of going to scale have yet to be faced. A good deal of innovative work is needed, coupled with a grounded approach to system-wide implementation