D O I N G quantitative research I N E D U C A T I O N D A N I E L M U I J S

Doing Quantitative Research in Education with SPSS 9079 Prelims (i-xii) 26/2/04 3:41 pm Page i

9079 Prelims (i-xii) 26/2/04 3:41 pm Page ii

Doing Quantitative Research in Education with SPSS Daniel Muijs Sage Publications London ��� Thousand Oaks ��� New Delhi 9079 Prelims (i-xii) 26/2/04 3:41 pm Page iii

�� Daniel Muijs 2004 First published 2004 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act, 1998, this publication may be reproduced, stored or transmitted in any form, or by any means, only with the prior permission in writing of the publishers, or in the case of reprographic reproduction, in accordance with the terms of licences issued by the Copyright Licensing Agency. Inquiries concerning reproduction outside those terms should be sent to the publishers. SAGE Publications Ltd 1 Oliver���s Yard 55 City Road London EC1Y 1SP SAGE Publications Inc. 2455 Teller Road Thousand Oaks, California 91320 SAGE Publications India Pvt Ltd B-42, Panchsheel Enclave Post Box 4109 New Delhi 100 017 British Library Cataloguing in Publication data A catalogue record for this book is available from the British Library ISBN 0-7619-4382-X ISBN 0-7619-4383-8 (pbk) Library of Congress Control Number: 2003115358 Typeset by Pantek Arts Ltd Printed in Great Britain by Athenaeum Press Ltd, Gateshead, Tyne & Wear 9079 Prelims (i-xii) 26/2/04 3:41 pm Page iv

List of figures viii List of tables x Preface xi 1 Introduction to quantitative research 1 What is quantitative research? 1 Foundations of quantitative research methods 3 When do we use quantitative methods? 7 Summary 11 Exercises 11 Further reading 12 2 Experimental and quasi-experimental research 13 Types of quantitative research 13 How to design an experimental study 15 Advantages and disadvantages of experimental research in education 22 Quasi-experimental designs 26 Summary 32 Exercises 33 Further reading 33 3 Designing non-experimental studies 34 Survey research 34 Observational research 51 Analysing existing datasets 57 Summary 60 Exercises 61 Appendix 3.1 Example of a descriptive form 62 Appendix 3.2 Rating the quality of interactions between teachers and pupils 63 Contents v 9079 Prelims (i-xii) 26/2/04 3:41 pm Page v

4 Validity, reliability and generalisability 64 Validity 65 Reliability 71 Generalisability 75 Summary 82 Exercises 83 Further reading 83 5 Introduction to SPSS and the dataset 85 Introduction to SPSS 85 Summary 90 Exercises 90 6 Univariate statistics 91 Introduction 91 Frequency distributions 91 Levels of measurement 97 Measures of central tendency 99 Measures of spread 105 Summary 110 Exercises 112 Further reading 112 7 Bivariate analysis: comparing two groups 113 Introduction 113 Cross tabulation ��� looking at the relationship between nominal and ordinal variables 114 The t-test: comparing the means of two groups 127 Summary 139 Exercises 140 Further reading 140 8 Bivariate analysis: looking at the relationship between two variables 142 The relationship between two continuous variables: Pearson���s r correlation coefficient 142 Spearman���s rho rank-order correlation coefficient: the relationship between two ordinal variables 151 vi ��� Contents 9079 Prelims (i-xii) 26/2/04 3:41 pm Page vi

Summary 156 Exercises 157 Further reading 158 9 Multivariate analysis: using multiple linear regression to look at the relationship between several predictors and one dependent variable 159 Introduction 159 What is multiple linear regression? 160 Doing regression analysis in SPSS 163 Using ordinal and nominal variables as predictors 169 Diagnostics in regression 176 Summary 183 Exercises 184 Further reading 184 10 Using analysis of variance to compare more than two groups 185 Want is ANOVA? 185 Doing ANOVA in SPSS 187 The effect size measure 194 Using more than one independent variable 196 Summary 200 Exercises 201 Further reading 201 11 One step beyond: introduction to multilevel modelling and structural equation modelling 202 Multilevel modelling 202 Structural equation modelling 209 Summary 217 Exercises 218 Further reading 219 References 220 Index 222 Contents ��� vii 9079 Prelims (i-xii) 26/2/04 3:41 pm Page vii

4.1 Shavelson���s multifaceted, hierarchical self-concept model 69 4.2 Type I and type II errors 76 5.1 SPSS opening screen 86 5.2 The ���Data View��� screen with our file opened 87 5.3 The ���Variable View��� 88 6.1 Producing a frequency table: steps 1���3 92 6.2 Producing a frequency table: steps 4���6 93 6.3 ���Frequencies��� output 94 6.4 Getting a chart in ���Frequencies��� 95 6.5 A bar chart 96 6.6 Measures of central tendency in SPSS 103 6.7 Measures of central tendency output 104 6.8 Measures of spread 108 6.9 Output measures of spread 109 6.10 Describing single variables 111 7.1 Producing a cross tabulation table: steps 1���3 116 7.2 Producing a cross tabulation table: steps 4 and 5 117 7.3 Producing a cross tabulation table: steps 6 and 7 118 7.4 ���Crosstabs��� output 119 7.5 Obtaining expected values in ���Crosstabs���: steps 8���10 120 7.6 ���Crosstabs��� output with expected values 121 7.7 Obtaining the chi square test in ���Crosstabs���: steps 11���13 123 7.8 Chi square text output 124 7.9 Selecting cases: steps 1 and 2 128 7.10 Selecting cases: steps 3 and 4 129 7.11 Selecting cases: steps 5���7 130 7.12 The t-test: steps 1���3 132 7.13 The t-test: steps 4���6 133 7.14 The t-test: step 7 134 7.15 T-test: output 135 7.16 Summary of bivariate data 140 viii List of figures 9079 Prelims (i-xii) 26/2/04 3:41 pm Page viii

8.1 Correlations: steps 1���3 148 8.2 Correlations: step 4 149 8.3 Pearson���s r output 150 8.4 Spearman���s correlation: step 5 154 8.5 Output of Spearman���s rho 155 8.6 Summary of bivariate relationships 156 9.1 Scatter plot of English and maths scores 161 9.2 Multiple linear regression: steps 1���3 164 9.3 Multiple regression: steps 4 and 5 165 9.4 Regression output: part 1 166 9.5 Regression output: part 2 167 9.6 Output including two ordinal variables 170 9.7 Transforming variables: steps 1���3 172 9.8 Recoding variables: steps 4���7 173 9.9 Recoding variables: steps 8���10 174 9.10 Output including dummy variables 175 9.11 Diagnostics: part 1 177 9.12 Diagnostics: part 2 178 9.13 Outliers casewise diagnostics output 179 9.14 Collinearity diagnostics 180 9.15 Collinearity diagnostics output 181 10.1 ANOVA: steps 1���3 188 10.2 ANOVA: steps 4���6 189 10.3 ANOVA: steps 7 and 8 190 10.4 ANOVA output: part 1 191 10.5 ANOVA output: part 2 ��� multiple comparisons 192 10.6 ANOVA output: part 3 ��� homogeneous subgroups 193 10.7 ANOVA: producing effect size measures ��� part 1 194 10.8 ANOVA: producing effect size measures ��� part 2 195 10.9 Effect size output 196 10.10 Multiple predictors and interaction effects: output 197 11.1 Predictors of achievement: a regression model 210 11.2 A more complex model 211 11.3 Four manifest variables determined by a latent maths self-concept 212 11.4 Year 5 structural equation model 215 List of figures ��� ix 9079 Prelims (i-xii) 26/2/04 3:41 pm Page ix

6.1 Height, gender and whether respondent likes their job 100 6.2 Height, gender and whether respondent likes their job ordered by height 101 6.3 Wages in an organisation 102 6.4 Test scores in two schools 105 6.5 Calculating the interquartile range 107 7.1 Cross tabulation of gender and ethnicity (1) 114 7.2 Cross tabulation of gender and ethnicity (2) 115 8.1 Actual responses 152 8.2 Ranking of actual responses 152 11.1 Multilevel model: end-of-year test scores predicted by beginning-of-year test scores 206 11.2 Multilevel model: end-of-year test scores predicted by beginning-of-year test scores and pupil variables 207 11.3 Multilevel model: end-of-year test scores predicted by beginning-of-year test scores, pupil, school and classroom variables 208 List of tables x 9079 Prelims (i-xii) 26/2/04 3:41 pm Page x

In this book we will be looking at quantitative research methods in edu- cation. The book is structured to start with chapters on conceptual issues and designing quantitative research studies before going on to data analysis. While each chapter can be studied separately, a better under- standing will be reached by reading the book sequentially. This book is intended as a non-mathematical introduction, and a soft- ware package will be used to analyse the data. This package is SPSS, the most commonly used statistical software package in the social sciences. A dataset from which all examples are taken can be downloaded from the accompanying website (www.sagepub.co.uk/resources/muijs.htm). The website also contains the answers to the exercises at the end of each chapter, additional teaching resources (to be added over time), and a facility to address questions and feedback to the author. I hope you find this book useful, and above all that it will give you the confidence to conduct and interpret the results of your own quantitative inquiries in education. Daniel Muijs xi Preface 9079 Prelims (i-xii) 26/2/04 3:41 pm Page xi

9079 Prelims (i-xii) 26/2/04 3:41 pm Page xii

��� ��� ��� What is quantitative research? Research methods in education (and the other social sciences) are often divided into two main types: quantitative and qualitative methods. This book will discuss one of these two main strands: quantitative methods. In this chapter we will have a look at what is meant by the term quantitative methods, and what distinguishes quantitative from qualitative methods. When you think of quantitative methods, you will probably have spe- cific things in mind. You will probably be thinking of statistics, numbers ��� many of you may be feeling somewhat apprehensive because you think quantitative methods are difficult. Apart from the last, all these thoughts capture some of the essence of quantitative methods. The following definition, taken from Aliaga and Gunderson (2002), describes what we mean by quantitative research methods very well: Quantitative research is ���Explaining phenomena by collecting numeri- cal data that are analysed using mathematically based methods (in particular statistics).��� Let���s go through this definition step by step. The first element is explain- ing phenomena. This is a key element of all research, be it quantitative or qualitative. When we set out do some research, we are always looking to explain something. In education this could be questions like ���why do teachers leave teaching?���, ���what factors influence pupil achievement?��� and so on. The specificity of quantitative research lies in the next part of the defini- tion. In quantitative research we collect numerical data. This is closely connected to the final part of the definition: analysis using mathematically ��� ��� ��� Chapter 1 Introduction to quantitative research 1 9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 1

based methods. In order to be able to use mathematically based methods our data have to be in numerical form. This is not the case for qualitative research. Qualitative data are not necessarily or usually numerical, and therefore cannot be analysed using statistics. Therefore, because quantitative research is essentially about collecting numerical data to explain a particular phenomenon, particular questions seem immediately suited to being answered using quantitative methods: ��� How many males get a first-class degree at university compared to females? ��� What percentage of teachers and school leaders belong to ethnic minority groups? ��� Has pupil achievement in English improved in our school district over time? These are all questions we can look at quantitatively, as the data we need to collect are already available to us in numerical form. However, does this not severely limit the usefulness of quantitative research? There are many phenomena we might want to look at, but which don���t seem to produce any quantitative data. In fact, relatively few phenomena in edu- cation actually occur in the form of ���naturally��� quantitative data. Luckily, we are far less limited than might appear from the above. Many data that do not naturally appear in quantitative form can be col- lected in a quantitative way. We do this by designing research instruments aimed specifically at converting phenomena that don���t nat- urally exist in quantitative form into quantitative data, which we can analyse statistically. Examples of this are attitudes and beliefs. We might want to collect data on pupils��� attitudes to their school and their teach- ers. These attitudes obviously do not naturally exist in quantitative form (we don���t form our attitudes in the shape of numerical scales!). Yet we can develop a questionnaire that asks pupils to rate a number of state- ments (for example, ���I think school is boring���) as either agree strongly, agree, disagree or disagree strongly, and give the answers a number (e.g. 1 for disagree strongly, 4 for agree strongly). Now we have quantitative data on pupil attitudes to school. In the same way, we can collect data on a wide number of phenomena, and make them quantitative through data collection instruments like questionnaires or tests. In the next three chapters we will look at how we can develop instruments to do just that. 2 ��� Doing Quantitative Research in Education 9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 2