Statistical notes for clinical researchers: simple linear regression 3 – residual analysis

Abstract

In the previous sections, simple linear regression (SLR) 1 and 2, we developed a SLR model and evaluated its predictability. To obtain the best fitted line the intercept and slope were calculated by using the least square method. Predictability of the model was assessed by the proportion of the explained variability among the total variation of the response variable. In this session, we will discuss four basic assumptions of regression models for justification of the estimated regression model and residual analysis to check them. FITTED REGRESSION LINE AND RANDOM ERROR Let's recall a bivariate relationship between 2 variables, X and Y, and its depiction of a SLR model in Figure 1A and 1B from the previous section [1]. The expression of the regression model is Y=β 0 +β 1 X+ε. The regression model is divided into two parts, the fitted regression line, 'β 0 +β 1 X' and random error, 'ε.' The need of error term is justified by the gap between the line and observed dots because regression line does not go through all the observed values as appeared in Figure 1A. The first part, fitted regression line, is made by connection of the expected means of Y values corresponding with X values such as χ 1 to χ 5 in Figure 1B. Please find the conceptual distribution of Y, which corresponds with subgroup of χ 1 , lying on upper vertical direction. The distribution is displayed as a bell-shaped normal distribution with a mean, μ y|χ1 , at the center. The symbol, μ y|χ1 means expected population mean of Y when X variable has the value χ 1. In accordance with the previous sections on regression, the expected mean of Y can be symbolized as Ŷ, which is equal to the fitted line, β 0 +β 1 X'. The expected population mean of Y changes from μ y|χ1 to μ y|χ5 as X changes from χ 1 to χ 5. The conceptual model suggests that the expected mean of Y can be depicted as the straight line 'β 0 +β 1 X' by connecting the expected mean of Y matched with subgroups of X. We call the straight line as 'mean function' because the expected mean of Y is expressed as the function of 'β 0 +β 1 X'. Please clearly understand that there are numerous means by numerous subgroups of continuous X, and they are linearly connected to make the linear mean function, 'β 0 +β 1 X'. Therefore, we should be able to reasonably assume that the mean function of Y has the form of fitted regression line when we apply the SLR model. Now let's discuss the second part, random error 'ε.' The conceptual form of the random error is depicted as bell-shaped distributions in Figure 1B. At the center of each distribution, Restor Dent Endod. 2019 Feb;44(1):e11 https://doi.