Introduction to Linear Regression Analysis - Douglas C. Montgomery

Скачать книгу

maximum-likelihood estimation requires more stringent statistical assumptions than the least-squares estimators. The least-squares estimators require only second-moment assumptions (assumptions about the expected value, the variances, and the covariances among the random errors). The maximum-likelihood estimators require a full distributional assumption, in this case that the random errors follow a normal distribution with the same second moments as required for the least- squares estimates. For more information on maximum-likelihood estimation in regression models, see Graybill [1961, 1976], Myers [1990], Searle [1971], and Seber [1977].

      The linear regression model that we have presented in this chapter assumes that the values of the regressor variable x are known constants. This assumption makes the confidence coefficients and type I (or type II) errors refer to repeated sampling on y at the same x levels. There are many situations in which assuming that the x’s are fixed constants is inappropriate. For example, consider the soft drink delivery time data from Chapter 1 (Figure 1.1). Since the outlets visited by the delivery person are selected at random, it is unrealistic to believe that we can control the delivery volume x. It is more reasonable to assume that both y and x are random variables.

      Fortunately, under certain circumstances, all of our earlier results on parameter estimation, testing, and prediction are valid. We now discuss these situations.

      Suppose that x and y are jointly distributed random variables but the form of this joint distribution is unknown. It can be shown that all of our previous regression results hold if the following conditions are satisfied:

      1 The conditional distribution of y given x is normal with conditional mean β0 + β1x and conditional variance σ2.

      2 The x’s are independent random variables whose probability distribution does not involve β0, β1, and σ2.

      While all of the regression procedures are unchanged when these conditions hold, the confidence coefficients and statistical errors have a different interpretation. When the regressor is a random variable, these quantities apply to repeated sampling of (xi, yi) values and not to repeated sampling of yi at fixed levels of xi.

      2.13.2 x and y Jointly Normally Distributed: Correlation Model

      Now suppose that y and x are jointly distributed according to the bivariate normal distribution. That is,

      (2.60) image

      where μ1 and in54-1 the mean and variance of y, μ2 and in54-2 the mean and variance of x, and

ueqn54-1

      is the correlation coefficient between y and x. The term σ12 is the covariance of y and x.

      The conditional distribution of y for a given value of x is

      (2.61) image

      where

      (2.62a) image

      (2.62b) image

      (2.62c) image

      That is, the conditional distribution of y given x is normal with conditional mean

      (2.63) image

      and conditional variance in55-1. Note that the mean of the conditional distribution of y given x is a straight-line regression model. Furthermore, there is a relationship between the correlation coefficient ρ and the slope β1. From Eq. (2.62b) we see that if ρ = 0, then β1 = 0, which implies that there is no linear regression of y on x. That is, knowledge of x does not assist us in predicting y.

      The method of maximum likelihood may be used to estimate the parameters β0 and β1. It may be shown that the maximum-likelihood estimators of these parameters are

      and

      (2.64b) image

      It is possible to draw inferences about the correlation coefficient ρ in this model. The estimator of ρ is the sample correlation coefficient

      (2.65) image

      Note that