Linear Regression
Linear regression is a statistical method used to model the relationship between a dependent variable 𝑦 and one or more independent variables 𝑥 by fitting a linear equation of the form:
Types of Lines of Regression:
- Line of Regression of y on x
- Line of Regression of x on y
Line of Regression of y on x
Equation | y – ȳ = byx(x – x̄) |
(i) Mean of x (x̄) | = ∑x/n |
(ii) Mean of y (ȳ) | = ∑y/n |
(iii) byx | = ∑\text{xy} - {1\over n}∑\text{x}∑\text{y}\over ∑\text{x}^2 - {1\over n}(∑\text{x})^2 |
= \text{cov(x,y)}\over \text{var (x)} | |
= \text{cov(x, y)}\over \text{σ}_\text{x} | |
= r\text{σ}_\text{y}\over \text{σ}_\text{x} | |
= ∑\text{(x - x̄)(y - ȳ)}\over ∑\text{(x - x̄)}^2 | |
= ∑\text{uv} - {1\over n}∑\text{u}∑\text{v}\over ∑\text{u}^2 - {1\over n}(∑\text{u})^2 |
Line of Regression of x on y
Equation | x – x̄ = byx(y – ȳ) |
(i) Mean of x (x̄) | = ∑x/n |
(ii) Mean of y (ȳ) | = ∑y/n |
(iii) byx | = ∑\text{xy} - {1\over n}∑\text{x}∑\text{y}\over ∑\text{y}^2 - {1\over n}(∑y)^2 |
= \text{cov(x,y)}\over \text{var (y)} | |
= \text{cov(x, y)}\over \text{σ}_\text{y} | |
= r\text{σ}_\text{x}\over \text{σ}_\text{y} | |
= ∑\text{(x - x̄)(y - ȳ)}\over ∑\text{(y - ȳ)}^2 | |
= ∑\text{uv} - {1\over n}∑\text{u}∑\text{v}\over ∑\text{v}^2 - {1\over n}(∑\text{v})^2 |
Properties of Lines of Regression
- bxy, byx and r or ρ (x, y) are of same sign
- Square of Coefficient of correlation (r2) = bxy. byx
- 0 ≤ bxy. byx ≤ 1 or 0 ≤ r2≤ 1
- If one of bxy. byx is numerically greater than one, the other is numerically smaller than one.
- The two regression lines intersect at (x̄, ȳ)
- If r = 0, then the lines of regression are parallel to the coordinate axes.
- The angle between two regression lines (θ) is given by tan θ = |{1 - \text{r}^2\over \text{b}_{\text{xy}} + \text{b}_{\text{yx}}}|