General Model

We have a quantitative response $Y$ and different predictors $X_1,X_2,\dots ,X_p$. $X=(X_1,\dots ,X_p)$

Which we can model as

Prediction

With new predictor values $x_o$ predict response $Y_0$

Inference

Understand the relationship between predictor and response in more detail and use that to make decisions. Quantify uncertainty.

Regression Model

Regression Analysis
Given observations $\{(x_i,y_i){\}}^{i=n}$ find relationship

where $x_i$ is assumed to be constant and $\mathbb{E}[{\epsilon }_i]=0⟹\mathbb{E}[y_i]=f(x_i)$
$f$ describes the systematic influence of $x_i$ on $y_i$

Prediction

Prediction ${\hat{y}}_0=\hat{f}(x_0)$
To measure the accuracy of our prediction we use MSE:

With the assumption that $\hat{f}(x_0)$ is independent of ${\epsilon }_0$

$\mathbb{E}[\hat{f}(x_0)]-f(x_0)$ is called bias.

It describes how far from the truth the prediction $\hat{f}(x_0)$ is on average.  
$\mathrm{Var}(\hat{f}(x_0))$ describes how much variation the estimator $\hat{f}(x_0)$ has.