{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Linear regression"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"*The explanations of statistical concepts are partly based on {cite:t}`Field2013`*\n",
"\n",
"Linear regression is the fundamental starting point for all regression methods and it is still a useful and widely used statistical learning method. Moreover, it serves as a good jumping point for newer approaches. Consequently, the importance of having a good understanding of linear regression before studying more complex learning methods cannot be overstated {cite:p}`James2021`.\n",
"\n",
":::{note}\n",
" Linear regression is a useful tool for predicting a quantitative response\n",
":::\n",
"\n",
"\n",
"One of the simplest models we use in statistics is the **mean**. It is a (simple) model because it represents a summary of data. Therefore, let's use the mean as a baseline model and compare the quality of fit between the mean and a simple linear regression model with only one predictor. \n",
"\n",
"We use a small sample of 20 women from whom we obtained their height (this is our outcome variable) and the average height of their parents (this is our feature)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Python setup"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"from scipy import stats\n",
"import statsmodels.formula.api as smf\n",
"\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline \n",
"import seaborn as sns \n",
"\n",
"# seaborn settings\n",
"custom_params = {\"axes.spines.right\": False, \"axes.spines.top\": False}\n",
"sns.set_theme(style=\"ticks\", rc=custom_params)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Import data"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# import data\n",
"ROOT = \"https://raw.githubusercontent.com/kirenz/modern-statistics/main/data/\"\n",
"DATA = \"height-clean.csv\"\n",
"df = pd.read_csv(ROOT + DATA)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
name
\n",
"
id
\n",
"
height
\n",
"
average_height_parents
\n",
"
gender
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
Stefanie
\n",
"
1
\n",
"
162
\n",
"
161
\n",
"
female
\n",
"
\n",
"
\n",
"
1
\n",
"
Peter
\n",
"
2
\n",
"
163
\n",
"
163
\n",
"
male
\n",
"
\n",
"
\n",
"
2
\n",
"
Stefanie
\n",
"
3
\n",
"
163
\n",
"
163
\n",
"
female
\n",
"
\n",
"
\n",
"
3
\n",
"
Manuela
\n",
"
4
\n",
"
164
\n",
"
165
\n",
"
female
\n",
"
\n",
"
\n",
"
4
\n",
"
Simon
\n",
"
5
\n",
"
164
\n",
"
163
\n",
"
male
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name id height average_height_parents gender\n",
"0 Stefanie 1 162 161 female\n",
"1 Peter 2 163 163 male\n",
"2 Stefanie 3 163 163 female\n",
"3 Manuela 4 164 165 female\n",
"4 Simon 5 164 163 male"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# show the first rows (i.e. head of the DataFrame)\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# check relationship with a joint plot\n",
"sns.jointplot(x=\"height_parents\", y=\"height\", hue=\"gender\", data=df);"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"We can observe a strong positive relationship between height and the average height of the parents. Hence, it would make sense to use the variable `height_parents` as a predictor for the outcome variable `height` in a statistical model."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Model"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"If we are interested in discovering something about a phenomenon in the real world, we need to collect data to test predictions from our hypotheses about that phenomenon. Testing these hypotheses involves building statistical **models** of the phenomenon of interest.\n",
"\n",
"It is important that the model accurately represents the real world, otherwise any conclusions we extrapolate to the real-world will be meaningless. Hence, the statistical model should represent the data collected (the observed data) as closely as possible. \n",
"\n",
"$$Outcome_i = (Model) + error_i$$\n",
"\n",
"This equation means that the data we observe can be predicted from the model we choose to fit plus some amount of error. There are different terms that basically refer to **error** like **residual**, **deviation** or **deviance**. The degree to which a statistical model represents the data collected is known as the **fit of the model** which is closely related to the error of the model.\n",
"\n",
"The model in the equation will vary depending on the design of your study, the type of data you have and what it is you’re trying to achieve with your model. Consequently, the model can also vary in its complexity.\n",
"\n",
"The important thing is that we can use the model computed in our **sample** to estimate the value in the **population** (which is the value in which we’re interested).\n",
"\n",
"**Parameters**\n",
"\n",
"Statistical models are made up of **variables** and **parameters**. Variables are measured constructs that vary across entities in the sample. In contrast, parameters are not measured and are (usually) constants believed to represent some fundamental truth about the relations between variables in the model.\n",
"\n",
"Some examples of parameters with which you already are familiar are: the **mean** and **median** (which estimate the centre of the distribution). We will also cover correlation and regression coefficients (which estimate the relationship between two variables) in other applications.\n",
"\n",
"If we’re interested only in summarizing the outcome, as we are when we compute a **mean**, then we won’t have any variables in the model, only a **parameter** (typically called *b*), so we could write our $Outcome_i = (Model) + error_i$ equation as:\n",
"\n",
"$Outcome_i = (b) + error_i$\n",
"\n",
"\n",
"Let's say we would like to compare the **quality of fit** of two models to predict the height of women in our dataset: the simple mean and a second model in which we use information about the average height of their parents as a predictor in a linear regression model.\n",
"\n",
"\n",
"**Model 1: Mean**\n",
"\n",
" * In the case of the **mean**, the *b* parameter is usually called $\\bar{x}$, which leads to:\n",
"\n",
"$height_i = (\\bar{x}) + error_i$\n",
" \n",
" * with\n",
"\n",
"$\\bar{x} = \\frac {\\sum_{i=1}^n x_{i}}{n}$\n",
"\n",
"\n",
"**Model 2: Linear Regression**\n",
"\n",
" * In our second model, we use the variable height of parents as predictor in a linear regression model:\n",
"\n",
"$height_i = (b_0 + b_1 \\times heightparents_i ) + error_i$\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Mean"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"165.0"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# calculate the mean \n",
"df[\"height\"].mean()"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
Notes: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. [2] The condition number is large, 1.67e+04. This might indicate that there are strong multicollinearity or other numerical problems."
],
"text/plain": [
"\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: height R-squared: 0.831\n",
"Model: OLS Adj. R-squared: 0.822\n",
"Method: Least Squares F-statistic: 88.78\n",
"Date: Sun, 05 Dec 2021 Prob (F-statistic): 2.21e-08\n",
"Time: 13:23:53 Log-Likelihood: -17.995\n",
"No. Observations: 20 AIC: 39.99\n",
"Df Residuals: 18 BIC: 41.98\n",
"Df Model: 1 \n",
"Covariance Type: nonrobust \n",
"==================================================================================\n",
" coef std err t P>|t| [0.025 0.975]\n",
"----------------------------------------------------------------------------------\n",
"Intercept 30.9651 14.226 2.177 0.043 1.077 60.853\n",
"height_parents 0.8121 0.086 9.422 0.000 0.631 0.993\n",
"==============================================================================\n",
"Omnibus: 4.700 Durbin-Watson: 1.384\n",
"Prob(Omnibus): 0.095 Jarque-Bera (JB): 2.492\n",
"Skew: -0.684 Prob(JB): 0.288\n",
"Kurtosis: 4.058 Cond. No. 1.67e+04\n",
"==============================================================================\n",
"\n",
"Notes:\n",
"[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
"[2] The condition number is large, 1.67e+04. This might indicate that there are\n",
"strong multicollinearity or other numerical problems.\n",
"\"\"\""
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# summary of regression results\n",
"lm.summary()"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"193.39\n"
]
}
],
"source": [
"# This is just a simple example of how regression works.\n",
"\n",
"# our parameters\n",
"b_0 = 30.9651\n",
"b_1 = 0.8121\n",
"\n",
"# Make a prediction for X=200\n",
"X = 200\n",
"\n",
"prediction = b_0 + b_1*(X)\n",
"\n",
"print(round(prediction,2))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"We use [Seaborn's lmplot](https://seaborn.pydata.org/generated/seaborn.lmplot.html) to plot the regression line:"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVwAAAFcCAYAAACEFgYsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3RklEQVR4nO3de1iVVd7/8TeCCJRYRni4cJzSEjSPY9PYUFloSZ5SyzyMaVZqPqNP2JOSYhaFmjV20EopRy21IsTUbNJEMbHSbMwDoz8nTw2jaIaGHGVv7t8fKzVKCZR93xv4vK6Ly9aWfe8vKB/v1l7ru3wsy7IQERGPq+V0ASIiNYUCV0TEJgpcERGbKHBFRGyiwBURsUmVD1yXy0VmZiYul8vpUkREylTlAzcrK4uoqCiysrKcLkVEpExVPnBFRKoKBa6IiE0UuCIiNlHgiojYRIErImITBa6IiE0UuCIiNlHgiojYRIErImITjwZubm4uPXr0IDMzE4Bt27bRv39/unfvzrhx4zh9+jQAGRkZ9OvXj169ejFy5EhycnI8WZaISJm27j7KxDc28VDCp0x8YxNbdx+tlOt6LHC3b9/OwIEDOXjwIGDCd8yYMcTHx7Nq1SoAkpOTAUhISGDs2LGsWLGCa665hnnz5nmqLBGRMm3dfZS5KTs4kVNA3UA/TuQUMDdlR6WErl8l1HdeSUlJTJkyhfHjxwOwadMm2rVrR3h4OABxcXG43W4ASkpKyMvLA6CgoIB69eqd95o5OTm/uvtVDwURqUwpad/i5+dDgL+JxwB/PwpxkZL2LR0jGlzStT0WuAkJCaXGhw4dIigoiJiYGPbv30+HDh2IjY0FIDY2luHDhzN16lQCAwNJSko67zUXLlzI7NmzPVWyiAhHs/OpG1g6GuvU9uVYdv4lX9u2N83cbjfp6emMGzeOlJQUCgoKSExMpLCwkEmTJrFgwQLS09MZNGgQEyZMOO81hg4dSmpqaqmPxYsX2/UliEgN0KB+EEXF7lKPFRW7Ca0fdMnXti1wQ0JCaNu2LU2aNMHX15fo6Gh27NjB3r17qVOnDm3atAHg/vvvZ8uWLee9RnBwMGFhYaU+GjZsaNeXICI1QN/OzXG5LApPu7As86vLZdG3c/NLvrZtgRsZGUlGRgZHjhwBYP369bRq1YqmTZuSlZXF/v37AUhNTaV169Z2lSUiUkrHiAaM7NuGK4MDyS1wcWVwICP7trnk+Vvw4BzuLzVq1Ij4+HhGjRpFUVERERERTJgwgcDAQKZNm8Zjjz2GZVlcddVVTJ061a6yRER+pWNEg0oJ2F/ysSzLqvSr2igzM5OoqChSU1MJCwtzuhwRkQvSTjMREZsocEVEbKLAFRGxiQJXRMQmClwREZsocEVEbKLAFRGxiQJXRMQmClwREZsocEVEbKLAFRGxiQJXRMQmClwREZsocEVEbKLAFRGxiQJXRMQmClwREZsocEVEbKLAFRGxiQJXROR81qyBTz+t1EvadmqviEiVYFnw1FPw3HPg5wc//ghBQZVyaQWuiMgZJSUwZgy8/roZ9+1baWELClwREaO4GIYNgyVLzPiBB2DevEp9Cc3hiogUFECfPufCduxYmD/fTClUIgWuiNRsP/4I3brBqlVm/Mwz8PLLUKvy41FTCiJScx07ZsJ22zYzfvVVM4frIQpcEamZvvsOunaFvXvB19dMIQwZ4tGXVOCKSM2zZ48J28xMqFMHPvgAevb0+MsqcEWkZvn6azONcPw41K0LK1ZA5862vLQCV0Rqjg0bzJ3sqVMQEgKffAJ/+INtL69VCiJSM6xcCXfdZcI2LAw2brQ1bEGBKyI1waJFZp1tURFcdx2kp0N4uO1lKHBFpHqbPdusPnC7oV07E7ZNmzpSigJXRKony4L4+HPraiMjIS0NQkMdK8mjb5rl5uYyYMAA5syZQ1hYGNu2bWPatGnk5eXRokULpk+fzr59+4iNjT37nOzsbOrVq8dHH33kydJEpDorKYFx4+CVV8z47rvN0q9KbERzMTwWuNu3bycuLo6DBw8CJnzHjBnDW2+9RXh4OOPGjSM5OZlBgwaxfPlyAAoKCrjvvvt4+umnPVWWSI20dfdRUtK+5Wh2Pg3qB9G3c3M6RjRwuizP1OVywUMPwdtvm/GgQbBgAdSufcn1XiqPTSkkJSUxZcoUQn+6fd+0aRPt2rUj/KeJ6ri4OLp27VrqOXPnzuXGG2+kY8eOnipLpMbZuvsoc1N2cCKngLqBfpzIKWBuyg627j5a/eoqLIR+/c6F7ejR8M47XhG24ME73ISEhFLjQ4cOERQURExMDPv376dDhw6lphJOnTpFUlISK1euvOA1c3JyyMnJKfVYVlZW5RYuUs2kpH2Ln58PAf7mxz3A349CXKSkfevoXW6l15WTA/fcA+vXm3FcnJnD9fGpvKIvkW0bH9xuN+np6bz//vs0btyYSZMmkZiYyJifJrRXrFhBly5duOqqqy54jYULFzJ79my7ShapFo5m51M3sPSPep3avhzLzneoIqNS6zp+HKKjYetWM545E2JiKqHKymVb4IaEhNC2bVuaNGkCQHR0NIsWLTr7+2vXrmXkyJFlXmPo0KH06dOn1GNZWVkMHjy48gsWqSYa1A/iRE7B2TtJgKJiN6H1nX0DqdLqysw0fRH27DEtFd96Cx58sJKrrRy2LQuLjIwkIyODI0eOALB+/XpatWoFgGVZZGRk0L59+zKvERwcTFhYWKmPhg0berx2kaqsb+fmuFwWhaddWJb51eWy6Nu5edWv69//hj//2YStvz8kJ3tt2IKNd7iNGjUiPj6eUaNGUVRUREREBBMmTADMUrDatWtTp04du8oRqTE6RjSAvm1ISfuWY9n5hHrJKoVLruubb8xW3WPH4LLLYPlyiIryaM2XyseyLMvpIi5FZmYmUVFRpKamEhYW5nQ5ImKH9HTo0cOc1lC/PvzjH/DHPzpd1W/STjMRqVo+/hjuvNOEbePG8NlnVSJsQYErIlXJe+9B797m0Mdmzcyd7k/vBVUFClwRqRreeMPsGnO5oE0bE7bXXON0VRWiwBUR72ZZMHWq2TVmWdCpk2lCUwVXKClwRcR7WRaMHw+TJpnxXXfBp5/ClVc6W9dF0hE7IuKdXC4YORL+/ncz7t/f9EXw93e2rkugO1wR8T5FRXD//efCdsQIWLKkSoctKHBFxNvk5kL37pCSYsaxsTBnDvj6OltXJdCUgoh4j+xs0yx882Yzfv55M4dbTShwRcQ7HD5sNjRkZJiWinPnwiOPOF1VpVLgiojz9u0zHb8OHDDNwhcvhvvuc7qqSqfAFRFn7dxp7myzssyZY8uWmXE1pMAVEed88YWZsz15Eq64wvRJ6NTJ6ao8RqsURMQZa9ZAly4mbBs2hA0bqnXYggJXRJyQnGzaK+bnm34I6emmP0I1p8AVEXu99ZbZ1FBcbDp9paebzl81gAJXROzzwgtmqVdJCdx0k+ll27ix01XZRoErIp5nWfDkk+c2MXTpAmvXmtMaahCtUhARz3K7TWvFxEQz7tvX9EWogWcY6g5XRDzn9GnTNPxM2A4fDu+/XyPDFhS4IuIpeXnQqxckJZnx44+bN8z8au7/WNfcr1xEPOfkSdPx6/PPzTghwczh+vg4WpbTFLgiUrmysszJDDt2mIB97TV49FGnq/IKClwRqTwHDpgmNPv2mamDt9+GgQOdrsprKHBFpHL8618mbA8fhoAAWLrU9EmQsxS4InLptmyB6GjTQDw4GD76CG65xemqvI5WKYjIpVm3DqKiTNiGhpojzBW256XAFZGL9+GH5s42Nxd+9zvYuBHat3e6Kq+lwBWRi7NgAfTrZzY3hIfDpk1w/fVOV+XVFLgiUnEvvwwPPmia0HTsaO5sw8KcrsrrKXBFpPwsC556CmJizLhzZ0hNhZAQR8uqKrRKQUTKp6QE/vd/YfZsM+7Vy/RFCAhwtq4qRIErIr+tuBiGDTNdvgAeeADmzavRfREuhqYURKRsBQXQp8+5sB07FubPV9heBAWuiFzYjz9Ct26wapUZP/OMecOslqLjYnj0n6jc3FwGDBjAnDlzCAsLY9u2bUybNo28vDxatGjB9OnT8ff3Z//+/UyZMoUff/yRq6++mpkzZ1KvXj1PliZSo0x8fSM792WfHbduVp+po39jc8KxYyZst20z41deMXe3ctE89s/U9u3bGThwIAcPHgRM+I4ZM4b4+HhW/fSvZXJyMpZl8eijj/LII4+wYsUKIiIiSDzTrFhELtkvwxZg575sJr6+8cJP+u47s1ts2zbw9YV33lHYVgKP3eEmJSUxZcoUxv90htGmTZto164d4eHhAMTFxeF2u8nIyCAoKIhbb70VgFGjRpGTk3Pea+bk5Pzq97Kysjz1JYhUC78M2996nD17TBOazExzMsMHH0DPnh6ssObwWOAmJCSUGh86dIigoCBiYmLYv38/HTp0IDY2lq+++oqQkBAmTpzI7t27ufbaa5k8efJ5r7lw4UJmn1mSIiKV7+uvzTTC8eNQty6sWGHW2kqlsG3m2+12k56ezrhx40hJSaGgoIDExERcLhdbtmxh4MCBLFu2jCZNmjB9+vTzXmPo0KGkpqaW+li8eLFdX4JI9bZhA9x+uwnbkBBYv15hW8lsC9yQkBDatm1LkyZN8PX1JTo6mh07dnD11VfTtGlTWrduDUCPHj3YsWPHea8RHBxMWFhYqY+GDRva9SWIVEmtm53/KPJSj69caU5pOHXKbNHduBH+8AebKqw5bAvcyMhIMjIyOHLkCADr16+nVatWtG/fnuzsbPbs2QPAunXraNWqlV1liVR7U0ff8qvQLbVKYdEis862qMg0n9m0yTSjkUpn28rlRo0aER8fz6hRoygqKiIiIoIJEyYQEBDAa6+9RlxcHAUFBTRs2JAZM2bYVZZIjXDBJWCzZp1bfdC+PXzyielpKx7hY1mW5XQRlyIzM5OoqChSU1MJU7cikfKxLHj2WZgyxYxvucVMK2j9u0dpb55ITVNSAuPGmY0MYI4z/+ADCAx0tq4aQIErUpO4XPDQQ+Y0XYBBg0wj8dq1HS2rptCGaJGaorAQ7r33XNiOHm12kClsbaPAFakJcnLMkeXLl5txXJzpa6smNLbSlIJIdXf8uDnocetWM54589yJDWIrBa5IdZaZafoi7Nlj7mbfesucRSaOUOCKVFf//jd06WI6f/n7w3vvmQ0O4hgFrkh19M03ZqvusWNw2WVm7jYqyumqajwFrkh1k54OPXqY0xrq14d//AP++EenqxK0SkGkevn4Y7jzThO2jRvDZ58pbL2IAlekunj3Xejd2xz62KyZaUKjRlBeRYErUh288QYMHmx2krVpY6YVfv97p6uSX1DgilRllgVTp5pdY5YFN98MaWmgPtFeSYErUlVZFowfD5MmmfFdd8GaNXDllc7WJRekVQoiVZHbDSNHwrx5ZnzffaaRuL+/s3VJmXSHK1LVFBXB/fefC9tHHjFvmClsvZ4CV6Qqyc01a2yXLjXjCRNg7lzw9XW2LikXTSmIVBXZ2abj1+bNZvz882YOV6oMBa5IVXD4sNnQkJFhmtDMnQsPP+x0VVJBClwRb7dvn+n4deCAaRa+eLF5k0yqHAWuiDfbudPc2WZlQVAQLFtmxlIlKXBFvNUXX5g525Mn4YorTJ+ETp2crkougVYpiHijNWtML9uTJ82usc8+U9hWAwpcEW/zwQdm6Vd+PlxzjemL0Lq101VJJVDginiTN9+EAQOguNh0+kpPN52/pFpQ4Ip4ixkzYMQIKCmBm24y0wiNGztdlVQiBa6I0ywLYmPNrjEwc7dr15rTGqRa0SoFESe53aa1YmKiGffrZ9bZ1qnjbF3iEQpcEaecPg1DhkBSkhkPH252kPnpx7K60pSCiBPy8sxxOGfC9vHH4a23FLbVXLkCd+3atb967MMPP6zsWkRqhhMnzG6xTz4x44QEeOEF8PFxti7xuDL/OV23bh0ul4sZM2ZQUlJy9nGXy8WsWbO45557PF2fSPWSlWVOZtixwwTsa6/Bo486XZXYpMzA3b17N19++SU//PAD77zzzrkn+fkxbNgwT9cmUr0cPGhWIOzbZ6YO3n4bBg50uiqxkY9lWdZvfdLixYsZPHiwHfVUWGZmJlFRUaSmphIWFuZ0OSLnl5FhphEOH4aAANNA/O67na5KbFauGfp+/fqxYsUKsrOz+Xk+P/jgg2U+Lzc3lwEDBjBnzhzCwsLYtm0b06ZNIy8vjxYtWjB9+nT8/f2ZPXs2S5cuJTg4GID+/ft7bcCLVNiWLRAdbRqI16sHH30EkZFOVyUOKFfgPv744xw5coTrr78en3JO7G/fvp24uDgOHjwImPAdM2YMb731FuHh4YwbN47k5GQGDRrErl27mDlzJu3bt7/oL0TEG0x8fSM792WfHd9TfICH5k82R+OEhsLq1dCune11bd19lJS0bzmanU+D+kH07dycjhENbK+jqtTlKeUK3L1797J69Wpq1Sr/KrKkpCSmTJnC+J+OANm0aRPt2rUjPDwcgLi4ONxuNwC7du1i7ty5/Pe//+XGG29kwoQJ1NHCb6lifhm2f/r3lzzw8YvgdsHvfgeffgrXX297XVt3H2Vuyg78/HyoG+jHiZwC5qbsgL5tHA03b63Lk8qVoFdddRUul6tCF05ISKBjx45nx4cOHSIoKIiYmBh69+7NrFmzCA4OJi8vj4iICJ544gmWLVtGTk4Or7/++nmvmZOTQ2ZmZqmPrKysCtUl4ik/D9uojFRiP5pBbbeL/9QPg02bHAlbgJS0b/Hz8yHA3w8fH/Orn58PKWnfOlKPt9flSWXe4c6fPx+Aq6++miFDhhAVFUXt2rXP/v5vzeH+nNvtJj09nffff5/GjRszadIkEhMTGTNmDG+++ebZzxs+fDgTJ04kJibmV9dYuHAhs2fPLvdrijih99creHjD3wH4d4PmPN33KRY7+Ibu0ex86gaW/lGvU9uXY9n5DlVkeGtdnlRm4O7duxeAyy+/nMsvv5wDBw5c9AuFhITQtm1bmjRpAkB0dDSLFi3i8OHDfP7559x7770AWJaF3wV22wwdOpQ+ffqUeiwrK0tvsIl3sCwGf76EAZs/AGB7k9Yk9HqSgjpBjpbVoH4QJ3IKCPA/93NVVOwmtL7qsluZgTtt2rRKe6HIyEhmzZrFkSNHaNSoEevXr6dVq1YEBATwwgsvcNNNNxEWFsbixYvp2rXrea8RHBx8diWDiFcpKWHi12/TafMyAL5s9kdmdP8/iv38ad3M2a5ffTs3Z27KDgpxUae2L0XFblwui76dm6sum5XrTbM77rij1OoEHx8fAgMDue6664iNjSU0NPQ3r9GoUSPi4+MZNWoURUVFREREMGHCBAIDA4mPj+fRRx+luLiYDh06VGiqQsRxxcUwbBidPjNhm9rydl6986+U1PKldbP6TB19i6PldYxoAH3bkJL2Lcey8wn1ktUA3lqXJ5Vr48PUqVPJy8tj8ODB1KpVi+Tk5LNrab/88kvmzJljR63npY0P4qiCAnNk+apVZjx2LLz0ElRgRY/UHOX6W7F161YSEhJo2bIl4eHhxMXF8e9//5thw4bx3//+19M1ininH380fRHOhO0zz8DLLyts5YLKNaWQl5dHbm4ul19+OWA2MRQWFnq0MBGvduwYdOsG27aZ8auvwpgxztYkXq/cW3v79+9Pt27dsCyLNWvWcN999/HOO+9w7bXXerpGEe/y3XfQtSvs3Qu+vrBgAfzlL05XJVVAuQJ3xIgRtGzZkg0bNuDn58fkyZP505/+xK5du361TEukWtuzx4RtZqY5BicpCXr1croqqSLKDNx9+/bRrFkzMjIyuPLKK0v1v83IyOCGG27wdH0i3uPrr800wvHjULcurFgBnTs7XZVUIWUG7owZM5g7dy5jzjM35ePjQ2pqqscKE/EqaWnmTvbUKQgJMac1/OEPTlclVUyZgTt37lzAnPwgUmOtXGmWfhUVQViYaULzUxMmkYoo1/qVvLw84uPjGTp0KCdPnuSpp54iLy/P07WJOG/RIujTx4TtdddBerrCVi5auQL3ueeeo27duvzwww/UqVOH3NxcnnrqKU/XJuKsWbPMMeZut+lhm54OTZs6XZVUYeUK3N27dxMTE4Ofnx+BgYG8+OKL7N6929O1iTjDsswmhrFjzfiWW8wcbjm2sIuUpVzLwn7ZeNztdleoGblIlVFSAjExZiMDmHPHPvgAgqpvByuxT7kC98Ybb+SFF16gsLCQjRs3smjRIv74xz96ujYRe7lc8NBD5jRdMCfqLlwIP+sBLXIpynWb+n//938EBQVRt25dXn75ZcLDw4mNjfV0bSL2KSyEe+89F7ajR5s3zBS2UonKFbg7duzgq6++orCwkNOnT7Nu3Tr69evn6dpE7HHqlJk6WL7cjOPiYPZsNaGRSleuKYXJkyfTv39/IiIiyn1qr0iVcPy4OcJ861YznjnTzOGKeEC5Atff359hw4Z5uBQRm2Vmwp13wu7d5m523jzQ33PxoHL9P9O1117Lzp07PV2LiH327oU//9mErb8/JCcrbMXjyrzD7dmzJ2B2mg0cOJAmTZqUOuBx5cqVnq1OxBO++cY0Dj92DC67zMzdRkU5XZXUAGUG7uTJk+2qQ8Qe6enQvTvk5ED9+vCPf4CWOIpNygxcrbWVauUf/4B+/cw5ZI0bmyY0LVs6XZXUIFr3IjXDu++a9ooFBdC8OWzapLAV2ylwpfp74w0YPNjsJGvTBjZuhN//3umqpAZS4Er1ZVmQkGB2jVkW3HwzbNgADRs6XZnUUOVahytS5VgWPPEE/O1vZnzXXbB0qVmVIOIQBa5UPy4XjBwJf/+7Gd9/v+mR4O/vbF1S42lKQaqXoiITsGfCduRIWLxYYSteQYEr1UduLvToASkpZjxhgnnDzNfX2bpEfqIpBakesrNNx6/Nm814xgwzhyviRRS4UvUdPmya0GRkmCY0c+fCww87XZXIryhwpWrbtw+6doUDB0yz8CVLTCNxES+kwJWqa8cOs9wrK8ucObZsmbnTFfFSClypmr74wszZnjwJV1wBH38MnTo5XZVImbRKQaqeNWugSxcTtg0bmt1jClupAhS4UrUkJ5ulX/n5ph9CerrpjyBSBShwpep46y2zqaG4GFq1Mh2/mjVzuiqRcvPoHG5ubi4DBgxgzpw5hIWFsW3bNqZNm0ZeXh4tWrRg+vTp+P9sB1BaWhrx8fGsW7fOk2VJBWzdfZSUtG85mp1Pg/pB9O3cnI4RDewv5IUXYPx489833cTS/3uZD17/hoKirQTW8aP3rdcy8M5w++v6Ba/5folX8tgd7vbt2xk4cCAHDx4ETPiOGTOG+Ph4Vq1aBUBycvLZzz9+/DjPP/+8p8qRi7B191HmpuzgRE4BdQP9OJFTwNyUHWzdfdS+IiwLnnzyXNh26cIHT77GO5uPUXjahV8tKDzt4v1P9/Lumj321XUeXvH9Eq/mscBNSkpiypQphIaGArBp0ybatWtHeLi5C4mLi6Nr165nPz8uLo6//vWvZV4zJyeHzMzMUh9ZWVme+hJqvJS0b/Hz8yHA3w8fH/Orn58PKWnf2lOA2w2jRsH06Wbcty989BFLvzoKPuBXqxY+PrXwq1ULfGD5Z/vtqesCHP9+idfz2JRCQkJCqfGhQ4cICgoiJiaG/fv306FDB2JjYwF4++23admyJW3bti3zmgsXLmT27NmeKll+4Wh2PnUDS/8VqVPbl2PZ+Z5/8dOnYcgQSEoy4wcfhMRE8POjoMjc2f6crw8UFLk8X1cZHP1+SZVg2zpct9tNeno677//Po0bN2bSpEkkJiZy1113sWbNGhYsWPCbd6tDhw6lT58+pR7Lyspi8ODBniy9xmpQP4gTOQUE+J/7a1JU7Ca0fpBnXzgvz+wW++QTM378cTOH6+MDQGAdPzOd4HPuKW7LPO4kx75fUmXYtkohJCSEtm3b0qRJE3x9fYmOjmbHjh188sknfP/99/Tr148RI0Zw7NgxBg0adN5rBAcHExYWVuqjobr3e0zfzs1xuSwKT7uwLPOry2XRt3Nzz73oiRNmt9iZsJ06tVTYAvS+9VqwwFVSgmWV4CopAeunxx3kyPdLqhTbAjcyMpKMjAyOHDkCwPr162nVqhVjx45l9erVLF++nMTEREJDQ1myZIldZUkZOkY0YGTfNlwZHEhugYsrgwMZ2beN5951z8qCzp3h889NwL7xhnnD7GdhCzDwznDu73o9Af5+uEogwN+P+7te7/gqBdu/X1Ll2Pb/YI0aNSI+Pp5Ro0ZRVFREREQEEyZMsOvl5SJ1jGhgT2AcOGCa0OzbB35+8M47MGDABT994J3hjgfs+dj2/ZIqyceyLMvpIi5FZmYmUVFRpKamEhYW5nQ5cjEyMsw0wuHDEBhodpPdfbfTVYlUOjWvEWdt2QLR0aaBeL168NFHEBnpdFUiHqGtveKc1FS44w4TtqGhkJamsJVqTYErzli2zEwb5OVB06amCU27dk5XJeJRClyx3/z5Zp3t6dMQEWHC9rrrnK5KxOMUuGKvl16C4cOhpAQ6doTPPgO92Sk1hAJX7GFZMHkyjBtnxrffDuvWQUiIs3WJ2EirFMTzSkpgzBh4/XUz7t0b3nsPAgKcrUvEZgpc8aziYhg2zJymC/DAAzBvntncIFLDaEpBPCc/H/r0ORe2//u/5g0zha3UUApc8Ywff4Ru3eCnZvM884x5w6yW/spJzaVbDal8x46ZsN22zYxffdXM4YrUcApcqVyHDpm+CHv3gq8vLFgAf/mL01WJeAUFrlSePXtMx6/MTKhTx5zW0KuX01WJeA0FrlSOr7820wjHj0PdurByJdx2m9NViXgVvYMhly4tzWxkOH7cbGRYv15hK3IeCly5NCtXmjvbU6egSRPYuBH+8AenqxLxSgpcuXiLFpl1tkVFcP31pglNuPedwiDiLRS4cnFmzTLHmLvd0L69ubP93e+crkrEqylwpWIsC+LjYexYM77lFjNnGxrqbF0iVYBWKUj5lZRATIzZyACmgfgHH0BQkLN1iVQRClwpn+JiePhhePttMx40yGxqqF3b0bJEqhJNKchvKyw0JzScCdv/+R9zjLnCVqRCFLhStpwcc6ruihVmHBdn3jBTExqRCtOUglzY8eMmbLduNeOXXoLHHnO0JJGqTIEr5/ef/5gmNHv2mLvZefNMI3ERuWgKXPm1vXtNE5rvvgN/f3j/fbjnHqerEqnyFLhS2jffwF13mZ62l18OH34IUVFOVyVSLShw5Zz0dOje3bxRVr8+/OMf8Mc/Ol2VSLWht5rF+PhjM2ebkwONG5utugpbkUqlwBV4911zdHlBATRrBps2QcuWTlclUu0ocGu6OXNg8GBwuaBNGzOt8PvfO12VSLWkwK2pLAumToVHHzX/ffPNppF4w4ZOVyZSbelNs5rIsuCJJ+BvfzPju+6CpUvhssucrUukmlPg1jQuF4wYAfPnm3H//qYvgr+/s3WJ1AAenVLIzc2lR48eZGZmArBt2zb69+9P9+7dGTduHKdPnwbg008/pWfPnnTv3p3Y2Nizj0slKyqC++8/F7YjRsCSJQpbEZt4LHC3b9/OwIEDOXjwIGDCd8yYMcTHx7Nq1SoAkpOTyc/PJz4+nvnz57Nq1SqKiopYtmyZp8qquXJzoUcPSEkx49hY84aZr2+ZT3t3zR4GxH1M7ydWMCDuY95ds8eGYkWqJ48FblJSElOmTCH0p5MANm3aRLt27Qj/6cyruLg4unbtSlBQEOvWrSMkJISCggJ++OEHgoODPVVWzZSdDV26wNq1Zvz88zBtGvj4lPm0d9fs4f1P91J42oVfLSg87eL9T/cqdEUuksfmcBMSEkqNDx06RFBQEDExMezfv58OHToQGxsLQO3atdmwYQPjx48nNDSUyMjI814zJyeHnJycUo9lZWV55guoLg4fNhsaMjJME5o5c+CRR8r11OWf7Qcf8PupFaOfD7hKSlj+2X4G3qnDIkUqyrZlYW63m/T0dMaNG0dKSgoFBQUkJiae/f3bbruNzZs3c/vtt/P000+f9xoLFy4kKiqq1MfgwYNt+gqqoH37IDLShG3t2qYJTTnDFqCgyIXvL26CfX3M4yJScbatUggJCaFt27Y0adIEgOjoaBYtWsTJkyfZtWvX2bvanj17EhMTc95rDB06lD59+pR6LCsrS6F7Pjt2mOVeWVnmzLFly8ydbgUE1vEz0wk/C123ZR4XkYqz7Q43MjKSjIwMjhw5AsD69etp1aoVlmXxxBNPcPjwYQA++eQTOnTocN5rBAcHExYWVuqjoRbq/9oXX8Btt5mwveIKM3dbwbAF6H3rtWCZaQTLKsFVUgLWT4+LSIXZdqvSqFEj4uPjGTVqFEVFRURERDBhwgQCAwN59tlnGTlyJD4+PjRv3pxnnnnGrrKqnzVroE8fyM83u8ZWrzZbdi/CmXna5Z/tp6DIRWAdP3rfeq3mb0Uuko9lWZbTRVyKzMxMoqKiSE1NJSwszOlynPXBB6YvQnExXHMNfPqpaUYjIl5BvRSqi7feggEDTNi2amWa0ChsRbyKArc6mDHDrD4oKYGbboLPPjM9bUXEqyhwqzLLgiefhAkTzPjM5ob69Z2tS0TOS+t7qiq3G0aPhjNrmfv1g8WLoU4dZ+sSkQtS4FZFp0/DkCGQlGTGw4fD3Lngpz9OEW+mKYWqJi8PevU6F7aPP27eMFPYing9/ZRWJSdOmI5fn39uxs89BxMn/mYTGhHxDgrcqiIry2zV3bHDBOxrr5njcUSkylDgVgUHDkDXrqYZjZ+fOaFhwACnqxKRClLgeruMDNMH4fBhCAyE5GS4+26nqxKRi6DA9WZbtkB0tGkgXq8efPSRabcoIlWSVil4q9RUuOMOE7ahoeYIc4WtSJWmwPVGy5aZaYO8PGja1PRFaNfO6apE5BIpcL3N/Plw771mc0NEhAnb665zuioRqQQKXG/y0ktm11hJCdx4o2lCU9NbTopUIwpcb2BZEBcH48aZ8e23mznckBBn6xKRSqVVCk4rKYExY+D11824d2947z0ICHC2LhGpdApcJxUXw7BhsGSJGT/wAMybp74IItWUphSckp9vzh47E7Zjx5o3zBS2ItWWAtcJP/4I3brBqlVmHB8PL78MtfTHIVKd6XbKbseOmSY033xjxq++auZwRaTaU+Da6dAh0xdh717w9YUFC+Avf3G6KhGxiQLXLnv2mI5fmZlmBUJSEvTs6XRVImIjBa4dtm41TWiOH4e6dWHlSrjtNqerEhGb6V0aT0tLM01ojh83GxnWr1fYitRQClxPWrHCrEY4dcps0d24Ef7wB6erEhGHKHA95Z13oG9fKCqC6683TWjCw52uSkQcpMD1hFmzzK4xtxvatzd3tk2bOl2ViDhMgVuZLAueecbsGgO45RYzZxsa6mxdIuIVtEqhspSUmG5fr7xixt27m6VfQUHO1iUiXkOBWxlcLnjoIXj7bTMeNMhsaqhd29GyRMS7aErhUhUWQr9+58L2f/7HvGGmsBWRX1DgXoqcHHP22IoVZjx5snnDTE1oROQ8NKVwsY4fN7vHtm4145kzISbG2ZpExKspcC9GZqbpi7Bnj7mbnTfPNBIXESmDRwM3NzeXAQMGMGfOHMLCwti2bRvTpk0jLy+PFi1aMH36dPz9/Vm7di2zZs3CsizCwsKYNm0a9erV80hNW3cfJSXtW45m59OgfhB9OzenY0SD8l9g714Ttt99B/7+8P77cM89HqlVRKoXj002bt++nYEDB3Lw4EHAhO+YMWOIj49n1U+Nt5OTk8nNzeXpp58mMTGRFStW0KJFC2bNmuWRmrbuPsrclB2cyCmgbqAfJ3IKmJuyg627j5bvAt98Y9bWfvcdXHYZfPyxwlZEys1jgZuUlMSUKVMI/WnR/6ZNm2jXrh3hP21vjYuLo2vXrhQXFzNlyhQaNDB3mS1atODIkSPnvWZOTg6ZmZmlPrKysspdU0rat/j5+RDg74ePj/nVz8+HlLRvf/vJ6enQubNpIF6/PqxbB1FR5X5tERGPTSkkJCSUGh86dIigoCBiYmLYv38/HTp0IDY2ljp16tC1a1cACgsLSUxMZMiQIee95sKFC5k9e/ZF13Q0O5+6gaW/5Dq1fTmWnV/2Ez/+GO69FwoKoHFjWLMGWrW66DpEpGaybf2S2+0mPT2dcePGkZKSQkFBAYmJiWd//9SpU4wYMYLw8HD69Olz3msMHTqU1NTUUh+LFy8udw0N6gdRVOwu9VhRsZvQ+mXsBnv3XXN0eUEBNGsGmzYpbEXkotgWuCEhIbRt25YmTZrg6+tLdHQ0O3bsAODYsWMMGjSIFi1a/OrO+OeCg4MJCwsr9dGwYcNy19C3c3NcLovC0y4sy/zqcln07dz8/E944w0YPNjsJGvTxkwr/P73FfmyRUTOsi1wIyMjycjIODs/u379elq1aoXb7WbUqFFER0czadIkfHx8PFZDx4gGjOzbhiuDA8ktcHFlcCAj+7b59SoFy4KEBBg92vz3zTfDhg1QgXAXEfkl29bhNmrUiPj4eEaNGkVRURERERFMmDCBdevW8a9//Qu3283q1asBuOGGG8q8070UHSMalL0MzLLgiSfgb38z427dIDnZrEoQEbkEPpZlWU4XcSkyMzOJiooiNTWVsLCwS7uYywUjR8Lf/27G/fubvgj+/pdeqIjUeNr0f0ZREQwYcC5sR4yAJUsUtiJSaRS4ALm50KMHLF1qxk8+CXPmgK+vs3WJSLWiXgrZ2abj1+bNZjxjhpnDFRGpZDU7cA8fhjvvhIwM04Rm7lx4+GGnqxKRaqrmBu6+faYJzYEDpln4kiVmN5mIiIfUzMDdvRvuuAOyssyZY8uWmTtdEREPqpmB+8ILJmyvuML0SejUyemKRKQGqJmBO2KEmbN97DG44QanqxGRGqJmBu6f/mQ+RERspHW4IiI2UeCKiNhEgSsiYhMFroiITRS4IiI2UeCKiNhEgSsiYhMFroiITRS4IiI2UeCKiNikym/tdbvdAGRlZTlciYiI0bBhQ/z8fh2vVT5wv//+ewAGDx7scCUiIsaFDrWt8qf2FhYWsmvXLq6++mp8y3kGWVZWFoMHD2bx4sU0bNjQwxWWn+qqGNVVMaqrYi6lrmp7hxsQEEDHjh0v6rkNGza89KPVPUB1VYzqqhjVVTGVWZfeNBMRsYkCV0TEJgpcERGb1MjADQ4O5q9//SvBwcFOl1KK6qoY1VUxqqtiPFFXlV+lICJSVdTIO1wREScocEVEbFKtAzc3N5cePXqQmZkJwLZt2+jfvz/du3dn3LhxnD59utTnjx8/npSUFK+pa+3atfTu3ZtevXoxevRofvzxR6+o69NPP6Vnz550796d2NjYX30fnarrjLS0NO644w6P1lSRumbPns3tt99O79696d27N4sXL/aKuvbv38+QIUPo1asXDz30kFf8/dq9e/fZ71Pv3r255ZZb6NGjh+N1AWRkZNCvXz969erFyJEjycnJqfiLWdXUN998Y/Xo0cNq1aqV9Z///Mc6deqU9ec//9navXu3ZVmWFRMTYy1evNiyLMvKysqyRo4cabVp08ZaunSpV9R15vGsrCzLsizr5Zdftp599lnH68rLy7MiIyOt77//3rIsy3rssces9957z/G6zvj++++tbt26WbfffrvHaqpoXSNHjrT++c9/erSeitZVUlJi3XnnndaGDRssy7KsF154wZoxY4bjdf1cfn6+1b17d+urr77yiroGDhxopaWlWZZlWdOmTbNmzpxZ4dertne4SUlJTJkyhdDQUAA2bdpEu3btCA8PByAuLo6uXbsCsHLlSqKiooiOjvaauoqLi5kyZQoNGjQAoEWLFhw5csTxuoKCgli3bh0hISEUFBTwww8/ePTd5Yr8OZ4Z//Wvf/VYPRdT165du5g7dy49e/YkPj6eoqIix+vKyMggKCiIW2+9FYBRo0Z5tB9JRf8cAebOncuNN9540TtJK7uukpIS8vLyACgoKCAgIKDCr1flt/ZeSEJCQqnxoUOHCAoKIiYmhv3799OhQwdiY2MBePjhhwH4+uuvvaauOnXqnP2DLiwsJDExkSFDhjheF0Dt2rXZsGED48ePJzQ0lMjISK+o6+2336Zly5a0bdvWY/VUtK68vDwiIiJ44oknaNq0KbGxsbz++uvExMQ4WtdXX31FSEgIEydOZPfu3Vx77bVMnjzZIzVVpK4zTp06RVJSEitXrvRYTRWtKzY2luHDhzN16lQCAwNJSkqq8OtV2zvcX3K73aSnpzNu3DhSUlIoKCggMTHR6bJ+s65Tp04xYsQIwsPD6dOnj9fUddttt7F582Zuv/12nn76acfr2rt3L2vWrGH06NG21VKeui677DLefPNNmjVrhp+fH8OHD2fDhg2O1+VyudiyZQsDBw5k2bJlNGnShOnTpzte1xkrVqygS5cuXHXVVbbVVFZdhYWFTJo0iQULFpCens6gQYOYMGFCha9fYwI3JCSEtm3b0qRJE3x9fYmOjmbHjh1Ol1VmXceOHWPQoEG0aNHiV/8SO1XXyZMnSU9PP/t5PXv25P/9v//neF2ffPIJ33//Pf369WPEiBFnv3dO13X48GGSk5PPfp5lWeftImV3XVdffTVNmzaldevWAPTo0cPWn4ff+nlcu3Ytd999t231/FZde/fupU6dOrRp0waA+++/ny1btlT4+jUmcCMjI8nIyDg7D7p+/XpatWrlcFUXrsvtdjNq1Ciio6OZNGkSPj4+XlGXZVk88cQTHD58GIBPPvmEDh06OF7X2LFjWb16NcuXLycxMZHQ0FCWLFnieF0BAQG88MIL/Oc//8GyLBYvXvyruUon6mrfvj3Z2dns2bMHgHXr1tn681DWz6NlWWRkZNC+fXvb6vmtupo2bUpWVhb79+8HTL/bM/9YVUS1ncP9pUaNGhEfH8+oUaMoKioiIiLiov6XwK661q1bx7/+9S/cbjerV68G4IYbbrDtTvdCdQUGBvLss88ycuRIfHx8aN68Oc8884wtNZVVl9PK+n7Fx8fz6KOPUlxcTIcOHXjwwQcdrysgIIDXXnuNuLg4CgoKaNiwITNmzHC8LoDs7Gxq165NnTp1bKvnt+oKDAxk2rRpPPbYY1iWxVVXXcXUqVMrfH1t7RURsUmNmVIQEXGaAldExCYKXBERmyhwRURsosAVEbGJAldstXnz5gp3f3rkkUf49ttvy/yc2NhY5s2bd97fmz17NmvXrq3Qa9rp1KlTPPDAA06XITaoMetwpep68803L+n5mzdvpnnz5pVUTeX78ccf2blzp9NliA0UuGK7/Pz8s81BioqKeO6552jTpg0vvvgiX331FW63m5YtWxIXF8fll1/OHXfcwSuvvELr1q1JTEwkOTmZyy67jI4dO5Kamsq6desA08d0wIABHD9+nOuuu46//e1vLFu2jF27djFjxgx8fX3L3OXVsmVLHnnkETZu3Eh+fj7jxo3jzjvvJD8/n6effppDhw5x8uRJLrvsMl588UWuvfZahgwZQr169di/fz8DBw7knnvuISEhgb1791JcXEynTp0YP348fn5+tG7dmhEjRrBp0yaOHTvGww8/zKBBg3jyyScpLCykd+/epKSk8Nprr/Hpp59Su3ZtrrzySqZNm3a2m5VUbZpSENtlZWUxbNgwli9fzoABA5g1axaJiYn4+vqSkpLCihUrCA0N5cUXXyz1vI0bN5KSkkJycjIpKSlnW+WdcfToUebPn8/q1as5evQoa9asYfDgwdxwww2MHz/+N7fUut1uAgMDSUlJ4eWXX2bixIlkZ2fz2WefERwczPvvv8/q1au54YYbSjURDw4O5uOPP2bIkCFMnTqVVq1akZKSwocffsiJEyeYP38+AKdPn+bKK6/kvffe49VXX2XatGkUFRUxbdo0AgICWL58OceOHWPhwoUsXbqUlJQU/vznP3tFzw+pHLrDFds1adLkbAvF8PBwli5dSlpaGqdOneLzzz8HoLi4+FedojZs2EC3bt3O9t8dPHgwX3755dnf79KlC4GBgQBcd911ZGdnV7i2v/zlL2fruv766/nqq6/o1q0bTZo04Z133uHQoUNs2bKl1D7/n/drTUtLY+fOnWcb1hQWFpa6flRUFACtWrXi9OnT5Ofnl/r9Bg0anO0Md+utt3LrrbfSqVOnCn8d4p0UuGK72rVrn/1vHx8fLMuipKSEiRMncttttwGQl5f3q0bdfn5+/Hwnuq+v769+/5fXraifX7OkpARfX1+WLFlCUlISgwcPpmfPnlxxxRVnj2MBCAoKKvWcV155hWbNmgGQk5NTqvHQmf4AZx77ZY21atVi0aJF7Ny5ky+++IKpU6dyyy23MH78+Ap/LeJ9NKUgXiEyMpLFixdz+vRpSkpKmDx5MjNnziz1Obfddhtr1qzh1KlTAKXaHpbF19cXl8tVrs/98MMPAXN+1YEDB7jxxhtJT0+nT58+3HfffVxzzTWsW7cOt9t9wa9jwYIFWJbF6dOnefTRR1m0aFGZr+nn54fb7cayLPbs2UOPHj1o1qwZI0eOZNiwYXpDrRrRHa54hdGjR/P888/Tp08f3G43ERERpU4AAOjUqRP9+/fn/vvvJyAggOuuu+7sFEJZ7rjjDmbOnElxcfFvNnH/5z//SVJSEiUlJbz00kvUq1eP4cOH89RTT50N+Hbt2rF3797zPn/SpEkkJCTQs2dPiouLufnmm8+eKHIhV199NW3atKF79+4sXryY6Oho+vXrR1BQEAEBAcTFxf3m1yhVg7qFSZWxc+dOtm3bdnbN6vz589m+fTsvv/xypVy/RYsWfPHFF9SvX79SrifyS7rDlSrjmmuu4c033yQpKQkfHx8aNWrEs88+W+7nv/XWWxc8I+uhhx6qrDJFLkh3uCIiNtGbZiIiNlHgiojYRIErImITBa6IiE0UuCIiNlHgiojY5P8D96osPjmorfMAAAAASUVORK5CYII=",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot regression line \n",
"sns.lmplot(x='height_parents', y='height', data=df, line_kws={'color':'red'}, height=5, ci=None);"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Quality of fit"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"In order to evaluate the performance of a statistical model on a given data set, we need some way to measure how well its predictions actually match the observed data. That is, we need to quantify the extent to which the predicted response value for a given observation is close to\n",
"the true response value for that observation. \n",
"\n",
"With most statistical models we can determine whether the model represents the data well by looking at how different the scores we observed in the data are from the values that the model predicts.\n",
"\n",
"$$observed_i = model_i + error_i$$\n",
"\n",
"hence\n",
"\n",
"$$error_i = observed_i - model_i$$\n",
"\n",
"In other words, the error for a particular entity is the score predicted by the model for that entity subtracted from the corresponding observed score."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Residuals and $R^2$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Note that residuals (errors) would cancel out because some are positive and others negative. Therefore, we square each deviation. If we add these squared deviations we get the **sum of squared residuals (SSR)** (also known as the residual sum of squares (RSS) or the sum of squared estimate of errors (SSE)).\n",
"\n",
"$ SSR = \\sum_{i=1}^n (observed_i - model_i)^2$ \n",
"\n",
"Using the **mean** of the outcome as a baseline model, we can calculate the difference between the observed values and the values predicted by the mean. We square these differences to give us the sum of squared differences. This sum of squared differences is known as the **total sum of squares** (denoted by $TSS$) and it represents how good the mean is as a model of the observed outcome scores:\n",
"\n",
"$$TSS = \\sum_{i=1}^n (observed_i - mean)^2$$\n",
"\n",
"We can use the values of $TSS$ and the sum of squared residuals ($SSR$) \n",
"\n",
"$$SSR = \\sum_{i=1}^n (observed_i - model_i )^2$$\n",
"\n",
"to calculate how much better the linear regression model is than the baseline model (the mean). The improvement in prediction resulting from using the linear model rather than the mean is calculated as the difference between $TSS$ and $SSR$.\n",
"\n",
"This difference shows us the reduction in the inaccuracy of the mean model resulting from fitting the regression model to the data. This improvement is the **model sum of squares** $SSM$ (also known as explained sum of squares (ESS))\n",
"\n",
"$$SSM = TSS - SSR$$\n",
"\n",
"- If the value of $SSM$ is *large*, then the linear model is very different from using the mean to predict the outcome variable. This implies that the linear model has made a big improvement to predicting the outcome variable. \n",
"\n",
"- If $SSM$ is *small* then using the linear model is little better than using the mean (i.e., the model is no better than predicting from ‘no relationship’). \n",
"\n",
"A useful measure arising from these sums of squares is the proportion of improvement due to the model. This is calculated by dividing the sum of squares for the model (SSR) by the total sum of squares (TSS) to give a quantity called $R^2$\n",
"\n",
"$$R^2 = \\frac {SS_M}{TSS}$$\n",
"\n",
"\n",
"with \n",
"\n",
"$$TSS = \\sum_{i=1}^n (observed_i - mean)^2$$\n",
"\n",
"$$SSR = \\sum_{i=1}^n (observed_i - model_i )^2$$\n",
"\n",
"$$SSM = TSS - SS_R$$\n",
"\n",
"To express the $R^2$-value as a percentage multiply it by 100. \n",
"\n",
"$R^2$ represents the amount of variance in the outcome explained by the model ($SSM$) relative to how much variation there was to explain in the first place ($TSS$): \n",
"\n",
"- it represents the proportion of the variation in the outcome that can be predicted from the model. \n",
"- Therefore, it can take any value between 0% and 100%. \n",
"\n",
"\n",
"**Adjusted** $R^2$: \n",
"\n",
"Whereas $R^2$ tells us how much of the variance in Y overlaps with predicted values from the model in our sample, adjusted $R^2$ tells us how much variance in Y would be accounted for if the model had been derived from the **population** from which the sample was taken (takes degrees of freedom into account). Therefore, the adjusted value indicates the loss of predictive power or shrinkage. \n",
"\n",
"As a general rule it is preferrable to use the adjusted $R^2$ instead of the simple $R^2$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Mean"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
name
\n",
"
id
\n",
"
height
\n",
"
height_parents
\n",
"
gender
\n",
"
average
\n",
"
pred
\n",
"
error
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
Stefanie
\n",
"
1
\n",
"
162
\n",
"
161
\n",
"
female
\n",
"
165.0
\n",
"
161.711048
\n",
"
-3.0
\n",
"
\n",
"
\n",
"
1
\n",
"
Peter
\n",
"
2
\n",
"
163
\n",
"
163
\n",
"
male
\n",
"
165.0
\n",
"
163.335222
\n",
"
-2.0
\n",
"
\n",
"
\n",
"
2
\n",
"
Stefanie
\n",
"
3
\n",
"
163
\n",
"
163
\n",
"
female
\n",
"
165.0
\n",
"
163.335222
\n",
"
-2.0
\n",
"
\n",
"
\n",
"
3
\n",
"
Manuela
\n",
"
4
\n",
"
164
\n",
"
165
\n",
"
female
\n",
"
165.0
\n",
"
164.959396
\n",
"
-1.0
\n",
"
\n",
"
\n",
"
4
\n",
"
Simon
\n",
"
5
\n",
"
164
\n",
"
163
\n",
"
male
\n",
"
165.0
\n",
"
163.335222
\n",
"
-1.0
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name id height height_parents gender average pred error\n",
"0 Stefanie 1 162 161 female 165.0 161.711048 -3.0\n",
"1 Peter 2 163 163 male 165.0 163.335222 -2.0\n",
"2 Stefanie 3 163 163 female 165.0 163.335222 -2.0\n",
"3 Manuela 4 164 165 female 165.0 164.959396 -1.0\n",
"4 Simon 5 164 163 male 165.0 163.335222 -1.0"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# calculate error (observation - average) and assign it to dataframe\n",
"df = df.assign(error = (df['height'] - df['average']))\n",
"df.head(5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Note, that we can’t simply add deviances (the individual errors) because some errors are positive and others negative and so we’d get a total of zero.\n",
"\n",
"total error = sum of errors \n",
"\n",
"total error $= \\sum_{i=1}^n (outcome_i - model_i)$ "
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.0"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# calculate the sum of the errors \n",
"df.error.sum()"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEXCAYAAABGeIg9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAArU0lEQVR4nO3dfVhUZd4H8O8AI/JqiyMxCmlvApGipJsmlYovEaCZrRvyEGZrUkYGFZCiGIWatlKKlWy7ygprsLimppsavjyJbpvporJuVog6OIiKOgwNwzBzP3/4OOt0QEQ5M4rfz3XNdTXnnPuc37mdzpfzrhBCCBAREV3BydEFEBHRzYfhQEREEgwHIiKSYDgQEZEEw4GIiCQYDkREJOHi6AKI2iswMBB9+/aFk5Pt3zbLly+Hv7+/3evRarWYNm0anJ2dMW/ePAwcONA67q9//SuampoQFxeHZcuW4fz585g7d67daxw4cCA2btx41f5xZH1082E40C0pPz8fPj4+ji4DAPDNN99ApVJh1apVknHfffcd7r//fvsXRXSDGA7UqXzzzTfIzs6Gu7s7GhoakJqaisWLF1u/r127FuvWrcPq1avh5OQElUqFOXPm4O6770Z6ejouXLiAkydPYvjw4XjzzTdt5l1UVCRpd/r0aXzwwQeor69HfHw8Vq9ebZ1+27Zt2L59O8rKytC1a1cAQGVlJeLj43HmzBmoVCosWbIEvr6+NstZtmwZTpw4gdOnT+PMmTMICQnBww8/jM8//xwajQZvvvkmoqOjYTKZsHDhQuzduxfOzs7o378/3nrrLXh6emLfvn145513oFAo0K9fP1gsFuv8t2/fjo8//hgmkwldu3ZFWlqazd4OEQBAEN1i+vbtK6Kjo8W4ceOsn5dfflkIIcQ//vEPERQUJDQaTYvf9+zZI0aNGiXOnTsnhBBi7dq1IjIyUlgsFpGWliYSEhJaXObV2q1du1a8+OKLLbZLS0sTn376qRBCiKVLl4qRI0da5/HSSy+J3NxcSZulS5eKESNGCJ1OJwwGgxg8eLBYsGCBEEKIbdu2iTFjxgghhPjwww/FK6+8IpqamoTZbBbp6elizpw5wmg0ikceeUTs2bNHCCHExo0bRd++fcXJkyfFsWPHRHR0tKirqxNCCHH06FExbNgw0dDQIJYuXSrefvvtdvxLUGfGPQe6JV3tsJJarUavXr1a/P7111/jySeftLZ9+umnkZ2dDY1GAwB46KGHWpxnW+2u1bBhw6zzCAoKQl1dXYvTPfLII/Dy8gIA+Pr64tFHHwUA3HXXXbhw4QIA4H//93+RnJwMpVIJAIiPj8eMGTNw9OhRuLi4YOjQoQCA6Oho63mEsrIy1NbWYsqUKdZlKRQKnDhxol3rQZ0fw4E6HXd391a/X3l45TIhBJqbm1tse63trpWLy3//l1MoFBCtPNqsS5curba7siaFQmHz3WQyWWtrqb3FYsHQoUPxwQcfWMdptVr4+vpi27Zt7VoX6tx4KSvdVh599FFs3rzZ+hf72rVrcccdd6B3796ytHN2dm53gFyrRx99FGvWrIHJZILFYkFhYSGGDRuGwMBACCGwa9cuAEBpaSkuXrwIABg6dCjKysrw008/AQB27dqFcePGobGxUZYa6dbFPQe6JSUkJEguZU1JSbGe+G3NsGHDMGXKFCQkJMBiscDHxwcrVqyQzKuj2j322GNYuHDhta1UO7300kt477338NRTT6G5uRn9+/fHnDlzoFQqsXz5csybNw9LlixBcHAwunfvDgC47777kJWVhZSUFAgh4OLigo8//hgeHh6y1Ei3LoVobb+WiIhuWzysREREEgwHIiKSYDgQEZEEw4GIiCRu+XBobm6GRqOR7XJBIqLb0S0fDjU1NYiIiEBNTY2jSyEi6jRu+XAgIqKOx3AgIiIJhgMREUkwHIiISILhQEREErKGg16vR3R0tPWZ9wcOHMCkSZMQFRWFlJQUNDU1AQAqKiowceJEjBs3DtOnT4dOp5OzLCIiaoNs4VBeXo7Y2FhUVVUBuBQUSUlJyMrKwqZNmwAAJSUlAIDs7Gy8+uqr2LBhA+6++2788Y9/lKssIqJOwWIRqK7V49CPZ1Bdq4fF0rHPUJXtkd3FxcXIzMxEamoqgEtvoBowYACCgoIAABkZGTCbzQAuvYCkoaEBAGAwGNCtWze5yiIiuuVZLAJ7D2mRs2Y/jCYzXJXOSI4Nw9B+ajg5KdqewTWQLRyys7Ntvh8/fhzu7u5ITk5GZWUlwsLCkJ6eDgBIT0/H1KlTMX/+fLi5uaG4uLjFeep0OskhJ978RkS3G+3ZBmswAIDRZEbOmv3oox6OXr6eHbIMu73sx2w2Y/fu3SgqKkLPnj0xe/Zs5OXlYdq0aZg9ezZWrVqF/v37Y+XKlUhLS0NeXp5kHvn5+cjNzbVXyUREN6U6ncEaDJcZTWbU1RtuvXBQqVQIDQ1FQEAAACAyMhIFBQU4evQoXF1d0b9/fwDAb3/7W3z44YctziMhIQETJkywGVZTU4O4uDh5iyciuon4eLvBVelsExCuSmf4eLl12DLsdilreHg4KioqoNVqAQA7duxASEgIevfujZqaGlRWVgK49L7bfv36tTgPb29v+Pv723z8/PzstQpERDcFtcoDybFhcFU6A4D1nINa1XGve7XbnoNarUZWVhYSExNhNBoRHByMtLQ0uLm5YcGCBXjttdcghED37t0xf/58e5VFRHTLcXJSYGg/Nfqoh6Ou3gAfLzeoVR4ddjIa6ATvkNZoNIiIiEBpaSn8/f0dXQ4RUafAO6SJiEiC4UBERBIMByIikmA4EBGRBMOBiIgkGA5ERCTBcCAiIgmGAxERSTAciIhIguFAREQSDAciIpJgOBARkQTDgYiIJBgOREQkwXAgIiIJhgMREUkwHIiISILhQEREEgwHIiKSYDgQEZEEw4GIiCQYDkREJMFwICIiCYYDERFJuMg5c71ej2effRaffPIJ/P39ceDAASxYsAANDQ0IDAzEwoUL8dNPPyE9Pd3apq6uDt26dcMXX3whZ2lERHQVsu05lJeXIzY2FlVVVQAuBUVSUhKysrKwadMmAEBJSQmCg4Oxfv16rF+/Hp999hm6deuGefPmyVUWEVGHsVgEqmv1OPTjGVTX6mGxCLu0tQfZ9hyKi4uRmZmJ1NRUAEBZWRkGDBiAoKAgAEBGRgbMZrNNmxUrVmDw4MEYNGiQXGUREXUIi0Vg7yEtctbsh9FkhqvSGcmxYRjaTw0nJ4Vsbe1FtnDIzs62+X78+HG4u7sjOTkZlZWVCAsLszmcVF9fj+LiYmzcuLHVeep0Ouh0OpthNTU1HVs4EdE10J5tsG7cAcBoMiNnzX70UQ9HL19P2drai6znHK5kNpuxe/duFBUVoWfPnpg9ezby8vKQlJQEANiwYQNGjRqF7t27tzqP/Px85Obm2qtkIqJW1ekM1o37ZUaTGXX1hjY38DfS1l7sFg4qlQqhoaEICAgAAERGRqKgoMA6/quvvsL06dOvOo+EhARMmDDBZlhNTQ3i4uI6vmAioqvw8XaDq9LZZiPvqnSGj5ebrG3txW6XsoaHh6OiogJarRYAsGPHDoSEhAAAhBCoqKjAwIEDrzoPb29v+Pv723z8/Pxkr52I6JfUKg8kx4bBVekMANbzBmqVh6xt7cVuew5qtRpZWVlITEyE0WhEcHAw0tLSAFy6fFWpVMLV1dVe5RAR3RAnJwWG9lOjj3o46uoN8PFyg1rlcU0nlG+krb0ohBA31/VT7aTRaBAREYHS0lL4+/s7uhwiok6Bd0gTEZEEw4GIiCQYDkREJMFwICIiCYYDERFJMByIiEiC4UBERBIMByIikmA4EBGRBMOBiIgkGA5ERCTBcCAiIgmGAxERSTAciIhIguFAREQSDAciIpJgOBARkQTDgYiIJBgOREQkwXAgIiIJhgMREUkwHIiISILhQEREEgwHIiKSkDUc9Ho9oqOjodFoAAAHDhzApEmTEBUVhZSUFDQ1NQEAKisrER8fj3HjxuGFF17AxYsX5SyLiIjaIFs4lJeXIzY2FlVVVQAuBUVSUhKysrKwadMmAEBJSQmEEHjppZcwbdo0bNiwAcHBwcjLy5OrLCKSicUiUF2rx6Efz6C6Vg+LRditvaPadmYucs24uLgYmZmZSE1NBQCUlZVhwIABCAoKAgBkZGTAbDajoqIC7u7ueOyxxwAAiYmJ0Ol0cpVFRDKwWAT2HtIiZ81+GE1muCqdkRwbhqH91HByUsja3lFtOzvZ9hyys7MxaNAg6/fjx4/D3d0dycnJGD9+PJYtWwZvb2+cOHECKpUKs2bNwoQJE5CZmQl3d/cW56nT6aDRaGw+NTU1cq0CEV0j7dkG6wYWAIwmM3LW7If2bIPs7R3VtrOz2wlps9mM3bt3IyUlBX/7299gMBiQl5eH5uZm/POf/0RsbCzWrVuHgIAALFy4sMV55OfnIyIiwuYTFxdnr1UgolbU6QzWDexlRpMZdfUG2ds7qm1n1+ZhpW+++QZ5eXmSk8QlJSXtWpBKpUJoaCgCAgIAAJGRkSgoKMCgQYPQu3dv9OvXDwAQHR2NV199tcV5JCQkYMKECTbDampqGBBEDubj7QZXpbPNhtZV6QwfLzfZ2zuqbWfXZjhkZGQgPj4ed9111w0tKDw8HMuWLYNWq4VarcaOHTsQEhKCgQMHoq6uDv/5z38QFBSE7du3IyQkpMV5eHt7w9vb+4bqIKKOp1Z5IDk2THLsXq3ykL29o9p2dm2GQ/fu3fHcc8/d8ILUajWysrKQmJgIo9GI4OBgpKWloWvXrli+fDkyMjJgMBjg5+eHRYsW3fDyiMh+nJwUGNpPjT7q4airN8DHyw1qlcc1n9S9kfaOatvZKYQQV71uKy8vDx4eHnj00Ufh4vLfLOnZs6fsxV0LjUaDiIgIlJaWwt/f39HlEBF1Cm3uOZw/fx5LliyBm9t/j8EpFArs379f1sKIiMhx2gyHHTt2YPfu3VCpVPaoh4iIbgJtXsravXt3+Pj42KMWIiK6SbS559C3b19MnjwZI0aMQJcuXazDn3/+eVkLIyIix2kzHBobG3H33Xdbn5FERESdX5vhsGDBAnz77bcYPHgwLly4gH379mHUqFH2qI2IiBykzXMOOTk5WLp0KYBLexF5eXn46KOPZC+MiIgcp81wKC0txZ/+9CcAgJ+fHwoKCrB582bZCyMiIsdpMxxMJhOUSqX1u1KphELBuweJiDqzNs85hIWF4fXXX8czzzwDhUKBzz//HKGhofaojYiIHKTNcJgzZw6WLl2KBQsWwMXFBUOHDsUrr7xij9qIiMhB2gwHd3d3pKen26MWIiK6SdjtZT9ERHTrkO0d0nRza9Y34NCsDNyXNANe998HADAbjaha9WfUH/keZmMj7hw9Cv5PPwUAOPanVThbthdKL08AQNeePRGU+rqjyicimTEcbkN1+77DsT+ugrG21mb48fwCNOv1CP39ezA3NuJfr72ObiEPwCuwL+r/8z0C30iGd3CQg6omIntqMxwMBgO+/PJLXLx4EVe++oHPVrrk4qHDOL66EF1UKjSeOgUnV1f4T5yAU19shqH6FLoPHYJ7fnepr+r++S1OFq+FaG6Gk2sX9Hk+Ad5BgWi6cAE/ffQJTBcuoun8Bbj69kDgm6+jyx3dsG9aInxHjsDFg4dgPHMGPUYMR++4WEkdB1NnwdJktBnmFRSEexOnSabVfrEZfVNm4vtF71uHCSFQu3MXQn//HhTOznDx8MCD774NFw9PWEwm6CuPoXrd5/jp49Nw66nG3S88D9cePTq2M4noptFmOKSmpqK6uhp9+/bl/Q2tqP/hR4QmToPnPfeg4u13oSlZhwez34bZYMC3z09DrwnjYTE24njBX/Dgu1lQenvh5xMncHju23jok+U4+/VueAUGwn/iBAghcOSdbJzZuQu9nhoHADA3NqLfgndhPHcO+xNfwZ2jRqLrnXfa1NB/0fxrrjdk3hzJMNNFHcwGAy786yB+zP0Y5oYG+EaMQM+YaDSePo07+j+Iu+Imw/2uAFSvW48j2e8hNGcxfxNEnVSb4fD9999j8+bNNm+BI1td7/SF5z33XPpvvzvh4u4OJ6USTkolnN3c0KzXQ1fxbzTVnUfF3HnWdgqFExq1NegZE42LFf9G9foNaDylRcPxk/Ds29c6nc+vBwMAXLt3h7KbN5rr9cAvwqE9ew4tEeZmwGJBY00NHnxnHkw6HQ7PngvXHj3QfcjDeGBuhnXaXhPGQ1NcAmNtrSSkiKhzaHOL7+fnZ486bmlOV9xBDgCKFoJUWCzo1r+fzUlc45mz6OLzK1Tlr0b90R9w56iR6NbvQViazcAVh/CcrnhUOhQKtPRm1/bsObRE6e0NhYsLfEcMh8LJCV3uuAO/GvQQ6r8/iq5+d6LhWBV8Rwz/7/oIAYUz/2Ag6qzavJS1b9++eO655/Dxxx9j5cqV1g+1zx39++HCv8rxs0YD4NJJ4QMzU2BpasKFA/9Cz3HR8B0xHMpu3XCxvBzCYrFrfU5KJXwGP4TaHTsBAGaDARfLD8LzvvsAhRMq//AnNJ4+DQCo+fsWePTpDVdVd7vWSET20+affg0NDejduzdOnDhhj3o6Lfe7AnDfjEQcfT/n///qdkbw7HQ4u7kh4Le/QdXKfJwoXAOFiwu8goPRqNXavcZ7Z7yEY5/+CftnzISwWNDj8UehGjYUAHDPiy/gyLsLICwWdOneHYFvJNu9PiKyH4Vo6RjFLUSj0SAiIgKlpaXw9/d3dDlERJ1Cq3sOM2fOxIcffoiYmJgWx2/cuFG2ooiIyLFaDYdp0y5d5TJnjvSyRyIi6txaDYcHH3wQAPDrX//6umeu1+vx7LPP4pNPPoG/vz8OHDiABQsWoKGhAYGBgVi4cCG6dOmC3NxcrF27Ft7e3gCASZMmIS4u7rqXS0REN0a2axHLy8uRkZGBqqoqAJeCIikpCZ9++imCgoKQkpKCkpISTJ48GYcPH8aSJUswcOBAucqh25TFIqA924A6nQE+3m5Qqzzg5HRtN+7dSFtHLtuR60ydh2zhUFxcjMzMTKSmpgIAysrKMGDAAAQFXXo2T0ZGBsxmMwDg8OHDWLFiBaqrqzF48GCkpaXB1dVVrtLoNmGxCOw9pEXOmv0wmsxwVTojOTYMQ/up29zg3UhbRy7bketMnUubVyt99dVXGDVqlM2wzz//HE899dQ1LWDkyJH485//jM2bN+PHH3+EyWRCZWUlwsLCkJ6ejubmZrz22mtIT09H7969kZ6ejl69eiE5WXqppE6ng06nsxlWU1ODuLi467pa6d9Z2Tj/3f52tSEiupn86qEwPDB3dofPt9U9h+3bt6O5uRmLFi2C5Yobspqbm7Fs2bJrDofLzGYzdu/ejaKiIvTs2ROzZ89GXl4ekpKS8Ic//ME63dSpUzFr1qwWwyE/Px+5ubntWi4REbVfq+Fw5MgR/OMf/8C5c+ewevXq/zZwccGUKVPavSCVSoXQ0FAEBAQAACIjI1FQUIBTp05hz549eOaZZwBceixDa89xSkhIwIQJE2yGXd5zuB5ypC3dPKpr9Zi5ZCeMJrN1mKvSGR+mDEcvX0/Z2jpy2Y5cZ+pcWn18xowZM7B69Wq88cYbWL16tfWzcuVKxMfHt3tB4eHhqKiogPb/7/zdsWMHQkJC0LVrVyxevBgnT56EEAKFhYUYPXp0i/Pw9vaGv7+/zYfPfqLWqFUeSI4Ng6vSGQCsx9DVKg9Z2zpy2Y5cZ+pc2jzn0NjYiK1bt6Kuru663udw+ZyDv78/du7ciZycHBiNRgQHB2P+/Plwc3PDli1bsGzZMphMJoSFheHtt99GlysfNncVvEOarsZ69U29AT5e13nlznW0deSyHbnO1Hm0GQ4zZsyAVquVvM9hwYIFshd3LRgOREQdr81LWY8ePYotW7bAyanNB7gSEVEn0eYWv3v37mhubrZHLUREdJNodc/h8jsbevTogfj4eEREREB5xUtt+A5pIqLOq9VwOHr0KADA09MTnp6eOHbsmN2KIiIix2o1HG6WE85ERGR/bZ6QHjlypM1VSgqFAm5ubrj//vuRnp4OX19fWQskIiL7azMcRo0ahYaGBsTFxcHJyQklJSXWR27PnTsXn3zyiT3qJCIiO2rzaqV9+/YhOzsbDzzwAIKCgpCRkYEffvgBU6ZMQXV1tT1qJCIiO2szHBoaGqDX663f9Xo9GhsbZS2KiIgcq83DShMnTsSkSZPwxBNPQAiBrVu34je/+Q1Wr16Ne+65xx41EhGRnbUZDi+++CIeeOAB7Nq1Cy4uLpgzZw6GDBmCw4cPS56QSkREnUOr4fDTTz/h3nvvRUVFBX71q1/ZvL+hoqLC+o5pIiLqfFoNh0WLFmHFihVISkqSjFMoFCgtLZW1MCIicpxWw2HFihUALr0RjoiIbi/XdLVSVlYWEhIScOHCBcydOxcNDQ32qI2IiBykzXB499134eXlhXPnzsHV1RV6vR5z5861R21EROQgbYbDkSNHkJycDBcXF7i5ueH999/HkSNH7FEbERE5SJvh8MuX/JjNZr74h4iok2vzPofBgwdj8eLFaGxsxNdff43CwkI8/PDD9qiNiIgcpM1dgDfeeAPu7u7w8vJCTk4OAgMDkZqaao/aiIjIQRRCCOHoIm6ERqNBREQESktL4e/v7+hyiIg6hVYPKyUmJl61IR/VTUTUebUaDmPHjrX+99KlS/Hqq6/apSAiInK8VsPhyofq5efn8yF7RES3kWu6JvXK14S2h16vR3R0NDQaDQDgwIEDmDRpEqKiopCSkoKmpiab6Xfu3ImRI0de17KIiKjjyHbDQnl5OWJjY1FVVQXgUlAkJSUhKysLmzZtAgCUlJRYpz979izee+89ucohABaLQHWtHod+PIPqWj0slmu/FuFG2jp62UTUfq0eVrpw4YL1v81mMy5evIgrL2y64447rjrj4uJiZGZmWi97LSsrw4ABAxAUFAQAyMjIgNlstk6fkZGBV155Bb///e+vZz2oDRaLwN5DWuSs2Q+jyQxXpTOSY8MwtJ8aTk5X3zO8kbaOXjYRXZ9Ww2HIkCFQKBTWQLjyxjeFQtHmIzSys7Ntvh8/fhzu7u5ITk5GZWUlwsLCkJ6eDgD485//jAceeAChoaFXnadOp4NOp7MZVlNTc9U2dIn2bIN1AwsARpMZOWv2o496OHr5esrW1tHLJqLr02o4/Oc//+nQBZnNZuzevRtFRUXo2bMnZs+ejby8PIwdOxZbt27FqlWr2tzQ5+fnIzc3t0Prul3U6QzWDexlRpMZdfWGNjeyN9LW0csmouvT5uMzOopKpUJoaCgCAgIAAJGRkSgoKIAQAmfOnMHEiRNhMplQW1uLyZMn4y9/+YtkHgkJCZKrpmpqahAXF2eXdbiV+Xi7wVXpbLOhdVU6w8fLTda2jl42EV0fuz1BLzw8HBUVFdBqtQCAHTt2ICQkBK+++iq2bNmC9evXIy8vD76+vi0GAwB4e3vD39/f5uPn52evVbilqVUeSI4Ng6vSGQCsx+7VKg9Z2zp62UR0fey256BWq5GVlYXExEQYjUYEBwcjLS3NXou/7Tk5KTC0nxp91MNRV2+Aj5cb1CqPazqpeyNtHb1sIro+fLYSERFJ8MUMREQkwXAgIiIJhgMREUkwHIiISILhQEREEgwHIiKSYDgQEZEEw4GIiCQYDkREJMFwICIiCYYDERFJMByIiEiC4UBERBIMByIikmA4EBGRBMOBiIgkGA5ERCTBcCAiIgmGAxERSTAciIhIguFAREQSDAciIpJgOBARkQTDgYiIJGQNB71ej+joaGg0GgDAgQMHMGnSJERFRSElJQVNTU0AgG3btiEmJgZRUVFIT0+3Du+sLBaB6lo9Dv14BtW1elgswi5tiYiulWzhUF5ejtjYWFRVVQG4FBRJSUnIysrCpk2bAAAlJSX4+eefkZWVhZUrV2LTpk0wGo1Yt26dXGU5nMUisPeQFjOX7MSsj/dg5pKd2HtIe00b+RtpS0TUHrKFQ3FxMTIzM+Hr6wsAKCsrw4ABAxAUFAQAyMjIwOjRo+Hu7o7t27dDpVLBYDDg3Llz8Pb2bnGeOp0OGo3G5lNTUyPXKshCe7YBOWv2w2gyAwCMJjNy1uyH9myDrG2JiNrDRa4ZZ2dn23w/fvw43N3dkZycjMrKSoSFhSE9PR0AoFQqsWvXLqSmpsLX1xfh4eEtzjM/Px+5ublylWwXdTqDdeN+mdFkRl29Ab18PWVrS0TUHnY7IW02m7F7926kpKTgb3/7GwwGA/Ly8qzjH3/8cXzzzTcYMWIE5s2b1+I8EhISUFpaavMpLCy00xp0DB9vN7gqnW2GuSqd4ePlJmtbIqL2sFs4qFQqhIaGIiAgAM7OzoiMjMTBgwdx4cIF7N692zpdTEwMvv/++xbn4e3tDX9/f5uPn5+fvVahQ6hVHkiODbNu5F2VzkiODYNa5SFrWyKi9pDtsNIvhYeHY9myZdBqtVCr1dixYwdCQkIghMCbb76JtWvXomfPnvjyyy8RFhZmr7LszslJgaH91OijHo66egN8vNygVnnAyUkha1siovawWzio1WpkZWUhMTERRqMRwcHBSEtLg5ubG9555x1Mnz4dCoUC9913H95++217leUQTk4K9PL1vK7zBDfSlojoWimEELf0dZAajQYREREoLS2Fv7+/o8shIuoUeIc0ERFJMByIiEiC4UBERBIMByIikmA4EBGRBMOBiIgkGA5ERCTBcCAiIgmGAxERSTAciIhIguFAREQSDAciIpJgOBARkQTDgYiIJBgOREQkwXAgIiIJhgMREUkwHIiISILhQEREEgwHIiKSYDgQEZEEw4GIiCQYDkREJMFwICIiCVnDQa/XIzo6GhqNBgBw4MABTJo0CVFRUUhJSUFTUxMA4KuvvsL48eMxbtw4vPzyy7h48aKcZXUIi0WgulaPQz+eQXWtHhaLcHRJREQdRrZwKC8vR2xsLKqqqgBcCoqkpCRkZWVh06ZNAICSkhLo9XrMmzcPeXl52LBhAwIDA7Fs2TK5yuoQFovA3kNazFyyE7M+3oOZS3Zi7yEtA4KIOg3ZwqG4uBiZmZnw9fUFAJSVlWHAgAEICgoCAGRkZGD06NEwmUzIzMzEnXfeCQAIDAyEVqttcZ46nQ4ajcbmU1NTI9cqtEp7tgE5a/bDaDIDAIwmM3LW7If2bIPdayEikoOLXDPOzs62+X78+HG4u7sjOTkZlZWVCAsLQ3p6OlxdXTF69GgAQGNjI/Ly8hAfH9/iPPPz85GbmytXydesTmewBsNlRpMZdfUG9PL1dFBVREQdR7Zw+CWz2Yzdu3ejqKgIPXv2xOzZs5GXl4ekpCQAQH19PWbMmIGgoCBMmDChxXkkJCRIxtXU1CAuLk72+q/k4+0GV6WzTUC4Kp3h4+Vm1zqIiORit6uVVCoVQkNDERAQAGdnZ0RGRuLgwYMAgNraWkyePBmBgYGSPY4reXt7w9/f3+bj5+dnr1WwUqs8kBwbBlelM4BLwZAcGwa1ysPutRARycFuew7h4eFYtmwZtFot1Go1duzYgZCQEJjNZiQmJiIyMhIvv/yyvcq5IU5OCgztp0Yf9XDU1Rvg4+UGtcoDTk4KR5dGRNQh7BYOarUaWVlZSExMhNFoRHBwMNLS0rB9+3b8+9//htlsxpYtWwAADz744FX3IG4GTk4K9PL15DkGIuqUFEKIW/r6S41Gg4iICJSWlsLf39/R5RARdQq8Q5qIiCQYDkREJMFwICIiCbudkJaL2XzpXgNH3ClNRHSr8/Pzg4uLNApu+XA4c+YMANj9Rjgios6gtYt5bvmrlRobG3H48GH06NEDzs7O7Wp7+e7qwsJCh9xM1xrW1T43a13AzVsb62qfzlxXp91z6Nq1KwYNGnRD8/Dz87spL4NlXe1zs9YF3Ly1sa72uZ3q4glpIiKSYDgQEZEEw4GIiCRu63Dw9vbGK6+8Am9vb0eXYoN1tc/NWhdw89bGutrndqzrlr9aiYiIOt5tvedAREQtYzgQEZHEbRMOGzduxJNPPokxY8agsLBQMv7IkSN4+umnMXbsWMyePRvNzc12qSs3NxdRUVGIiorCokWLWhw/YsQIjB8/HuPHj2+xdjnEx8cjKirKutzy8nKb8Y7or7/+9a/WesaPH4+HHnoIWVlZNtPYu7/0ej2io6Oh0WgAAHv27EFMTAzGjBmDnJycFtucOnUKcXFxeOKJJ/DSSy+hoaFB9rqKiooQHR2NmJgYvPXWW2hqapK0WbduHcLDw61911r9HVnXW2+9hTFjxliXuW3bNkkbe/fXrl27bH5nQ4YMwfTp0yVt5O6vlrYNdv19idtATU2NGDFihDh//rxoaGgQMTEx4ocffrCZJioqShw4cEAIIcRbb70lCgsLZa+rrKxM/Pa3vxVGo1E0NTWJ5557TmzdutVmmunTp4v9+/fLXsuVLBaLCA8PFyaTqdVpHNFfVzp69KgYPXq0OHfunM1we/bXv/71LxEdHS1CQkLEyZMnhcFgEI8//rg4ceKEMJlMYurUqWLnzp2Sdi+++KL44osvhBBC5ObmikWLFslaV2VlpRg9erSor68XFotFpKamipUrV0raZWVliY0bN3ZoLVerSwghoqOjxenTp6/azt79daXa2loREREhjh07JmknZ3+1tG3YuHGjXX9ft8Wew549ezBkyBDccccdcHd3x9ixY/Hll19ax1dXV6OxsREDBgwAADz99NM24+XSo0cPpKeno0uXLlAqlbj33ntx6tQpm2kOHz6MFStWICYmBllZWTAajbLXVVlZCQCYOnUqxo0bh4KCApvxjuqvK82bNw/Jycnw8fGxGW7P/iouLkZmZiZ8fX0BAAcPHkTv3r0REBAAFxcXxMTESPrFZDLh22+/xdixYwHI03e/rKtLly7IzMyEp6cnFAoF+vbtK/mdAcChQ4ewbt06xMTE4I033sDFixdlrctgMODUqVOYNWsWYmJisHTpUlgsFps2juivKy1atAjPPvss+vTpIxknZ3+1tG2oqqqy6+/rtgiH2tpa9OjRw/rd19cXp0+fbnV8jx49bMbL5f7777duYKuqqvD3v/8djz/+uHV8Q0MDgoOD8eabb2LdunXQ6XT46KOPZK9Lp9Nh6NChWL58OVatWoXPPvsMZWVl1vGO6q/L9uzZg8bGRkRGRtoMt3d/ZWdn2zy6pa3fGQCcP38enp6e1mfZyNF3v6yrV69eGDZsGACgrq4OhYWFiIiIkLTr0aMHXn75ZWzYsMH6Wl856zp79iyGDBmC+fPno7i4GPv27UNJSYlNG0f012VVVVX45z//ieeee67FdnL2V0vbBoVCYdff120RDhaLBQqFwvpdCGHzva3xcvvhhx8wdepUpKam2vyF4uHhgT/84Q+499574eLigqlTp2LXrl2y1zNw4EAsWrQIXl5e8PHxwTPPPGOzXEf312effYbnn39eMtxR/XXZtfRLS8Ps1XenT59GQkICJk6ciIcfflgyfvny5XjooYegUCjwu9/9Dl9//bWs9QQEBGD58uXw9fWFm5sb4uPjJf9ejuyvoqIiTJ48GV26dGlxvD3668ptQ0BAgF1/X7dFOPj5+Vkf7Q1cesz3lbuQvxx/9uzZFncx5fDdd99hypQpeP311zFhwgSbcadOnbL5S0oI0eLTEzvavn37sHfv3laX68j+ampqwrfffouRI0dKxjmqvy5r63cGAD4+Pqivr7e+h6SlaeTw008/4dlnn8WECRMwY8YMyfj6+nqsWrXK+l0I0e6nHLfX999/jy1bttgs85f/Xo7qL+DSo6yffPLJFsfZo79+uW2w9+/rtgiHRx55BHv37kVdXR0MBgO2bt2Kxx57zDq+V69ecHV1xXfffQcAWL9+vc14uWi1WsyYMQPvv/8+oqKiJOO7du2KxYsX4+TJkxBCoLCwEKNHj5a9rvr6eixatAhGoxF6vR7r1q2zWa6j+gu4tEHp06cP3N3dJeMc1V+XhYaG4tixYzh+/DjMZjO++OILSb8olUoMGjQImzdvBgB8/vnnsvedXq/HCy+8gJkzZ2Lq1KktTuPu7o5PP/3UelVaQUGB7H0nhMD8+fNx8eJFmEwmFBUVSZbpiP4CLh1+a2xsREBAQIvj5e6vlrYNdv99Xddp7FvQhg0bRFRUlBgzZozIy8sTQgjxu9/9Thw8eFAIIcSRI0fExIkTxdixY0VKSoowGo2y1/TOO++IAQMGiHHjxlk/f/nLX2zq+vLLL611p6en26UuIYTIyckRTzzxhBgzZoxYtWqVEMLx/SWEEJs2bRKvvfaazTBH99eIESOsV7ns2bNHxMTEiDFjxojs7GxhsViEEELMmjVLfPXVV0IIITQajfif//kfERkZKaZOnSouXLgga10rV64UISEhNr+zDz74QFLXt99+K5566inxxBNPiMTERKHT6WStSwghCgoKRGRkpBg9erRYvHixdRpH9pcQQpSXl4vf/OY3kmns1V+tbRvs+fvi4zOIiEjitjisRERE7cNwICIiCYYDERFJMByIiEiC4UBERBL2u0OI6DYwcuRIfPjhh5g5cyaUSiW6du0KIQSEEHjyyScxbdo0u96YR3S9uOdAJJP3338f69evx4YNG1BUVIRDhw5hwYIFji6L6JowHIjswN3dHXPnzkVRURH0er2jyyFqE8OByE78/Pzg6elpfSQ60c2M4UBkRwqFAm5ubo4ug6hNDAciO6mursbPP/+Mu+66y9GlELWJ4UBkBzqdDu+88w7i4uLg6urq6HKI2sRr6ohk8sYbb6Br165wdnaG2WzGmDFjkJiY6OiyiK4Jn8pKREQSPKxEREQSDAciIpJgOBARkQTDgYiIJBgOREQkwXAgIiIJhgMREUkwHIiISOL/AGZynM3h2EPsAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"sns.residplot(x=\"height_parents\", y=\"height\", data=df, scatter_kws={\"s\": 80});"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"34.919735599622285\n"
]
},
{
"data": {
"text/plain": [
"34.919735599622285"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Explained sum of squares (SS_M = TSS - SS_R)\n",
"SSM = TSS - SSR\n",
"print(SSM)\n",
"# Explained sum of squres (SS_M) from statsmodel function\n",
"lm.ess"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"$R^2$ is the proportion of the variance in the dependent variable that is predictable from the independent variable"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.831422276181483\n"
]
}
],
"source": [
"# R_Squared: explained sum of squared residuals\n",
"R_squared = SSM / TSS\n",
"print(R_squared)"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.831422276181483"
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# R_Squared of statsmodel\n",
"lm.rsquared"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.8220568470804543"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Adjusted R_Squared: \n",
"lm.rsquared_adj"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Pearson's correlation coefficient"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"We just saw that $R^2$ represents the proportion of the variation in the outcome that can be predicted from the model: \n",
"\n",
"$R^2 = \\frac{SSM}{TSS}$\n",
"\n",
"with \n",
"\n",
"$TSS = \\sum_{i=1}^n (observed_i - mean)^2$\n",
"\n",
"$SSR = \\sum_{i=1}^n (observed_i - model_i )^2$\n",
"\n",
"$SSM = TSS - SSR$\n",
"\n",
"We can take the square root of this value to obtain **Pearson’s correlation coefficient** (we cover the topic of correlation in a separate application in detail) for the relationship between the values of the outcome predicted by the model and the observed values of the outcome. With only one predictor in the model this value will be the same as the Pearson correlation coefficient between the predictor and outcome variable.\n",
"\n",
"$r = \\sqrt{R^2} = \\sqrt{\\frac{SSM}{TSS}}$\n",
"\n",
"\n",
"As such, the correlation coefficient provides us with a good estimate of the overall fit of the regression model (i.e., the correspondence between predicted values of the outcome and the actual values), and $R^2$ provides us with a gauge of the substantive size of the model fit.\n",
"\n",
"**Interpretation of r:**\n",
"\n",
"$- 1 ≤ r ≤ 1$\n",
"\n",
"With a perfect relationship (r = −1 or 1) the observed data basically fall exactly on the line (the model is a perfect fit to the data), but for a weaker relationship (r = −0.5 or 0.5) the observed data are scattered more widely around the line.\n",
"\n",
" * | r | = 0.10 (small effect)\n",
" * | r | = 0.30 (medium effect)\n",
" * | r | = 0.50 (large effect)\n",
"\n",
"It’s worth bearing in mind that r is not measured on a linear scale, so an effect with r = 0.6 isn’t twice as big as one with r = 0.3."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Let`s caculate the $R^2$ for our regression model:"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.911823599267689\n"
]
}
],
"source": [
"# correlation coefficient r\n",
"r = np.sqrt(R_squared)\n",
"print(r)"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(0.9118235992676889, 2.2144128916549524e-08)"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# correlation coefficient with p-value\n",
"stats.pearsonr(df['height'], df['height_parents'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Mean squared error\n",
"\n",
"Although the **sum of squared errors (SS)** is a good measure of the accuracy of our model, it depends upon the quantity of data that has been collected – the more data points, the higher the SS. \n",
"\n",
"By using the **average error**, rather than the total, we can overcome this problem. This measure is called the \"**Mean Squared Error (MSE)**\". In the regression setting, it is the most commonly-used measure to evaluate the performance of a model. \n",
"\n",
"To compute the average error we divide the sum of squares by the number of values that we used to compute that total. We again come back to the problem that we’re usually interested in the error in the model in the **population** (not the **sample**). \n",
"\n",
"To estimate the mean error in the population we need to divide not by the number of scores contributing to the total, but by the **degrees of freedom (df)**, which is the number of scores used to compute the total adjusted for the fact that we’re trying to estimate the population value. Note: you may encounter different formulas for calculating the mse. Some only devide by the number of observations and not the df.\n",
"\n",
"Mean squared error of our regression model:\n",
"\n",
"$$ MSE = \\frac {SS}{df} = \\frac {\\sum_{i=1}^n (outcome_i - model_i)^2}{df}$$\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Variance"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"In the case of our mean model, we use the **variance** as a measure of dispersion. Furthermore, the **standard deviation** is the square root of the variance:\n",
"\n",
"Variance = $s^2 = \\frac {SS}{df} = \\frac {\\sum_{i=1}^n (outcome_i - model_i)^2}{df}$\n",
"\n",
"Standard Deviation = $\\sqrt{s^2} = \\sqrt{\\frac{\\sum\\limits_{i=1}^{n} \\left(x_{i} - \\bar{x}\\right)^{2}} {df}}$\n",
"\n",
"which equals:\n",
"\n",
"$s = \\frac {\\sum_{i=1}^n (outcome_i - model_i)}{df}$\n",
"\n",
"A small **standard deviation** represents a scenario in which most data points are close to the mean, whereas a large standard deviation represents a situation in which data points are widely spread from the mean. "
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"variance of the mean: 2.210526315789474\n"
]
}
],
"source": [
"# calculate variance of the model mean\n",
"# Number of obeservations (lenght of DataFrame)\n",
"n = len(df[\"height\"])\n",
"# calculate \n",
"variance = (TSS/(n-1))\n",
"# print output\n",
"print('variance of the mean:', variance)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Variance = $\\frac {SS}{df} = \\frac {\\sum_{i=1}^n (outcome_i - model_i)^2}{df} = \\frac {\\sum_{i=1}^n(x_i-\\bar{x})^2}{df} = \\frac {42}{19} = 2.210$\n",
"\n",
"In our example, $n=20$, we have one parameter, p=1 (the mean), and therefore, the degrees of freedom are df = (p-1) = 20-1 = 19."
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Standard deviation (SD) of model 1 = 1.49\n"
]
}
],
"source": [
"# obtain the standard deviation\n",
"print(f'Standard deviation (SD) of model 1 = {round(np.sqrt(variance),2)}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Linear Regression:"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Statsmodel provide us with different options (we use mse_resid and mse_total): \n",
"\n",
" * mse_model : Mean squared error the model. This is the explained sum of squares divided by the model degrees of freedom.\n",
" * mse_resid : Mean squared error of the residuals. The sum of squared residuals divided by the residual degrees of freedom.\n",
" * mse_total : Total mean squared error. Defined as the uncentered total sum of squares divided by n the number of observations."
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total mean squared error (MSE_T): 2.210526315789474\n"
]
}
],
"source": [
"# Total MSE_T (this is the MSE of the basline mean model) from statsmodel\n",
"MSE_T = lm.mse_total\n",
"# print output\n",
"print('Total mean squared error (MSE_T):', MSE_T)\n",
"# compare this result to mse... they are the same"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mean squared error of residuals (MSE_R): 0.3933480222432064\n",
"Mean squared error od residuals (MSE_R): 0.39334802224320636\n"
]
}
],
"source": [
"# Mean squared error of residuals (MSE_R)\n",
"MSE_R = SSR / (20-2)\n",
"print('Mean squared error of residuals (MSE_R):', MSE_R)\n",
"# MSE of residuals from statsmodel (preferred)\n",
"print(f'Mean squared error od residuals (MSE_R): {lm.mse_resid}')"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Standard deviation (SD) of model 2 = 0.63\n"
]
}
],
"source": [
"# the standard deviation equals the root of the MSE_R\n",
"print(f'Standard deviation (SD) of model 2 = {round(np.sqrt(MSE_R),2)}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### F-Statistic"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"A second use of the sums of squares in assessing the model is the **F-test**. Test statistics (like F) are usually the amount of **systematic variance** divided by the amount of **unsystematic variance**, or, put another way, the model compared to the error in the model. This is true here: F is based upon the ratio of the improvement due to the model ($SS_M$) and the error in the model ($SS_R$). \n",
"\n",
"Because the sums of squares depend on the number of differences that were added up, the **average sums of squares** (referred to as the **mean squares** or **MS**) are used to compute F. \n",
"\n",
"The **mean sum of squares** is the sum of squares divided by the associated degrees of freedom (this is comparable to calculating the variance from the sums of squares). \n",
"\n",
"For $SS_M$ the degrees of freedom are the number of predictors in the model (*p*), and for $SS_R$ they are the number of observations (*n*) minus the number of parameters being estimated (i.e., the number of b coefficients including the constant). \n",
"\n",
"We estimate a \"b\" for each predictor and the intercept ($b_0$), so the total number of b's estimated will be *p + 1*, giving us degrees of freedom of n - (p + 1) or, more simply, n - p - 1. Thus:\n",
"\n",
"$MS_M = \\frac{SS_M}{p}$\n",
"\n",
"$MS_R = \\frac{SS_R}{n-p-1}$\n",
"\n",
"**The F-statistic computed from these mean squares**\n",
"\n",
"$$F = \\frac{Systematic Variance}{Unsystematic Variance} = \\frac{MS_M}{MS_R}$$\n",
"\n",
"is a measure of how much the model has *improved* the prediction of the outcome compared to the level of inaccuracy of the model. If a model is good, then the improvement in prediction from using the model should be large ($MS_M$ will be large) and the difference between the model and the observed data should be small ($MS_R$ will be small). \n",
"\n",
"In short, for a good model the numerator in the equation above will be bigger than the denominator, resulting in a large F-statistic (greater than 1 at least).\n",
"\n",
"This F has an associated probability distribution from which a **p-value** can be derived to tell us the probability of getting an F at least as big as the one we have if the null hypothesis were true. The null hypothesis in this case is a flat model (predicted values of the outcome are the same regardless of the value of the predictors). \n",
"\n",
"The F-statistic is also used to calculate the significance of $R^2$ using the following equation:\n",
"\n",
"$F = \\frac{(n-p-1)R^2}{p(1-R^2)}$\n",
"\n",
"in which n is the number of cases or participants, and p is the number of predictors in the model. This F tests the null hypothesis that $R^2$ is zero (i.e., there is no improvement in the sum of squared error due to fitting the model)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Linear Regression:"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSM = 34.919735599622285\n",
"MS_M = 34.919735599622285\n"
]
}
],
"source": [
"# Mean squared error of the model (MSE_M)\n",
"p = 1 # we only have one predictor (height_parents)\n",
"MSM = (SSM / p)\n",
"print('MSM =', MSM)\n",
"# MSE_M of residuals from statsmodel\n",
"print(f'MS_M = {lm.mse_model}')"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"88.77567351293678\n"
]
}
],
"source": [
"# Adjust notation and calculate F-value\n",
"MSR = MSE_R\n",
"# F-value\n",
"F_value = (MSM / MSR)\n",
"print(F_value)"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"88.77567351293679\n",
"88.77567351293679\n"
]
}
],
"source": [
"# statsmodel\n",
"# Alternative way to obtain F-value (preferred)\n",
"print(lm.fvalue)\n",
"# which of course equals\n",
"F_val = (lm.mse_model / lm.mse_resid)\n",
"print(F_val)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Standard error"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"We just learned that the **standard deviation** tells us about how well the mean represents the sample data. However, if we’re using the **sample mean** to estimate this parameter in the **population** (like we did), then we need to know how well it represents the value in the population, especially because samples from a population differ. \n",
"\n",
"Imagine that we were interested in the height of *all* adult women in germany (so adult women in germany are the **population**). We could take a **sample** from this population (like we did with our 20 women), and when we do we are taking one of many possible samples. If we were to take several samples from the same population, then each sample would have its own mean, and some of these sample means will be different. \n",
"\n",
"Imagine we could obtain the height of all adult women in germany and could compute the mean of their height (which would equal $\\mu$ = 168 cm). Then we would know, as an absolute fact, that the mean of the height is 168 cm (this is the population mean, **µ**, the parameter that we’re trying to estimate).\n",
"\n",
"In reality, we don’t have access to the population, so we use a sample. In this sample we calculate the average height, known as the **sample mean** ($\\bar{x}$), and discover it is 168 cm; that is, adult women in our sample are 168 cm tall, on average. Now we take a second sample and find that women in this sample are, on average, only 167 cm tall. In other words, the sample mean is different in the second sample than in the first. \n",
"\n",
"This difference illustrates **sampling variation**: that is, samples vary because they contain different members of the population; a sample that, by chance, includes relatively tall women will have a higher average than a sample that, by chance, includes some women who are relatively short.\n",
"\n",
"Imagine that we now take a lot of samples (let's say 1000). If we plotted the resulting sample means as a histogram, we would observe a symmetrical distribution known as a **sampling distribution**. \n",
"\n",
"A **sampling distribution** is the frequency distribution of sample means (or whatever parameter you’re trying to estimate) from the same population. You need to imagine that we’re taking hundreds or thousands of samples to construct a sampling distribution. \n",
"\n",
"The sampling distribution of the mean tells us about the behaviour of samples from the population, and it is centred at around the same value as the mean of the population. Therefore, if we took the average of all sample means we’d get the value of the **population mean**. \n",
"\n",
"We can use the sampling distribution to tell us how *representative a sample* is of the population. Think back to the **standard deviation**. We used the standard deviation as a measure of how representative the mean was of the observed data. \n",
"\n",
"A small standard deviation represented a scenario in which most data points were close to the mean, whereas a large standard deviation represented a situation in which data points were widely spread from the mean. If our ‘observed data’ are sample means then the standard deviation of these sample means would similarly tell us how widely spread (i.e., how representative) sample means are around their average. Bearing in mind that the *average of the sample means* is the same as the *population mean*, the standard deviation of the sample means would therefore tell us how widely sample means are spread around the population mean: put another way, it tells us whether sample means are typically representative of the population mean.\n",
"\n",
"The standard deviation of sample means is known as the **standard error of the mean (SE)** or standard error for short. Theoretically, the standard error could be calculated by taking the difference between each sample mean and the overall mean, squaring these differences, adding them up, and then dividing by the number of samples. Finally, the square root of this value would need to be taken to get the standard deviation of sample means: the standard error. In the real world, it would be to costly to collect thousands of samples, and so we compute the standard error from a mathematical approximation. \n",
"\n",
"Statisticians have demonstrated something called the **central limit theorem**, which tells us that as samples get large (usually defined as greater than 30), the sampling distribution has a normal distribution with a mean equal to the population mean, and we can calculate the standard deviation as follows: \n",
"\n",
"$$\\sigma_{\\bar{X}} = \\frac{s} {\\sqrt{N}}$$\n",
"\n",
"Hence, if our sample is largeer than n=30 we can use the equation above to approximate the standard error (because it is the standard deviation of the sampling distribution).\n",
"Note: when the sample is relatively small (fewer than 30) the sampling distribution is not normal: it has a different shape, known as a **t-distribution**, which we’ll cover later.\n",
"\n",
"**Summary**\n",
"\n",
" * The standard error of the mean is the standard deviation of sample means. \n",
" * As such, it is a measure of how representative of the population a sample mean is likely to be. \n",
" * A large standard error (relative to the sample mean) means that there is a lot of variability between the means of different samples and so the sample mean we have might not be representative of the population mean. \n",
" * A small standard error indicates that most sample means are similar to the population mean (i.e., our sample mean is likely to accurately reflect the population mean)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Mean:"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.33245498310218435\n"
]
}
],
"source": [
"# calculate standard error (...we ignore the fact that our sample is small since n < 30) \n",
"se = df['height'].sem()\n",
"print(se)\n",
"# assign se to df\n",
"df = df.assign(se=se)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.33245498310218435\n"
]
}
],
"source": [
"# alternative way to calculate standard error (se)\n",
"# calculate standard deviation (s)\n",
"s = df[\"height\"].std()\n",
"# calculate se\n",
"se = (s/np.sqrt(n))\n",
"print(se)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Linear Regression:"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Standard error (SE) od model 2: Intercept 14.226306\n",
"height_parents 0.086190\n",
"dtype: float64\n"
]
}
],
"source": [
"# Get standard error of parameters\n",
"se_2 = lm.bse\n",
"print('Standard error (SE) od model 2:', se_2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Significance testing"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Next, we perform a statistical hypothesis test to asssess the slope of the model: \n",
"\n",
"- $H_0$: $\\beta_1 = 0$. The true linear model has slope zero.\n",
"- $H_A$: $\\beta_1 \\neq 0$. The true linear model has a slope different than zero. The height of the parents is predictive for the height of their daughter.\n",
"\n",
"We would reject $H_0$ in favor of $H_A$ if the data provide strong evidence that the true slope parameter ($\\beta_{expected}$) is different than zero. To assess the hypothesis, we use the standard error for the estimate ($SE_\\beta$), compute a test statistic, and identify the p-value.\n",
"\n",
"The **p-value** is the probability of seeing an coefficient at least as extreme as our result (0.8121), had there truly been no difference between our cofficient and the expected coefficient (with a vlaue of 0.0). In other words: the p-value is the probability of obtaining a coefficient at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct. Therefore, the lower p the better since a small p-value means that such an extreme observed outcome would be very unlikely under the null hypothesis. \n",
"\n",
"In particular, the null hypothesis is rejected if the p-value is less than a pre-defined threshold value $\\alpha$, which is referred to as the alpha level or significance level. By convention, we use an \\alpha level of 0.05. This means we reject $H_0$ if the p-value is equal or below 5%.\n",
"\n",
"To obtain the p-value for our coefficient, we use the parameter estimate and the respective standard error to compute a *t statistic* to tell us the likelihood of the observed parameter estimates compared to some expected value under the null hypothesis:\n",
"\n",
"$$\n",
"\\begin{array}{c}\n",
"t_{N - p} = \\frac{{\\beta_1} - \\beta_{expected}}{SE_{\\hat{\\beta}}}\\\\\n",
"t_{N - p} = \\frac{{\\beta_1} - 0}{SE_{\\hat{\\beta}}}\\\\\n",
"t_{N - p} = \\frac{{\\beta_1} }{SE_{\\hat{\\beta}}}\n",
"\\end{array}\n",
"$$\n",
"\n",
"*Review [this notebook](https://kirenz.github.io/inference/t_test.html) to learn more about the t-statistic.*"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"
\n",
"
coef
std err
t
P>|t|
[0.025
0.975]
\n",
"
\n",
"
\n",
"
Intercept
30.9651
14.226
2.177
0.043
1.077
60.853
\n",
"
\n",
"
\n",
"
height_parents
0.8121
0.086
9.422
0.000
0.631
0.993
\n",
"
\n",
"
"
],
"text/plain": [
""
]
},
"execution_count": 55,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"lm.summary().tables[1]"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"t-statistic: 9.443023255813955\n"
]
}
],
"source": [
"model_result = pd.read_html(lm.summary().tables[1].as_html(),header=0,index_col=0)[0]\n",
"\n",
"t_statistic = model_result['coef'].values[1] / model_result['std err'].values[1]\n",
"\n",
"print(\"t-statistic:\", t_statistic)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"We see that the intercept is significantly different from zero (which is not of importance) and that the effect of `height_parents` on `height` is significant (p = .000). "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"To obtain this result, we first computed the t statistic by deviding the coefficient with the standard error and found that t = 9.422. The question that we want to ask next is: What is the likelihood that we would find a t statistic of this size, if the true coefficent value is zero (i.e. the null hypothesis)? \n",
"\n",
"We can use the t distribution to determine this probability. The result P>|t| = 0.000 is the probability of finding a value of the t statistic greater than or equal to our observed value under the assumption of $H_0$. We find that p(t > 9.422) = 0.000, which tells us that our observed t statistic value of 9.422 is relatively unlikely if the null hypothesis really is true.\n",
"\n",
"Since the p-value is below our significance level of 5%, we reject $H_0$.\n",
"\n",
"To learn more about the concept of p-values, review this excellent post series [\"Decision Making at Netflix\"](https://netflixtechblog.com/decision-making-at-netflix-33065fa06481) provided by Netflix Tech Blog."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Confidence interval"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Now let's cover confidence intervalls. As a brief recap, we usually use a **sample value** as an estimate of a **parameter** (e.g., the mean or any other parameter b) in the **population**. We’ve just seen that the **estimate of a parameter** (e.g., the mean) will differ across samples, and we can use the **standard error** to get some idea of the extent to which these estimates differ across samples. We can also use this information to calculate boundaries within which we believe the population value will fall. Such boundaries are called **confidence intervals**. \n",
"\n",
"For example, perhaps we might want to know how often, *in the long run*, an interval contains the true value of the parameter we’re trying to estimate (in the case of model 1, the mean). This is what a **confidence interval** does. \n",
"\n",
"Typically, we look at 95% confidence intervals, and sometimes 99% confidence intervals, but they all have a similar interpretation: they are limits constructed such that, for a certain percentage of samples (be that 95% or 99%), the true value of the population parameter falls within the limits. So, when you see a 95% confidence interval for a mean, think of it like this: \n",
"\n",
" * if we’d collected 100 samples, and for each sample calculated the parameter (e.g. the mean) and a confidence interval for it, then for 95 of these samples, the confidence interval contains the value of the parameter (e.g. the mean) in the population, and in 5 of the samples the confidence interval does not contain the population paramater (e.g. the mean). \n",
" \n",
"The trouble is, you do not know whether the confidence interval from a particular sample is one of the 95% that contain the true value or one of the 5% that do not.\n",
"\n",
"Here is an example of a common wrong interpretation of confidence intervalls:\n",
"\n",
"\n",
"* **Wrong** interpretation: a 95% confidence interval has a 95% probability of containing the population parameter. \n",
"\n",
"It is a common mistake, but this is *not* true. The 95% reflects a *long-run probability*. It means that if you take repeated samples and construct confidence intervals, then 95% of them will contain the population value. That is not the same as a particular confidence interval for a specific sample having a 95% probability of containing the value. In fact, for a specific confidence interval, the probability that it contains the population value is either 0 (it does not contain it) or 1 (it does contain it). You have no way of knowing which it is.\n",
"\n",
"We know (in large samples) that the sampling distribution of parameters (e.g. means) will be normal, and the **normal distribution** has been precisely defined such that it has a mean of 0 and a standard deviation of 1. We can use this information to compute the probability of a score occurring, or the limits between which a certain percentage of scores fall. \n",
"\n",
"We make use of the fact that 95% of **z-scores** fall between −1.96 and 1.96. This means that if our sample parameters (e.g. means) were normally distributed with a mean of 0 and a standard error of 1, then the limits of our confidence interval would be −1.96 and +1.96. Luckily we know from the **central limit theorem** that in large samples (above about 30) the sampling distribution will be normally distributed. \n",
"\n",
"**Visualize confidence intervals in plots**\n",
"\n",
"We saw that confidence intervals provide us with information about a parameter, and, therefore, you often see them displayed on graphs.\n",
"\n",
"The confidence interval is usually displayed using something called an **error bar**, which looks like the letter ‘I’. An error bar can represent the **standard deviation**, or the **standard error**, but more often than not it shows the **95% confidence interval** of the mean. So, often when you see a graph showing the mean, perhaps displayed as a bar or a symbol, it is accompanied by this I-shaped bar. "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Model 1: The Mean"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"First of all, we convert scores so that they have a mean of 0 and standard deviation of 1 (**z-scores**) using this equation:\n",
"\n",
"$$z = \\frac{X-\\bar{X}}{s}$$"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 -2.070197\n",
"1 -1.380131\n",
"2 -1.380131\n",
"3 -0.690066\n",
"4 -0.690066\n",
"5 -0.690066\n",
"6 -0.690066\n",
"7 0.000000\n",
"8 0.000000\n",
"9 0.000000\n",
"10 0.000000\n",
"11 0.000000\n",
"12 0.000000\n",
"13 0.690066\n",
"14 0.690066\n",
"15 0.690066\n",
"16 0.690066\n",
"17 1.380131\n",
"18 1.380131\n",
"19 2.070197\n",
"Name: height, dtype: float64\n"
]
}
],
"source": [
"# calculate z-scores\n",
"z = stats.zscore(df.height)\n",
"print(z)\n",
"# assign z-scores to df\n",
"df = df.assign(z = z)"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEJCAYAAACaFuz/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVGUlEQVR4nO3de3SU1b3G8SdhMpSIYwQJUbRNAyhewPslphatGIVwS8hSMJapohKKIpxWZMVAqlyKHBGFqS7bokQuKagE0SO2iFbl4rFkrQNLPFBEo0QyiEUZYijJJDl/0M5xJJcJvDuT2fl+1mJlZs877/7Ny8rD5t3vuyeuoaGhQQAAa8VHuwAAgFkEPQBYjqAHAMsR9ABgOYIeACzX7oI+GAyqoqJCwWAw2qUAgBXaXdD7/X7deOON8vv9ju9704hR2jRilOP7BYCWRDN/2l3QAwCcRdADgOUIegCwHEEPAJYj6AHAcgQ9AFjOaNC/9dZbysnJ0eDBgzVr1iyTXQEAmmAs6Pfu3auioiI9/fTTWrt2rT766CO98847proDADTBZWrH69ev15AhQ5SSkiJJWrBggTp37hy2TSAQUCAQCGszcaMUAHRkxoL+s88+U0JCgvLz81VZWanrr79ekydPDtumuLhYPp/PVAmwWG2wXgmujjXF1BE/M5xhLOjr6uq0detWLV26VImJiZowYYJKS0uVk5MT2sbr9So7OzvsfX6/X3l5eabKgiUSXPEqeHpTtMtoU3N+mRHtEhCjjAX9GWecofT0dHXr1k2SNGjQIG3fvj0s6D0ejzwej6kSAAAyOBl7ww03aOPGjQoEAqqrq9N7772nCy+80FR3AIAmGBvRX3zxxbr77rt1++23q7a2VhkZGRo1ipUjAaCtGQt6ScrNzVVubq7JLgAALWAKHwAsR9ADgOUIegCwHEEPAJYj6AHAcgQ9AFiOoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCWI+gBwHIEPQBYjqAHAMsR9ABgOYIeACxH0AOA5Qh6ALAcQQ8AliPoAcByBD0AWI6gBwDLEfQAYDmCHgAs5zK585///Oc6ePCgXK5j3Tz66KO6+OKLTXYJAPgeY0Hf0NCg8vJyvf3226GgBwC0PWOnbj755BNJ0l133aXhw4dr2bJlproCADTD2FA7EAgoPT1d06dPV21trcaOHasf//jHysjICNsmEAiEvc/v95sqCQA6JGNBf+mll+rSSy8NPc/NzdU777wTFvTFxcXy+XymSgAAyGDQb926VbW1tUpPT5d07Jz998/Ve71eZWdnh7X5/X7l5eWZKgsAOhxj5+gPHz6sefPm6ejRo6qqqlJpaaluuummsG08Ho/OPvvssD8pKSmmSgKADsnYiP6GG27Qtm3bNHLkSNXX1+v2228PO5UDAGgbRq97nDx5siZPnmyyCwBAC7gzFgAsR9ADgOUIegCwHEEPAJYj6AHAcgQ9AFiOoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCWI+gBwHIEPQBYjqAHAMsR9ABgOYIeACxH0AOA5Qh6ALAcQQ8AliPoAcByBD0AWI6gBwDLEfQAYDmCHgAsR9ADgOWMB/1jjz2madOmme4GANAEo0G/ZcsWlZaWmuwCANACl6kdf/PNN1qwYIHy8/O1c+fORrcJBAIKBAJhbX6/31RJANAhGQv6GTNmaMqUKaqsrGxym+LiYvl8PlMldBi1wXoluJhusV1H/HvuiJ/ZBCNB/+KLL+rMM89Uenq6Vq9e3eR2Xq9X2dnZYW1+v195eXkmyrJWgiteBU9vinYZbWrOLzOiXUKb4+8ZJ8pI0L/++us6cOCARowYoUOHDqm6ulpz5sxRQUFB2HYej0cej8dECQCAfzES9M8//3zo8erVq/XBBx8cF/IAgLbByS8AsJyxydh/y8nJUU5OjuluAABNYEQPAJYj6AHAcgQ9AFiOoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCWI+gBwHIRBX1jC5JNmjTJ8WIAAM5rdq2boqIi7d+/X2VlZTp48GCoPRgMau/evcaLAwCcvGaDPjc3V7t379auXbt08803h9o7deqkSy65xHRtAAAHNBv0/fv3V//+/XXttdcqJSWlrWoCADgoomWKKysr9eCDD+rQoUNqaGgItb/66qvGCgMAOCOioJ8xY4ZycnJ0wQUXKC4uznRNAAAHRRT0LpdLd955p+laAAAGRHR5Zd++fbVr1y7TtQAADIhoRL93716NGjVKZ511ljp37hxq5xw9ALR/EQX9lClTTNcBADAkoqA/99xzTdcBADAkoqC/5pprFBcXp4aGhtBVNz169NC7775rtDgAwMmLKOh37twZelxTU6PXXntNn376qbGiAADOafXqlW63Wzk5Odq0aZOJegAADotoRP/NN9+EHjc0NOjDDz9UIBAwVRMAwEGtPkcvSd27d9fDDz/c4vueeuop/fnPf1ZcXJxyc3O56QoAoqDV5+gj9cEHH+j999/X2rVrFQwGNWTIEA0cOFBpaWmt3hcA4MRFFPT19fVavHix3n33XQWDQWVkZCg/P18uV9Nvv+qqq/TCCy/I5XJp//79qqurU2JiomOFAwAiE1HQz58/Xzt37pTX61V9fb1WrlypefPmNfrNU9+VkJCghQsX6rnnntMtt9yinj17hr0eCASOO9fv9/tb+REAAM2JKOjfe+89vfzyy0pISJAkXX/99Ro+fHiLQS8d+8rBe+65R/n5+Vq1apVuu+220GvFxcXy+XwnWHrjaoP1SnDxVbiADTra77OpzxtR0Dc0NIRCXjp2ieV3nzdmz549qqmp0fnnn68uXbooMzPzuIXRvF6vsrOzw9r8fr/y8vIirf84Ca54FTzd+KWfWf/62dTrsWrOLzOiXQJgRHO/z7Emkvwx9bscUdD369dPc+bM0R133KG4uDgtXbq0xWURKioqtHDhQpWUlEiSNmzYoFGjRoVt4/F45PF4TrB0AEAkIgr6oqIizZo1S6NHj1Z9fb2uu+46TZ8+vdn3DBw4UNu3b9fIkSPVqVMnZWZmKisrq9n3AACc12zQ19TUaPr06Ro0aJDmzp0rSbr33nvVqVMnde3atcWd33///br//vudqRQAcEKaPeu/cOFCVVVV6bLLLgu1zZw5U4FAQIsWLTJeHADg5DUb9H/96181f/58de/ePdTWs2dPzZs3T2+++abx4gAAJ6/ZoE9ISNAPfvCD49q7du0qt9ttrCgAgHOaDfr4+HhVVVUd115VVaVgMGisKACAc5oN+qFDh6qwsFDV1dWhturqahUWFiozM9N4cQCAk9ds0Hu9Xp166qnKyMjQrbfeqtzcXGVkZMjj8WjixIltVSMA4CQ0e3llfHy8Zs6cqfz8fO3YsUPx8fEaMGCAkpOT26o+AMBJiuiGqV69eqlXr16mawEAGNBxVgsCgA6KoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCWI+gBwHIEPQBYjqAHAMsR9ABgOYIeACxH0AOA5Qh6ALAcQQ8AliPoAcByBD0AWI6gBwDLRfRVgifK5/Np3bp1kqSBAwdq6tSpJrsDADTC2Ih+8+bN2rhxo0pLS7VmzRrt2LFD69evN9UdAKAJxkb0PXr00LRp0+R2uyVJvXv31r59+8K2CQQCCgQCYW1+v99USQDQIRkL+r59+4Yel5eXa926dSopKQnbpri4WD6fz1QJAAAZPkcvSbt379b48eM1depUpaamhr3m9XqVnZ0d1ub3+5WXl2e6LADoMIwGfVlZmSZNmqSCggJlZWUd97rH45HH4zFZAgB0eMaCvrKyUhMnTtSCBQuUnp5uqhsAQAuMBf3ixYt19OhRzZ07N9Q2evRojRkzxlSXAIBGGAv6wsJCFRYWmto9ACBC3BkLAJYj6AHAcgQ9AFiOoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCWi6mgP/z33QpWV0uS/vHff1P5C8v0jy3vR7kqAGjfYibovyh9Rbsef0LBqirte/W/9NkLy6T6eu1d+ZL2rnop2uUBQLtlfD16p+xf/6YuefIJuRK76Mu33tZFsx+VO+k01R05om2/fkjn3Job7RIBoF2KmRF9fOfOciV2kSR1SkyUO+m0Y4+7dJHi4qJZGgC0azET9J7z++nvTzylf/r96nHdT1RevFTVn3+u8iUvqGvv3tEuDwAc5eScZMwEfeovxsrVtau2/eohffL7P+qL0lf0P1Me1D/3f6m0e8ZFuzwAcIzTc5Ixc44+3u1W2r3jlHbvONUeOqT6YJ0STvMo3hUzHwEAIuL0nGRMpWTl62/oH5u3qObrrxXncqnLmSnqMXCguqdfHe3SAMAxTs9Jxsypm89LVurbTz/VWSOHq2vv3kq5JVPdr01XxcurtW/ta9EuDwAc4/ScZMyM6A/+basueeI/JUlJFw/Qjt/MVP/Zj6rblVdo26+n6azhQ6NcIQA4I/UXY1W+ZKm2/eoh1R05oob6eu1b+5q6XXWl+kyc0Or9xUzQ1x05ovpgUPEulxrq6hQ8fFjSsf/KxMVzeSUAezg9JxkzQX9a/4u0c85cnfGTn+irjZt0+mWXqubrr7Xn6Wd16nnnRbs8AHCUk3OSMRP0afeMU8VLq/XV5s06td956pUzUsGqKp1+5eXqeePPol0eADjm85KVqjl4UGeNHK6v3t2oU88/T65TTlHFy6t19MCBVp+qjpmgj09I0A/H3BbW5k5KUkrmTVGqCADMcHpOMmauumnOx797JtolAIBj/j0nKcmROcmYGdE3hyUQANjE6TlJoyP6qqoqDR06VBUVFSa7UcotmUb3DwBtKe2ecerat29oTvKHd9wuxcXp9CsvV+8J97Z6f8ZG9Nu2bVNhYaHKy8tNdQEAVnJ6TtLYiH7VqlUqKipScnKyI/ur2vNJs38AoCM4kTlJYyP62bNnt7hNIBBQIBAIa/P7/Y1uu3uhT//cV6mE05Okhu+9GCdd8XsmZAHYL+aWQCguLpbP54to2/6zZ2r71Gk69z8mq2sfJl8BdEwnMicZ1aD3er3Kzs4Oa/P7/crLyztuW1fXU5R61y/0+YoSXTCjsK1KBICYF9Wg93g88ng8EW/f7YrL1e2Kyw1WBADR19K8Y9feaa3aX0xfR3/wb1vV7corol0GADjK6TlJ40H/1ltvGdv35yv+RNADsI7Tc5KxvQTC9/+lAwALfHdO0pH9ObKXKHF3Oz3aJQCAEU7OScb0iP6CGQ9HuwQAaPdiOugBAC0j6AHAcgQ9AFgu5oL+k9//MewnANjKqbyLuaAP/O/OYz8/2hnlSgDALKfyLuaCHgDQOgQ9AFiOoAcAy8Vc0Me73cd+dnZHuRIAMMupvIu5oB/w2JywnwBgK6fyLuaCHgDQOgQ9AFiOoAcAy8Vc0H++4k+qO3o0rO3j37Xu21YAIBY4lXcxF/QVL5fqw4dnqPbQoVBb1cd7olgRAJjhVN7FXNAnnnOOkm/8mbY/VKDqii+ONfJNUwAs5FTexd43TMVJZw6+We6kJO2Y8Rud+6vJinN1inZVAOA8h/Iu9oL+X/+adU+/WglJp2nXvPlqqAtGtyYAMMGhvIu5Uzc9MweFHnvO76cLHy3SKWlpUawIAMxwKu9iLujPHHJL2PPEc87Whb+ZHqVqAMAcp/Iu5oIeANA6BD0AWI6gBwDLGQ36V199VUOGDFFmZqaWL19usisAQBOMXV65f/9+LViwQKtXr5bb7dbo0aN19dVXq0+fPqa6BAA0wljQb968Wddcc42SkpIkSTfffLPeeOMN3XfffaFtAoGAAoFA2Pu++OLY3V9+v/+E+/428FWj7V/V1DT7eqyqqKiw7jO1hM/cMdj0mSPJn4qKipPuJyUlRS5XeLTHNTQ0GFlA4Nlnn1V1dbWmTJkiSXrxxRe1fft2zZw5M7TNokWL5PP5THQPAB3Shg0bdPbZZ4e1GRvR19fXKy4uLvS8oaEh7Lkkeb1eZWdnh7XV1NRo7969Sk1NVadOJ7+0gd/vV15enpYvX66UlJST3p+tOE4t4xhFhuPUMpPHqLH9GQv6lJQUbd26NfT8wIEDSk5ODtvG4/HI4/Ec9940A3e6pqSkHPevHI7HcWoZxygyHKeWtdUxMnbVzbXXXqstW7bo4MGDOnLkiP7yl7/opz/9qanuAABNMDai79mzp6ZMmaKxY8eqtrZWubm5GjBggKnuAABNMLp65bBhwzRs2DCTXQAAWmD9nbEej0f33Xdfo3MB+H8cp5ZxjCLDcWpZWx8jY5dXAgDaB+tH9ADQ0RH0AGC5DhP0ZWVlys3N1YgRI+T1ekNLLeB4Tz75pBYtWhTtMtodFumLTFVVlYYOHerI7fw28vl8ysrKUlZWlubNm9cmfXaYoH/wwQc1a9YsvfLKKxo2bJhmzZoV7ZLancOHD6ugoEDPP/98tEtpd/69SN+KFSu0Zs0arVy5Uh9//HG0y2p3tm3bpjFjxqi8vDzapbRLmzdv1saNG1VaWqo1a9Zox44dWr9+vfF+O0TQ19TU6IEHHlC/fv0kSeedd54qKyujXFX7s2HDBqWmpurOO++MdintzncX6UtMTAwt0odwq1atUlFR0XF3weOYHj16aNq0aXK73UpISFDv3r21b98+4/0avY6+vXC73RoxYoSkY2vw+Hw+DRo0qIV3dTwjR46UJE7bNOLLL79Ujx49Qs+Tk5O1ffv2KFbUPs2ePTvaJbRrffv2DT0uLy/XunXrVFJSYrxf64J+3bp1+u1vfxvWlpaWpiVLlqimpkbTpk1TMBjU+PHjo1Rh9DV3jNC4SBbpAyK1e/dujR8/XlOnTlVqaqrx/qwL+sGDB2vw4MHHtX/77beaMGGCkpKS9MwzzyghISEK1bUPTR0jNC2SRfqASJSVlWnSpEkqKChQVlZWm/TZIc7RS8cmY3/0ox/pySeflNvtjnY5iDEs0gcnVFZWauLEiXr88cfbLOQlC0f0jfnoo4+0YcMG9enTJ7T+fXJysv7whz9EuTLEChbpgxMWL16so0ePau7cuaG20aNHa8yYMUb7ZQkEALBchzl1AwAdFUEPAJYj6AHAcgQ9AFiOoAcAyxH0AGA5gh4ALNchbpgCTsby5cu1atWq0PM9e/bo7rvv1uTJk6NXFNAK3DAFtMKKFSv00ksvadmyZUpMTIx2OUBEGNEDEVq/fr2ee+45lZSUEPKIKQQ9EIGysjI98sgjWrJkSdi69EAsYDIWaMGePXv0wAMPaP78+erTp0+0ywFajXP0QAvGjRunDz/8UL169VJdXZ0k6aKLLuLblBAzCHoAsBynbgDAcgQ9AFiOoAcAyxH0AGA5gh4ALEfQA4DlCHoAsBxBDwCW+z/60GiraZCfugAAAABJRU5ErkJggg==",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt = sns.histplot(x=\"z\", data = df, bins=5);\n",
"\n",
"# draw a vertical line\n",
"plt.axvline(1.96, 0, 1, linewidth=2, color='r');\n",
"\n",
"# add text\n",
"plt.text(2.1, 0.3,'z = 1.96', rotation=90, color='r');\n",
"plt.axvline(-1.96, 0, 1, linewidth=2, color='r');\n",
"plt.text(-2.2, 0.3,'z = -1.96', rotation=90, color='r');"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"If we know that our limits are −1.96 and 1.96 as z-scores, then to find out the corresponding scores in our raw data we can replace z in the equation (because there are two values, we get two equations):\n",
"\n",
"$$1.96 = \\frac{X-\\bar{X}}{s}$$\n",
"\n",
"and \n",
"\n",
"$$-1.96 = \\frac{X-\\bar{X}}{s}$$\n",
"\n",
"We rearrange these equations to discover the value of X:\n",
"\n",
"$$1.96 \\times s = X - \\bar{X}$$ \n",
"\n",
"$$(1.96 \\times s) + \\bar{X} = X$$ \n",
"\n",
"and\n",
"\n",
"$$-1.96 \\times s = X - \\bar{X}$$\n",
"\n",
"$$(-1.96 \\times s) + \\bar{X} = X$$ \n",
"\n",
"Therefore, the confidence interval can easily be calculated once the **standard deviation** (s in the equation) and **mean** ($\\bar{x}$ in the equation) are known. \n",
"\n",
"However, we use the **standard error** and not the standard deviation because we’re interested in the variability of sample means, not the variability in observations within the sample. \n",
"\n",
"The lower boundary of the confidence interval is, therefore, the mean minus 1.96 times the standard error, and the upper boundary is the mean plus 1.96 standard errors: \n",
"\n",
"\n",
"**lower boundary of confidence intervall** = $\\bar{X} - (1.96 \\times SE)$\n",
"\n",
"**upper boundary of confidence intervall** = $\\bar{X} + (1.96 \\times SE)$\n",
"\n",
"As such, the mean is always in the centre of the confidence interval. We know that 95% of confidence intervals contain the population mean, so we can assume this confidence interval contains the true mean; therefore, if the interval is small,the sample mean must be very close to the true man. Conversely, if the confidenve interval is very wide then the sample mean could be very different from the true mean, indicating that it is a bad representation of the population."
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Lower boundary of CI 164.34838823311972\n",
"Upper boundary of CI 165.65161176688028\n"
]
}
],
"source": [
"# lower boundary\n",
"lb = (df.height.mean() - (1.96*se))\n",
"# upper boundary\n",
"up = (df.height.mean() + (1.96*se))\n",
"print('Lower boundary of CI', lb)\n",
"print('Upper boundary of CI', up)"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEJCAYAAACaFuz/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAom0lEQVR4nO3deVRU58EG8Gc2hnUAZVNwww1RFLdERCUaYxLXon4uIQlN2iQ2TfNpzknqMVr7xdTms23SJnxpa06amNS2MUYT0yZRI264L4kL4IooCIMowjADzHq/P9DREUE03HmHO8/vHA9yZ+A+L8vj9Z1736uSJEkCEREpllp0ACIikheLnohI4Vj0REQKx6InIlI4Fj0RkcL5XNE7HA6UlpbC4XCIjkJEpAha0QFuZTQa8eCDD2LLli1ISEgQHYd8xJ7ZWXA1NEAdGIi0T1YLyTBr0b9Rb3UiSK/BmuWThWTwha8DtT8+d0RPdDuuhgaPtyLUW50eb0Xwha8DtT8seiIihWPRExEpHIueiEjhWPRERArHoiciUjhZiz43NxfTp0/Ho48+itdff13OXRERUTNkK/qSkhIsXboU7777LjZs2ICCggJs375drt0REVEzZLtgavPmzZg4cSLi4uIAAG+99Rb0er3Hc0wmE0wmk8c2o9EoVyQiIr8kW9GfP38eOp0O8+bNQ3l5OR544AHMnz/f4zmrVq1CTk6OXBFIwewOF3Ra/3qJye5wiY5A7ZRsRe90OnHw4EF8/PHHCA4Oxs9+9jOsX78e06dPdz8nOzsbmZmZHh9nNBqRlZUlVyxSCJ1WjUXv7hK2fxH7Xv58utf3ScogW9FHRUUhLS0NHTp0AACMHz8eR48e9Sh6g8EAg8EgVwQiIoKML8aOHTsWeXl5MJlMcDqd2LlzJ/r37y/X7oiIqBmyHdEPGjQIP/3pT/HYY4/BbrcjPT0dM2bMkGt3RETUDFmXKZ45cyZmzpwp5y6IiOgO/Ou0BSIiP8SiJyJSOBY9EZHCseiJiBSORU9EpHAseiIihWPRExEpHIueiEjhWPRERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoVj0RMRKRyLnohI4Vj0REQKx6InIlI4Fj0RkcKx6ImIFI5FT0SkcCx6IiKFY9ETESkci56ISOFY9ERECqeV85M/8cQTqKqqglbbuJvXXnsNgwYNknOXRER0C9mKXpIkFBcXY+vWre6iJyIi75Nt6qaoqAgA8PTTT2Pq1Kn4+9//LteuiIioBbIdaptMJqSlpWHJkiWw2+148skn0aNHD6Snp3s8x2QyeXyc0WiUKxIRkV+SregHDx6MwYMHu9+fOXMmtm/f7lH0q1atQk5OjlwRiIgIMhb9wYMHYbfbkZaWBqBxzv7Wufrs7GxkZmZ6bDMajcjKypIrFhGR35Ftjr62thYrVqyA1WqF2WzG+vXr8dBDD3k8x2AwICEhweNPXFycXJGIiPySbEf0Y8eOxZEjR/CjH/0ILpcLjz32mMdUDhEReYes5z3Onz8f8+fPl3MXRER0B7wylohI4Vj0REQKx6InIlI4Fj0RkcKx6ImIFI5FT0SkcCx6IiKFY9ETESkci56ISOFY9ERECseiJyJSOBY9EZHCseiJiBSORU9EpHAseiIihWPRExEpHIueiEjhWPRERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoVj0RMRKRyLnohI4Vj0REQKJ3vR/+///i8WLlwo926IiKgZshb9nj17sH79ejl3QUREd6CV6xNXV1fjrbfewrx583DixInbPsdkMsFkMnlsMxqNckUiIvJLshX9r371KyxYsADl5eXNPmfVqlXIycmRK4LfsDtc0Gn5covS2R0u0RG8jj/bbUOWov/000/RqVMnpKWlYd26dc0+Lzs7G5mZmR7bjEYjsrKy5IilWDqtGove3SU6hqwm3fT3Re/uwvLn04VlEeXWwlP69xyAX36f5SBL0X/11VeorKzEtGnTUFNTg7q6OixfvhyLFi3yeJ7BYIDBYJAjAhERXSNL0X/wwQfuv69btw779+9vUvJEROQdnPwiIlI42V6MvW769OmYPn263LshIqJm8IieiEjhWPRERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoVj0RMRKRyLnohI4Vj0REQK16qiv92CZC+++GKbhyEiorbX4lo3S5cuRUVFBQ4dOoSqqir3dofDgZKSEtnDERHRD9di0c+cOROnT5/GyZMn8fDDD7u3azQapKamyp2NiIjaQItFn5KSgpSUFIwcORJxcXHeykRERG2oVcsUl5eX4+WXX0ZNTQ0kSXJv//LLL2ULRkREbaNVRf+rX/0K06dPR3JyMlQqldyZiIioDbWq6LVaLZ566im5sxARkQxadXpl7969cfLkSbmzEBGRDFp1RF9SUoIZM2agc+fO0Ov17u2coyci8n2tKvoFCxbInYOIiGTSqqLv06eP3DmIiEgmrSr6ESNGQKVSQZIk91k30dHR2LFjh6zhiIjoh2tV0Z84ccL9d5vNhn//+984d+6cbKGIqG0E1ZvgUqlhDQwVHYUEuuvVKwMCAjB9+nTs2rVLjjzkB0bt/rvoCIqVeuQ/AIDA+lqM2v13jN79ETLyPkTavk8Q2FArOB2J0qoj+urqavffJUnC8ePHYTKZ5MpECjJ61yoAnhfZhdRVY/SujwAAO9OfFJBKuUItVwEAySe3oaxTEop6DAMAdLvwPQYe34T9w2aIjEeC3PUcPQB07NgRr7766h0/7k9/+hM2btwIlUqFmTNn8qIrP1QSn4KexQdQ2DcDDfoQAMDA/M3I7zdWcDJlC7FcxeHUKe73z3dNRbcLRwQmIpHueo6+tfbv34+9e/diw4YNcDgcmDhxIjIyMpCYmHjXn4var+LuQ1ATHov+hVtR2HcMrnTsCodGh6oOXURHUySdvR4R1WWoC45AqPkKzKEdATTO1UtcvcRvtaroXS4X3n//fezYsQMOhwPp6emYN28etNrmP/y+++7DRx99BK1Wi4qKCjidTgQHB7dZcGo/rkbGY//QTAw++jUiq8tEx1E0Y2wfJJ/cDoPpElSSEweHZCK+rADJJ7YhP4n/i/JXrSr6P/zhDzhx4gSys7PhcrnwySefYMWKFbe989TNdDod3n77bfztb3/DI488gtjYWI/HTSZTk7l+o9F4l0Og9sCmD8G+YTOQdGon9LY60XEUqzApAwCgcjkRaDUDAExh0dhz3yyYQ6NERiOBWlX0O3fuxGeffQadTgcAeOCBBzB16tQ7Fj3QeMvBZ555BvPmzcOaNWswe/Zs92OrVq1CTk7OPUa/PbvDBZ2Wt8L1SSoVLiSkoDqiE/QNZp7yJ5Pw6nJYQjugPigcMZfOIrK6DNXhce2y6P3t91mu8baq6CVJcpc80HiK5c3v387Zs2dhs9nQr18/BAUFYcKECU0WRsvOzkZmZqbHNqPRiKysrNbmb0KnVWPRu/516ufy59NFR2hWeI0Rqce+hk0XhLM9hmPQsW9QGxaN/nW5+G7gRM7Vt7HEcwfRreQI9g7/LyRczEfXkqO4FJ2I3mf3Isx8BWd6jhAd8a742++zXL/LrSr6pKQkLF++HI8//jhUKhU+/vjjOy6LUFpairfffhv//Oc/AQBbtmzBjBmep3YZDAYYDIZ7jE7tQf8TW1HYNwPBddUYcuTf2H3/XJgMMQg1X8bA45uxe8Rc0REVpcvFY9gx8gk4tQFIuFiAvcNnwaYPhsZhw6i9/2h3RU9to1VFv3TpUrz++uuYM2cOXC4XRo8ejSVLlrT4MRkZGTh69Ch+9KMfQaPRYMKECZg0aVKbhKb2Q+104FJ0IiBJSDx3ECZDDADAHBoFtcspOJ3yONU6OLUBAACHNgA2feMJEE5tACTwtBt/1WLR22w2LFmyBOPHj8cbb7wBAHj22Weh0WgQGnrn+dVf/OIX+MUvftE2SaldklRqhNVWojYsGnlpj7u3R1SXQeLdytrc1cjOSD36FU72GomyTklIOrkDpfHJSLhYgJrwGNHxSJAWZ/3ffvttmM1mDBkyxL1t2bJlMJlMeOedd2QPR+3fiT6jcd/BzwDJ5T66jK04g+GH1qPg2hki1HYK+4yBTReIUXtWo39hLhKLD2LUntUIrq/h6ZV+rMUj+m3btmHt2rUIDAx0b4uNjcWKFSswe/ZsrlNPd3SlY1fkZjwDqNQ3beuC3Ixn4dS2/II+3T2XRouCfuNQ0G8cAmx1ULlcsAUEQVJrREcjgVo8otfpdB4lf11oaCgCAgJkC0XKcmvJOLR6OLU6GEwVghL5B1tAMKyBoe6vf8ylIsGJSJQWi16tVsNsNjfZbjab4XA4ZAtF/mHg8U2iI/iVbiVc68ZftVj0kydPxuLFi1FXd+NKxrq6OixevBgTJkyQPRwpW97IJ0RH8CsHhmbe+UmkSC0WfXZ2NsLCwpCeno5Zs2Zh5syZSE9Ph8FgwM9//nNvZSSF4RSCfIKvLVMMAAZTBXqf2YOeRfsQYqkSmIpEa/HFWLVajWXLlmHevHnIz8+HWq3GwIEDERPD07SodW43D9/v1A40BIYCkGAyxDb9ILpnQ478B3kjH0dsxWmkFGxBWVwfqACMOPAp8pPGwhjH+z/7o1ZdMBUfH4/4+Hi5s5AC3X/wM6hcLtgDggA03s9A32DB0O83AFBh65ifCM2nVD3PHcCe4bNgCe0AACjqPgzDD69n0fupVhU90b3KG5GF1GNfozS+P0oSUgAAo3Z/zPl5maklFywhke7364MMvDLWj/nPsnAkRH1wOPYNn4mI6nIMPLYRaqcDt95akNpOSN1VDMj/Fg6NDl1LjwIANA47ehQfglXP+0H4KxY9yc6l1uLYgAm4GtkZafs/gcZlFx1JsXbdPxc1hhjUBYUjrPYyAKD7hcPoZDyJY8njBacjUTh1Q15TkpCCGkMs4ssKREdRLHNYFMxhUSjpMtC97WyP+3A28X6BqUg0HtGTrFQuJxLPHYDeaobaaUd8WQEiq8uQXJgLjcMmOp7i6Gz1GFCwBUH1NdDZ6jH0uy/w4LaVGHr4C+itFtHxSBAWPclq4PFNiKgxwqXSoN/JHdBbLTjdcwQ0LidSCr4VHU9xBh3fCKdaC7suEMknt8ESHIF9w2eiJjyWVyL7MRY9ySrcdAmHU6fAHhCEyOoyHEl5FJXRiTiWPB7hpkui4ylOUL0JhUkZcGj1CKu9jBN9M2AO7YgzPUcgqMF0509AisSiJ1m5NBpo7VYAgFUf4r7ZiNrl5Ol+MnCpNQisrwXQeEplwLUbsets9deuYiB/xBdjSVYl8QMwct+/UNR9KKoiEzDsu89hjO2NTsaTKOvUV3Q8xTnbYzjS9/0DJfEDYA7pgPsPfIrKqO6IrTyHc92G3PkTkCKx6ElW57umwqoPQXxZAUIs1VBJTnQynkJ5bB+c75oqOp7iGOP6oD7IgM7lJxBquQqrPhQBtnoU9hmDSzGJouORICx6kp0xtjeMsb1Fx/AbNeFxqAmPEx2DfAjn6ImIFI5H9CSrHsWHWnz8XPehXkriH5ILc1t8vKDfOC8lIV/CoidZhdVWolPFaZTH9UGT0z540k2bc2j16HH+MIq6DwW/wHQdi55kdTTlEQQ11KKyYzeUd0oSHUfxTvVOR6ilCraAYL7YTW6coyfZHe/3IDpUl4mO4Tfykx6AzlYvOgb5EBY9yc4S2gH5nBv2GmtgGM70ShMdg3wIi56ISOFknaPPycnB119/DQDIyMjAK6+8IufuiIjoNmQ7ot+9ezfy8vKwfv16fP7558jPz8fmzZvl2h0RETVDtiP66OhoLFy4EAEBAQCAnj17oqzM8wU5k8kEk8lzRT2j0ShXJCK/obU3wKELdL8l/ybbEX3v3r2RmpoKACguLsbXX3+NjIwMj+esWrUKDz74oMefrKwsuSKRQCP3/tPjLclrxIG1Hm/Jv8l+Hv3p06fx3HPP4ZVXXkH37t09HsvOzkZmZqbHNqPRyLJXoBvLEzsEJ/E3XJyYZC76Q4cO4cUXX8SiRYswadKkJo8bDAYYDAY5IxAR+T3Zir68vBw///nP8dZbbyEtjef0EhGJIlvRv//++7BarXjjjTfc2+bMmYO5c+fKtUsiIroN2Yp+8eLFWLx4sVyfntqZqsj4a28TBCeRl83uhEoF6LQaoTnK4vpce8u7eBEXNSMvKeg31uOtUlyoMKFrrAE2hxPny02wOVyABATqNegaGyas8IsS7/N4S/6NRU/0A1htjWcTlV+2ICJMj+iIYADAlZp6lF4yo0fncJHxiABwrRuiNmGzO90lDwAdw4Ngd7gEJiK6gUVP3iE1PZ9ba28QEKRtOV0S6hrsCNBp0GC7cY2Aze4Uet8PfUNtk22h5isCkpAvYNGTV4zas7rJtpH7PxGQpG2FhwSg/LIFtXU2GK9YAABXaxtwprQaMRFBXs+js9VDZ6vHfYfWQ2tvcL8fYLVg6PcbvJ6HfAPn6ElW9x/4FOE1FdC47JjwbY57uwou1BjiBCZrG52iQgEALkmC49pUTVCAFonx4QgM8P6v1+CjXyHqynkAwITcd93bJZUa5bG9vZ6HfAOLnmR1aPBU6OwNGHh8E44OeNi93aVSwaoPFZis7dQ12KEP0CBAp4HJYkVdgwNBeq2Qot8/bAYAYODxjR5fb/JvLHqSXX1QOA4Pmtxku87eAHuA96c32lJldR2u1DQgsXM4rprqUWVqQFhIAC5drYPV7kRMZPCdP0kbCjFXwRLaAcVdU2EwVTR53GSI9Woe8g0sepLViP2fIm/k43ho65/R+OrkzS/KqvDVwwsEJWsbV01W9O4SAY1ajau1jYWv1arhjAzG2dJqrxd9v1PbcXBIJoZ+/+VtHlVh65ifeDUP+QYWPckqb+TjAICvHn5JcBJ5qFSARt14ToNGrYZWe/3vYk65OTikcTXYrWN+KmT/5JtY9OQVaqcdsZfOIuCWUyrPd00VE6iNhATqUFJRi9gOwQgPDUD5FQsiw/S4WmtFkF7cr5feakHCxXzobvl6n+g7RlAiEolFT14x/PDn0DpsqA+6sSy1BFW7L/q4jiEwVllwprQaTlfjtNSVmnqEBQcgIVrci83DDn+OhsAwWIJ5ZS6x6MlLAq0WbB/1Y9Ex2pxarULnqFB0jgqFw+mCJEnQatRQqQReLQVAJblwaPBUoRnId/CCKfIKU2gU9FaL6Biy0mrU0Gk1wkseAEyGGITWXhYdg3wEj+jJK8rj+iAj70PUhkbBpb5xfLFv+H8JTPXD1VtbvjWiqHn6qoh4jN7zMaz6EEiqGyto8qwb/8SiJ6/oc3YPziTeh7ogZc0Zl16qhdXuhFZz+/8cJ3Xr4OVEjXoWH8T3AyfCEhQhZP/kW1j05BUOjQ5FPYaLjtHmEjuH4+zFGnSJDRN6ls2t7Fo9ynnTEbqGc/TkFVc6dkW3C99DbzW7F9rS2epFx/rBNBq1+8wbX3KlYxf0O7kdEdVlMJgq3H/IP/nOIQgpWo/iQ1C7nOhfmHvT1vZ/ZSwAGEICYAgJEB3DQ+fyEwCAuIrTN23llbH+ikVPXvHNQ/8tOoJf4ZWxdDMWPckqvqwAFzsno0fxods+fq77UC8nUrbEcwdQ1GM4kj3+53RDQb9xXk5EvoBFT7IKrqsGAITxnG6vcGj1AAC7rn2vCkpti0VPsjrdayQA4GgK10b3hgtdBgIATvdKE5yEfAmLnmQ1etcqtHTz1J3pT3ovjIzKKs3oHB3qfivKw9++A6mFr/em8S94MQ35ChY9ySrfT+aELQ12j7ei7EjPFrp/8k2yFr3ZbMacOXPwl7/8BQkJCXLuinxUVYcuoiP4lZtXByW6TrYLpo4cOYK5c+eiuLhYrl0QEVEryFb0a9aswdKlSxETEyPXLoiIqBVkm7r5zW9+c8fnmEwmmEwmj21Go1GuSCRQ31M7cbLPaNExZKO+dutAtaBbCN5q0LGvcSTlUdExyEcIfTF21apVyMnJERmBvCSmskjRRd8zPsLjrWgGUyUgSY03tSW/J7Tos7OzkZmZ6bHNaDQiKytLUCKSS11QBO47+BmqIjrDqb2xLgyvjJVHgz4EY3atQnVEJzg0Ovd2Xhnrn4QWvcFggMHAswT8gV0XCDuA4Pqbpup4sCmb6ojOqBYdgnwGz6Mnr7h+ZazW3gCHLlBwGuU73SsNaqcdIXXVqA2NgtrlgOumI3vyL7KvR5+bm8tz6AkhlqsYk/chMnatgr7BjIy8DxFirhIdq03Z7E7UWx0ef0SJqC7H2J1/w/DDnyPQasaD299D5NUyYXlILN54hLyif2EuCpLGwhoQDGtgKIq7piKlYLPoWG2mosqCUyVXcd5o8vgjSr+TO7Bv2EzYdIFoCAzD9ymPIPnEVmF5SCxO3ZBXBNjrcTmqG3Cq8f3zXVPRpfSY2FBt6GqtFX27RkKn1dz5yV6gcdlhDu3ofr8yOhF9T+8SmIhE4hE9eYUEFdROB66/Aqu3WqCSXGJDtSGdVu0zJQ8ALpUaWnsDrn+9QyzKmiaju8MjevKKC10G4r5DnyHAVoe+p3aic/lJnFXQzcJDgwJQfsUCQ3CAx0VTom4YfiZxBNL2r4HeakHqkf8g+sp5HEseLyQLiceiJ68oSUiBJTgSMZVFUEsuHOv/UONUjkJcrW0AANSYrR7bk7p1EBEHl2ISYQ6JRNSVC1BJLpzpOcJjKof8C4uevCL1yH9gjO2NU73SFHman6hCb4lackLjckBSqeBScZbWn7HoySsuRSeiU8Up9C/MxdWIzjDG9kZFTE+Pq2TbM4fThepaK5wu6doWCVa7C11jw4Tk6VpyBH3O7EZlVA+oJBd6Fe1Hfr+xKI/rKyQPicWiJ68o69wPZZ37QeVyopPxFPqezsPA/E345qH/Fh2tTVww1kKtBhpsToQG6WCutyMkUNz/XHoUH8LOtMdhDWz8hyaw3oT7Dq9n0fspFj15RYeqEkRduYCoK+cRaDXjSoeuqOyonDl6u8OJvt064GKlGR0MgYjtEIzzxlpheRxavbvkAaAhyACnmr/u/orfefKKEQc+hVUfgtOJI1CSkAJJraw5Y622cTx6nQYNNgciwwIhSdIdPko+lzt2w4CCb1HcJRWSSo2EsgJYgiNgMFUAAEyGWGHZyPtY9OQVWx54FtGXixF9+Tx6Fh+EKTQKl6O64XzXVNHR2oRWo0bl1ToEB+pQcbUOGrUKLoFF39l4AgAQfbnYY/vQ778EoMLWMT/xfigShkVPXmHVh6I0fgAuRScipvIcEs8dQMfTpYop+vioUFSbrQgJ0iGoTouKqjrEdQgRlmfrmJ8K2zf5HhY9eUWf07sQfbkYgVYzLkUnorBvBi537Co6VpvRatXoYAhEg9WBuA7BkCKDhd5tSuOwoffZvYi+XAxJpUZFTCLOJt4HF+fp/RK/6+QVGqcdBUkP4GpEZ0Xe9aiuwY7zxlqoVEDP+HCcLqlGt04GYWfeDMzfDEmlQkHSAwAkdC09huTCrTje/yEheUgsFj15RWHfMUgsPoS+p3dBJblQ2bEbziber5gXZcuvWNCjswElFbXQaTVIiAlD+WULeiVECMljMF3C9tFPud+/0qErMnatEpKFxFPGbxn5vL6nd6HjlQs4120IznUfisjqMiSd2iE6VptxuSQEBtw4bjKEBAg968aqD4HOVu9+X+O0w6YLEpaHxOIRPXlFTOU55KVlQVI3rvB4KboHRu/+GIV4QGywNqJSqeB03liN02oTd9MRAKgPDMOoPX+HMa4PXCo14i6dhTUgGMmFuQB471h/w6InL5HcJQ8ALrVWUeuvxEQGo6isBnaHCxcqTDDX2REfHSosT11wBOqCI9zvl/GKWL/GoievMIXFoN+JbTjfNRUSgO4XvkdtWLToWG3GEBIAfYAG5jobJDQW/81TOd52uleasH2T72HRk1fkJ49DcuFWjNz3LwASKjt2R37SWNGxfjDHTdM1GrUK4aF6j8e0Gu/+r2XE/jWQbjqrSVKpYNcF4VJ0D1zsnOzVLOQ7WPTkFQ6tHkdTHvHYFmq+DHtA+36BsLC45Ts3pfSM8lKSRsW3uQBNb7Wg+/nvEGCrx7nuQ72ah3wDi56EGbn3X9g0/gXRMX6QiDA96hrsMATrEWnQC52uAQBjXJ/bbr/YuR/S9q9h0fspFj0Jo4K40w/bSpeYMLhcEmosVpRftsDpkhAZpkdEqB4aL0/btMShC/SY0iH/wqInYSQoo3jUahUiwwIRGRYIu8OJq7VWFJXVQK/ToGucQXS8RpIEtUs5N2Onu8OiJ2pDDqcEh9MFp1OCU+P9/7HcfJGUe5u9AT0ufIerEZ28nod8A4ueZPXwt+/c9shdBQkap9iLitqKzeFEda0V1bVWQAVEhurRMyEcOq3mzh/cxh7a+mcAKsA9LaaCNSAIlVE9rq17Q/5I1qL/8ssv8ec//xkOhwPZ2dnIysqSc3fkg3akZ4uOIKuiizWw2p0IDw1Al9gwBOnFHjt99fBLQvdPvkm2n8qKigq89dZbWLduHQICAjBnzhzcf//96NWrl1y7JB9UH+Qjc9QysTTYoVIBV01WXDVZmzzeP7GjgFREnmQr+t27d2PEiBGIiIgAADz88MP45ptv8MILN06nM5lMMJlMHh938eJFAIDRaLznfVtMl+/5Y9uj0tJSxY/5ss3m/rvFdFnImO11N86Zt1z7se3S4fqW28/Ht2XG0tLSJl8HpfOHn+2blZaW/uDPERcXB63Ws9pVkkxL7P31r39FXV0dFixYAAD49NNPcfToUSxbtsz9nHfeeQc5OTly7J6IyC9t2bIFCQkJHttkO6J3uVxQ3XwptiR5vA8A2dnZyMzM9Nhms9lQUlKC7t27Q6O5uxezjEYjsrKysHr1asTFxd17eB+glLEoZRwAx+KrlDKWthrH7T5WtqKPi4vDwYMH3e9XVlYiJibG4zkGgwEGQ9M53MTExB+871v/RWuvlDIWpYwD4Fh8lVLGIsc4ZLt0b+TIkdizZw+qqqpQX1+PTZs2YcyYMXLtjoiImiHbEX1sbCwWLFiAJ598Ena7HTNnzsTAgQPl2h0RETVD1pN+p0yZgilTpsi5CyIiugPfWXWpDRgMBrzwwgu3nfdvb5QyFqWMA+BYfJVSxiLnOGQ7vZKIiHyDoo7oiYioKRY9EZHCtcuiN5vNmDx5svty4e+++w6zZs3CpEmT8NJLL8F27TLxb7/9FtOmTcPUqVPx/PPPo6amRmTs22rtWDZv3owpU6Zg0qRJWLhwoXu7r2jtOK7btm0bxo0bJyLqHbV2LDk5ORg7diymTZuGadOmYfXq1SJj31Zrx1JUVIQnnngCU6dOxU9+8pN2+7tSWFjo/n5MmzYNo0ePxuTJkwUn99Ta70l+fj5mzJiBqVOn4rnnnmuyXMxdkdqZ77//Xpo8ebLUv39/qaSkRKqtrZXS09OlwsJCSZIkacGCBdLq1avd241GoyRJkvTHP/5RWrZsmcjoTbR2LBaLRRo1apRUWVkpSZIkzZ8/X/rXv/4lMrqH1o7jusrKSumRRx6Rxo4dKypys+5mLM8995x0+PBhkXFb1NqxuFwuacKECdL27dslSZKk3/3ud9KKFStERm/ibn/GJEmS6urqpEmTJkkHDhwQEfm27mYcc+fOlbZt2yZJkiT99re/ld5888173m+7O6Jfs2YNli5d6r7KdteuXUhNTUVSUhIAYPHixXjooYdgt9uxdOlSxMbGAgD69u2L8vJyYblvp7VjCQ4ORm5uLqKiolBfX48rV6741BkGrR3HdYsXL/ZY3M6X3M1Yjh8/jr/+9a+YMmUKXnvtNVitTVevFKm1Y8nPz0dwcLD7gsZ58+b53JLid/szBjSutzV8+HAMGzbM63mbczfjcLlcsFgsAID6+noEBgbe837b3Y1HfvOb33i8f/78eQQHB2PBggUoKirCkCFDsHDhQuj1evcXrKGhAStXrsQTTzwhInKzWjsWANDpdNi+fTteeeUVxMTEYNSoUSIi39bdjOOjjz5CcnIyBg0aJCLqHbV2LBaLBf369cPLL7+Mbt26YeHChXj33Xfdi/j5gtaO5cCBA4iKisKiRYtQWFiIxMRELFmyRFDq27ubnzEAqK2txZo1a/Dll196O2qL7mYcCxcuxNNPP43ly5cjKCgIa9asuef9trsj+ls5nU7k5eXhpZdewrp161BfX4+VK1e6H6+trcWzzz6LpKSkJguo+Zo7jSUjIwP79u3D2LFj8etf/1pc0DtobhynTp3Cpk2b8Pzzz4uO2GrNjSUkJATvvfceevbsCa1Wi6effhrbt28XHbdFzY3F4XBg//79mDt3LtavX48uXbrgjTfeEB23RXf6XdmwYQPGjx+Pjh19+34AzY2joaEBr776Kj788EPk5eXhsccewy9/+ct73k+7L/qoqCgMGjQIXbp0gUajwaOPPoqjR48CAC5duoTHHnsMffv2bfIvqS9qbizV1dXIy8tzP2/KlCk4efKkwKQta24c33zzDSorKzFjxgw8++yz7u+PL2tuLGVlZVi7dq37eZIkNVkD3Nc0N5bo6Gh069YNKSkpAIDJkye7f4d8VUu/90DjiRgTJ04UmLB1mhvHqVOnoNfr3cvGzJ49G/v377/n/bT7oh81ahTy8/Pd8+9bt25F//794XQ6MW/ePDz66KN49dVXmyyR7IuaG4skSXj55ZdRVlYGAPjmm28wZMgQkVFb1Nw4XnzxRWzcuBFffPEFVq5ciZiYGPzjH/8QnLZlzY0lMDAQv/vd71BSUgJJkrB69eomc8S+prmxDB48GFVVVThx4gQAIDc3F/379xcZ9Y6aGwvQ+I9ufn4+Bg8eLDJiqzQ3jm7dusFoNKKoqAhA4xrz1/8hvhe+fQjSCp06dcJrr72GefPmwWq1ol+/fvjlL3+J3NxcFBQUwOl0YuPGjQCAAQMG+PSRfXNjCQoKwrJly/Dcc89BpVKhV69e+J//+R/RcZvV3Djao5a+J6+99hp+9rOfwW63Y8iQIXjqqadEx21Rc2MJDAzE//3f/2Hx4sWor69HXFwcVqxYITpui1r6GauqqoJOp4Nerxec8s5a+vn67W9/i/nz50OSJHTs2BHLly+/5/1wCQQiIoVr91M3RETUMhY9EZHCseiJiBSORU9EpHAseiIihWPRk9/Zt2/fXa9o+Mwzz+DMmTMtPmfhwoV4//33b/tYTk4Ovv3227vaJ1Fbaffn0RN5w3vvvfeDPn7fvn3o1atXG6UhujssevJLdXV17oWkrFYrXn/9dQwcOBC///3vceDAATidTiQnJ2Px4sUIDQ3FuHHj8Kc//QkpKSlYuXIl1q5di5CQEAwbNgxbtmxBbm4ugMa1xefMmYPLly+jd+/e+MMf/oD169fj+PHjWLFiBTQajc9fQUvKw6kb8ktGoxE//vGP8cUXX2DOnDl45513sHLlSmg0Gqxbtw4bNmxATEwMfv/733t83M6dO7Fu3TqsXbsW69atcy8je11FRQU++OADbNy4ERUVFdi0aROysrIwYMAAvPLKKyx5EoJH9OSXunTp4l4qOSkpCZ999hm2bduG2tpa7N69GwBgt9ubrH64fft2PPLII+77AWRlZWHv3r3ux8ePH4+goCAAQO/evVFVVeWN4RC1iEVPfkmn07n/rlKpIEkSXC4XFi1ahIyMDACAxWJpcjMRrVaLm1cN0Wg0TR6/9fMSicapG6JrRo0ahdWrV8Nms8HlcmHJkiV48803PZ6TkZGBTZs2oba2FgA8lipuiUajgcPhaPPMRK3Boie65vnnn0d8fDwyMzMxceJESJLkcdciAEhLS8OsWbMwe/ZsTJ8+HbW1te6pmpaMGzcOb775JtavXy9XfKJmcfVKortw7NgxfPfdd3jyyScBAB988AGOHDmCP/7xj2KDEbWARU90F8xmMxYtWoSioiKoVCp06tQJy5Ytc9+EnsgXseiJiBSOc/RERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoX7f30MkSV8UbnqAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# draw limits of confidence intervall\n",
"plt = sns.histplot(x=\"height\", data=df, bins=5);\n",
"# draw a vertical line to mark the mean \n",
"plt.axvline(165, 0, 1, linewidth=3, color='b');\n",
"# add text\n",
"plt.text(165.1, 0.1,'Mean = 165', rotation=90, color='b');\n",
"# draw a vertical line to mark the lower limit of the confidence intervall\n",
"plt.axvline(164.348388, 0, 1, linewidth=3, color='r');\n",
"# add text\n",
"plt.text(164, 0.15,'Lower limit = 164.34 ', rotation=90, color='r');\n",
"# draw a vertical line to mark the upper limit of the confidence intervall\n",
"plt.axvline(165.651612, 0, 1, linewidth=3, color='r');\n",
"plt.text(165.8, 0.15,'Upper limit = 165.65', rotation=90, color='r');"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Model 2: Linear Regression"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Confidence intervall for parameter $b_1$ (height_parents)\n",
"\n",
"**lower boundary of confidence intervall** = $b_1 - (1.96 \\times SE(b_1))$\n",
"\n",
"**upper boundary of confidence intervall** = $b_1 + (1.96 \\times SE(b_1))$"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"text/html": [
"
"
],
"text/plain": [
" mean mean_se mean_ci_lower mean_ci_upper obs_ci_lower obs_ci_upper\n",
"0 167.4 0.29 166.79 168.01 165.94 168.85"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Make a prediction for height when parents average height is 168 cm\n",
"to_predict = pd.DataFrame({'height_parents':[168]})\n",
"results = lm.get_prediction(to_predict)\n",
"round(results.summary_frame(alpha=0.05),2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"- The predicted height for an average parents height of 168 cm is 167.4.\n",
"- For 95% the confidence interval is [166.79, 168.01] and the prediction interval is [165.94, 168.85]"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"How much will the outcome vary from our prediction? We use **prediction intervals** (obs_ci_lower and obs_ci_uppper) to answer this question. Prediction intervals are always wider than confidence intervals, because they incorporate both the error in the estimate for f(X) (the reducible error) and the uncertainty as to how much an individual point will differ from the population regression (the irreducible error). \n",
"\n",
"We interpret this to mean that 95% of intervals of this form will contain the true value of Y for parents with this average height. Note that both intervals are centered at 167.4 cm, but that the **prediction interval** is substantially wider than the confidence interval, reflecting the increased uncertainty about the individual height for given parents height in comparison to the average height of many parents. "
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAHsCAYAAACAI1gbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABlnElEQVR4nO3daXhUVbr28btSmYGEIYwGaUUFRAY52E60E040oCItghERVCYNg+c0cASbPvRBPHbry6RAUBEFFYSoILaiYVBwgm4ERZQWhDZKAAlkrgyV/X5YhgIETEhV7V1V/991eSkLqP2kxHhn5VnPclmWZQkAAACAI0TZXQAAAAAAHwI6AAAA4CAEdAAAAMBBCOgAAACAgxDQAQAAAAcJ+YBeUVGh7OxsVVRU2F0KAAAAUGshH9BzcnLUvXt35eTk2F0KAAAAUGshH9ABAACAcEJABwAAAByEgA4AAAA4CAEdAAAAcBACOgAAAOAgBHQAAADAQQjoAAAAgIMQ0AEAAAAHIaADAAAADkJABwAAAByEgA4AAAA4CAEdAAAAcBACOgAAAOAgBHQAAADAQQjoAAAAgIMQ0AEAAAAHIaADAAAADhLQgF5YWKhevXopOztbkrRlyxb169dPPXv21MMPP6yysjJJ0vbt29W3b1/dcsstGjZsmPLz8wNZFgAAAOBYAQvoW7du1YABA7Rnzx5JJqynp6drypQpWrVqlSRp2bJlkqSpU6dq1KhRWrFihc455xw999xzgSoLAAAAEW7zjv16ZM5G3Tf1PT0yZ6M279hvd0nHCVhAX7p0qSZPnqwmTZpIkjZu3KjOnTurbdu2kqRJkybphhtukCRVVlaqqKhIklRSUqL4+PhAlQUAAIAItnnHfmVkblNFaZnOS4nX4fwSzcvc5qiQHh2oF546depxP967d68SExM1duxY7d69W126dNGECRMkSRMmTNCQIUP02GOPKSEhQUuXLj3pa+bn5/+i/SUnJycwHwAAAADCzqqN3+nsulKdolxZscmKj42WRxXKXPeturZrand5kgIY0E/k9Xq1YcMGLVmyRC1atNDEiROVkZGhBx54QBMnTtQLL7ygjh07asGCBRo/frwyMjJ+8RoLFy7U7Nmzg1UyAAAAwonHo+j9++T2FKmkwlJ8/WRJUlyMWwdyi20uzidoAT0lJUWdOnVSy5YtJUk9evTQokWLtHPnTsXFxaljx46SpDvvvFMzZsw46WsMGjRIffr0OW4tJydHaWlpgS0eAAAAocvrlQ4flo4cUXy5R/klFYqLcR/96dJyr5o0TLSxwOMFbcxit27dtH37du3bt0+StHbtWrVv316tWrVSTk6Odu/eLUnKyspShw4dTvoaSUlJSk1NPe6vZs2aBetDAAAAQKjJz5e+/17KzZUqK3Xjpa3k9VoqLffKsix5yipUUWHp9mvOs7vSo4K2g968eXNNmTJFw4cPV2lpqdq1a6fx48crISFB06ZN05gxY2RZlho1aqTHHnssWGUBAAAgHHk80k8/SSUlkmUdXe7QOkW6UVr96V6VlFaqQVJd3X7NeY7pP5ckl2UdU3EIys7OVvfu3ZWVlaXU1FS7ywEAAICdvF6zW56XJ1VWnv7XNm4sNWgQnLpqIGg76AAAAEDAWJZpZzl0SKqosLuaWiGgAwAAILSdop0lVBHQAQAAEJoqKnztLGEQzKsQ0AEAABBaLMuE8tzckG9nORkCOgAAAEJHSYmvnSVMEdABAADgfBUV5gBofn5YtbOcDAEdAAAAzmVZ0pEjpp3F67W7mqAgoAMAAMCZiotNO4vHY3clQUVABwAAgLOUl5t2loKCsG9nORkCOgAAAJzBsqTDh81fEdLOcjIEdAAAANivqMjsmkdYO8vJENABAABgn7IyE8wLCyOyneVkCOgAAAAIvspKM50lwttZToaADgAAgOAqLDS75qWldlfiSAR0AAAABEdpqQnmRUW0s5wGAR0AAACBVVlpWlmOHKGdpRoI6AAAAAicggKza15WZnclIYOADgAAAP+jneWMEdABAADgP16vbzpLZaXd1YQkAjoAAAD8Iz9fys2lnaWWCOgAAACoHY/HtLMUF9PO4gcEdAAAAJwZr9c3nYV2Fr8hoAMAAKDmaGcJGAI6AAAAqs/jkX76SSopoZ0lQAjoAAAA+HVer9kxz8ujnSXACOgAAAA4vfx8s2teUWF3JRGBgA4AAICTo53FFgR0AAAAHI/pLLYioAMAAMCnsNDsmofrdBbLkt57T3ruOSklRXr3XSkqyu6qjkNABwAAgOkvP3TI9JuHYzuLZUnr10szZ0rbt5u15GTzhUh8vL21nYCADgAAEOnC+RCoZUmffCJNny59/rlv/dprpb/8xXHhXCKgAwAARK6yMhPMi4rCc9d882Zpxgzps898a1deKY0eLXXqJDVubF9tp0FABwAAiDSWZQ6BHj5sDoSGm23bTDDfsMG31rWrNGaMdMkltpVVXQR0AACASFJcbHbNPR67K/G/r782PeZZWb61jh3NjvmVV0oul3211QABHQAAIBJUVPhuAg23dpZdu6RZs6S//9231q6dCebXXBMywbwKAR0AACDcFRRIBw+G3yHQf/9bevppacUK37z2886T0tOlG2903PjE6iKgAwAAhKvyctPOUlgYXrvmP/4ozZkjZWb6vug4+2wTzHv2lNxue+urJQI6AABAOArH0YkHDkgZGdKrr5ovPiSpRQvpwQel226TosMj2obHRwEAAAAjHHfNc3Ol+fOlxYul0lKz1rixNGKEdMcdUmysvfX5GQEdAAAgXITbrnlenvT889KLL5rpM5LUoIE0bJg0YIAjLxnyBwI6AABAqCsrM4dAi4vDY9e8sFBauFBasMAccJWkpCTpvvukgQOlOnXsrS/ACOgAAAChyrKkI0dMC0g4XDhUUmLaWObPNx+XZML4vfeav5KSbCwueAjoAAAAocjjMbvmJSV2V1J7ZWXSkiXS3LmmRUcy7St33212zRs2tLe+ICOgAwAAhJLKSrNjfuSIb/Z3qCovN6MS58yR9u0zazExpr986FBzEDQCEdABAABCRWGh2WEuK7O7ktrxeqWVK6XZs6Xvvzdr0dHSH/5gJrM0a2ZvfTYjoAMAADhduIxOrKyU3nlHmjVL2r3brEVFmRnmI0dKLVvaWp5TENABAACcyrLMqMFDh0L7EKhlSVlZ0syZ0jffmDWXS/r976WHHpLOPdfe+hyGgA4AAOBE4XAI1LKkDz+UZsyQvvzSt37jjVJ6unTBBfbV5mABDeiFhYXq37+/5s6dq9TUVG3ZskXTpk1TUVGR2rRpo8cff1y7du3ShAkTjv6e3NxcJScn66233gpkaQAAAM5UWSkdPmz+CuVDoJ9+Kk2fLv3zn761q6+WRo2SLrrItrJCQcAC+tatWzVp0iTt2bNHkgnr6enpevbZZ9W2bVs9/PDDWrZsme666y69+eabkqSSkhLdcccd+vOf/xyosgAAwBnavGO/Mtd9q/25xWraMFG3X3OeurZrandZ4aWkRF9t+lrvrPtGPx0uUaPkeN14aSt1aJ1id2XVt2WLCeaffOJbu/xyafRo6eKLbSsrlAQsoC9dulSTJ0/WuHHjJEkbN25U586d1bZtW0nSpEmT5D2hl2revHm65JJL1LVr10CVBQAAzsDmHfs1L3OboqNdqpcQrcP5JZqXuU26vSMh3R9+Hp24/Z/f6pW/75Db7VKdeLfyC0u1ZPU30o1yfkjfvt20sqxf71vr0kUaM0a69FLbygpFAQvoU6dOPe7He/fuVWJiosaOHavdu3erS5cux7W2FBQUaOnSpVq5cuUpXzM/P1/5+fnHreXk5Pi3cAAA8AuZ675VdLRL8bEmOsTHRsujCmWu+5aAXltFRWZCS2mp3vnoO7ndLsXFuCVJcTFulcqr1Z/udW5A/+YbM5Xlvfd8axddZIJ5t27mMChqJGiHRL1erzZs2KAlS5aoRYsWmjhxojIyMpSeni5JWrFiha6//no1atTolK+xcOFCzZ49O1glAwCAn+3PLVa9hONjQ1yMWwdyi22qKAx4vWY6S17e0dGJh/I8qhPvPu6XxcZEKTfPY0eFp/fdd2aO+apVvtGPF1xgWlm6dyeY10LQAnpKSoo6deqklj/Pt+zRo4cWLVp09Offf/99DRs27LSvMWjQIPXp0+e4tZycHKWlpfm/YAAAcFTThok6nF9ydAddkkrLvWrSMNHGqkLYKS4capQcr/zC0qM76JJUVl6phsnxwa7w1L7/XnrmGemNN3yHWM85xxz+vPlmM9cctRK0gN6tWzfNmjVL+/btU/PmzbV27Vq1b99ekmRZlrZv366Lf+XgQFJSkpKSkoJRLgAAOMbt15yneZnb5FGFabso96qiwtLt15xnd2mhpaLCBPOCgpNeOHTjpa20ZPU3KpVXsTFRKiuvlNdr6cZLW9lQ7AlycqQ5c6Rly8zHIUmpqWZcYq9e5iZQ+EXQ3snmzZtrypQpGj58uEpLS9WuXTuNHz9ekhmtGBMTo7i4uGCVAwAAaqBru6bS7R2Vue5bHcgtVhOmuNRcQYGZa14Vbk+iQ+sU6UZp9ad7lZvnUUMnTHH56Sdp3jzp1Vd9O/7NmkkPPij16SPFxNhXW5hyWVYo3xcrZWdnq3v37srKylJqaqrd5QAAAByvrMz0mhcWnnTX3LEOH5aee05atMh3WVJKijRsmHTnnVI4bKw2biw1aGB3Fb/A9yIAAAACwbLMAdBDh8yB0FBRUCAtWCC98IKZMCNJ9etLQ4dKd90lJSTYWV1EIKADAAD4m8djWkOKQ2jKTVGR9NJL0vPPmy8sJKlePWnIEOmee6S6de2tL4IQ0AEAAPzFsqTcXNMeUjXhxOk8HumVV6SMDFO7JCUmmlA+ZIiUnGxvfRGIgA4AAOAPxcVm19zjwJnlJ1NWJr32mpnMcvCgWYuLk9LSpAcekBo2tLe+CEZABwAAqI2KCrPzfMyFQ45WXm5mmD/zjPTjj2YtJkbq39/0mTdpYmt5IKADAACcufx8s2t+mtGJjuH1mls/Z8+W9u41a263dPvt0siRUosW9taHowjoAAAANVVa6jsE6vRd88pK6b33pJkzpW+/NWtRUdItt5hZ5mefbW99+AUCOgAAQHVVVJgDoHl5zj8EalnSunXSjBnSjh2+9R49zO2frVvbVhpOj4AOAADwayxLOnLEhHOnt7NYlvTRRyaYb93qW7/2WmnMGKltW9tKQ/UQ0AEAAE6noMAcAi0ttbuSX7dpkwnmmzb51rp1k0aPljp2tK8u1AgBHQAA4GQ8HnMLaCj0mW/daoL5xo2+td/+1gTzrl3tqwtnhIAOAABwrPJys2Oen+/8YL5jhwnma9f61jp3Nq0sl10muVx2VYZaIKADAABIZgzhkSPmL6/X7mpO79tvzVSWd9/1rV14odkxv/pqgnmII6ADAIDIZllmt/zwYXO7ppPt3WvmmK9c6dvdP/98adQo6YYbCObV4XJJ8fFS/fpSnTp2V3NSBHQAABC5PB7fPHMn++EHc/Pn66/7dvdbtTLjEn//e3PhEE4vKkqqV09KSpISEuyu5rQI6AAAIPJUVpod88OHnT3PfP9+ae5c6bXXTG+8JJ11lrn587bbpGii3Gm5XFJMjAnlSUkh836FRpUAAAD+UlRkds2dPDbx0CFp/nzp5Zd9dTZpIo0YIf3hD1JsrL31OZ3LJSUmmlBet27Itf4Q0AEAQGSoqDDB18nTWY4ckRYskF580dd206iRNHSo1L+/6Z3GqbndJpTXqxfS7xUBHQAAhLeqQ6CHDjn3FtDCQumFF0w4Lyw0a8nJ0n33SXff7djDjI7gcpnvKCQnm2AeBv34BHQAABC+nH4ItLhYWrxYevZZs3sumZaMwYOlQYNM4MTJRUWZL1ySksLuCxgCOgAACD9erzkAeuSIMw+BlpZKr74qZWSYLyAkM1lk4EBpyBCpQQN763Mql8sc9KxqYwnTXnwCOgAACC8FBaadxYkzzcvKpOXLpTlzzIQWyYTMu+6SHnhASkmxtz6ncrvNLnnduubvIXbos6YI6AAAIDx4PCaYFxc77xBoRYW0YoW5ZOiHH8xaTIx0xx3S8OFS06b21udEUVHmuwr16plQHga95dVFQAcAAKGtosK0s+TlOa+dpbJSevttadYsac8es+Z2mxnmI0dKqal2Vuc8LpcUF2d2yuvWDdsWll9DQAcAAKHJskyP+eHDzpvOYlnS++9LM2dKO3eaNZdL6tVLeugh6Te/sbU8R3G5fOMR69YN6fGI/kJABwAAoae42Byu9HjsruR4liV98IE0Y4a0fbtv/aabpPR06fzz7avNaSKsr7wmCOgAACB0OPmyoY8/NsF8yxbf2rXXSqNGSRdeaF9dThLBfeU1QUAHAAChIT/f7Jo7rZ3lH/+Qpk+XPvvMt3bFFdLo0VLnznZV5RxVFwnVqxfRfeU1QUAHAADOVlrqu2zISbvmX3xhdsw//NC39h//YYL5pZfaV5cT0FdeKwR0AADgTJWVvsuGvF67q/H5+mtz+DMry7fWoYMJ5t26RXYvNX3lfkFABwAAzlNQIOXmmt1zp9i1y8wxf/tt31qbNiaYX3dd5IZRl8vXV163Ln3lfkBABwAAzlFaag6BFhU5p53l++9NMF+xwjdn/ZxzzOHPm282Bx8jDS0sAUVABwAA9nNiO8u+fdKcOdLy5b6DqS1bmjnmvXtH5k5xVJSUmOjbLY/U7xoEGAEdAADYq6DA7JqXldldiXHggJSRIb36qlRebtaaN5cefNDcABoTY2t5toiOlpKTTTBnCkvAEdABAIA9PB4znaWkxBntLLm50rPPSosX+y5AatxYGj5c6tcv8oJpVW95crI58BmJrTw2IaADAIDgqqgwYTgvzxnBPD9fev55aeFCM8pRkho0kIYOlQYMMCE1krjdZqc8KYnecpsQ0AEAQHBYlukxz811Rp95YaH00ksmnOfnm7WkJGnIEGngQNNjHSmqLhOqamOJxP56ByGgAwCAwCsp8bWz2K2kRHrlFdNnfviwWUtMlO69Vxo82IT0SBEVZdpXkpLMe8ChT0cgoAMAgMDxes2O+ZEj9rezlJVJS5dKc+dKBw+atfh4KS1Nuv9+qWFDe+sLpqoRiUlJUlyc3dXgBAR0AAAQGE6ZzlJeLr3xhvT002Z0omQmsfTvLw0bZg6CRorYWF8wjyYGOhX/ZgAAgH+VlZlgXlho76651yutXGkuGfr+e7MWHS317SuNGGFGJ0YCl8t8p6Cqv5w2FscjoAMAAP+orDStLIcP23sItLJSeucdE8x37TJrUVHSLbeYS4ZatrSvtmByuUx/edWYRIQMAjoAAKi9wkKza15aal8NliWtWSPNmCF9841vvWdPc8lQ69b21RZM9JeHPAI6AAA4c6Wl5hCone0sliVt2GCC+Rdf+NZvuEFKT5fatLGnrmCLiTG75fSXhzz+7QEAgJrzek07y5Ej9razfPaZNH269I9/+NZ+9ztp9GipQwfbygoal8vsktevb+a2c9tnWCCgAwCA6rMsM50lN9fe6Syff252zD/6yLd22WUmmHfpYltZQUN/eVgjoAMAgOopLjZ95nZeNrR9uzRzprRunW/t4otNML/8ctvKChq320xiSUoyk1kQlgjoAADg9MrKzI55QYF9feY7d0qzZkmrV/vW2rc3wfyqq8J/dGB0tNktT06mvzwCBPTfcGFhofr376+5c+cqNTVVW7Zs0bRp01RUVKQ2bdro8ccfV2xsrHbv3q3JkycrLy9PjRs31lNPPaXk5ORAlgYAAH7NCX3mTy7erJ3fHzn60xe0rK//TOsa2Bq++86MS1y1yvfFwQUXmGDevXt4B/Oq/vKq+eX0l0eMgP2b3rp1qwYMGKA9e/ZIMmE9PT1dU6ZM0apVqyRJy5Ytk2VZGjFihB544AGtWLFC7dq1U0ZGRqDKAgAAv8ayTCjfu9e0tJwknEvSzu+P6MnFmwNTQ3a29N//bUYkvvWWqek3v5Geekp6803p+uvDN5y7XObAZ4sW0tlnm4BOOI8oAdtBX7p0qSZPnqxx48ZJkjZu3KjOnTurbdu2kqRJkybJ6/Vq+/btSkxM1FVXXSVJGj58uPLz80/6mvn5+b/4uZycnEB9CAAARJ6CAnPRkMdz3PKJ4fzX1s/Y/v3SnDnSsmVSeblZS001Fwz17h3e7R1RUb755fSXR7SA/SmfOnXqcT/eu3evEhMTNXbsWO3evVtdunTRhAkTtGnTJqWkpOiRRx7Rjh07dO655+rRRx896WsuXLhQs2fPDlTJAABELo/H7JYXF9vTZ/7TT1JGhvTKK77pME2bSiNGSH37SrGxwa8pWKr6y5OSzCxzRLygfRnq9Xq1YcMGLVmyRC1atNDEiROVkZGhVq1a6bPPPtOiRYvUoUMHTZ8+XY8//rgef/zxX7zGoEGD1KdPn+PWcnJylJaWFqwPAwCA8OL1mh3zI0ekysrgP//IEem556SXXvJNh0lJkYYNk+68M3xvwnS5zC55cjLzy/ELQQvoKSkp6tSpk1q2bClJ6tGjhxYtWqSuXbuqVatW6vDzZQK9evXSqFGjTvoaSUlJSkpKClbJAACEt/z8as8zv6Bl/ZO2s1zQsv6ZPbugQHrhBfNXYaFZq19fuv9+KS1NSkw8s9d1uqgoE8iTksL3Y0StBe3LtW7dumn79u3at2+fJGnt2rVq3769Lr74YuXm5urrr7+WJK1Zs0bt27cPVlkAAESe0lLpxx9Nv3c1Lxv6z7SuvwjjZzTFpbjYtLJ0726msxQWmsCani5lZUkPPBCewTU6WmrUyBz6bNYsPD9G+E3QdtCbN2+uKVOmaPjw4SotLVW7du00fvx4xcfH6+mnn9akSZNUUlKiZs2a6YknnghWWQAARI4TxibWVK1GKno80quvmnB+6JBZS0yU7rlHGjzY7J6Hm6goKSHB7JbXrRu+U2fgdy7LsuvGAf/Izs5W9+7dlZWVpdTUVLvLAQDAeSxLyssz7SwVFcF9dlmZmcgyZ4504IBZi4szbSwPPCA1bBjceoKh6tBn3brh20OPgArjWUUAAEAFBSaYl5YG97kVFWZe+dNPSz/8YNZiYqR+/cwB0KZNg1tPoFUd+qxfn91y1BoBHQCAcFRUZIK5xxPcsYler/T226a//OfLCuV2S7ffbkYmnnVW8GoJBg59IgAI6AAAhJPyctPjXVAQ3GBuWdJ770kzZ0r/+pdZc7mkW26RHnxQatUqeLUEg9vtm10ezjPaYQsCOgAA4aCqz/zQoTM6AFqr565fL82YIX31lW/95pvNZJbzzgteLYHmcpkwnpws1atnQjoQAAR0AABCncdjbuIsLg7eMy1L+vhjE8w//9y3fu210ujRUrt2wasl0KKiTPtKcrJUp47d1SACENABAAhVlZWmzzzYt4Bu3myC+Wef+da6dTPBvGPH4NURaG63aWFJSmIaC4KKgA4AQCgqLDTtLMGczrJtmwnmGzb41rp2lcaMkS65JHh1BJLLZcJ4UhJtLLANAR0AgFBixyHQr782wXzNGt9ap05mx/yKK8JjpGBUlGlfSUqijQW2I6ADABAKLEvKzze95sE6BLprlzRrlvT3v/vW2rUzwfyaa8IjmFddKlSvHtNY4BgEdAAAnC7Yh0D//W8zx3zlSl9v+3nnmaksN95odptDWVUbS1UwD/WPB2GHgA4AgFN5vdLhw8E7BPrjj9KcOdLy5b5d+latzBzzXr1Cvx+baSwIEQR0AACcqKDA9JqXlQX+WQcOSPPmSUuWmB53SWrRwgTz224zbSChLDrad+iTaSwIASH+XxwAAGGmtNQE86KiwB8Czc2V5s+XFi/2TYNp0kQaPly6447Q7sl2uaSEBN9uOW0sCCEEdAAAnKCy0tfOEuhDoHl50vPPSy++6Otrb9hQGjZM6t9fio8P7PMDqWq3vG7d0P44ENEI6AAA2C1Y7SyFhdLChdKCBeaZktlhvu8+6e67Q7cvOyrK7JZXjUhktxwhjoAOAIBdSkpMMC8pCWw7S0mJaWOZP9/s0EsmyA4eLN17r+nNDjUu1/G75fSWI4wQ0AEACLbyctP/nZ8f2GBeWmoOfs6bZ8Y0SqbtY+BAs2veoEHgnh0oVZNY6tUzwTwcZrEDJyCgAwAQLF6v2cEOdJ95ebmUmSk984yUk2PWYmNNf/nQoVLjxoF7dqDExLBbjohBQAcAINCqbgE9fDiwfeYVFeZyodmzpexssxYdLf3hD9KIEVKzZoF7diAc21vObjkiCAEdAIBAKiw07SweT+CeUVkpvfOONHOm9N13Zi0qyswwHzlSatkycM/2N5fL7JZXtbCwW44IREAHACAQPB5zALS4OHB95pYlZWVJM2ZIO3eaNZdL6tlTeugh6ZxzAvPcQKC3HDiKgA4AgD+Vl5tWlry8wAbzDz80wfzLL33rN94opadLF1wQmOcGQnS0GfVYr15oX4wE+BEBHQAAfwjWRUOffipNny7985++tWuukUaNktq3D9xz/anqls+q3nLmlgPHIaADAFAblmUu/cnNDewB0C1bTDD/5BPf2uWXS6NHSxdfHLjn+pPbbXbKk5K45RM4DQI6AABnqrjY9Jl7PIFrZ9m+3bSyrF/vW+vSRRozRrr00sA805+qDn1WtbFEEz2AX8N/JQAA1FRpqdkxLywMXDDfuVOaNUtavdq3dtFFJph36+b8Q5Qulzn0yYhEoMYI6AAAVFdFhe+iocrKwDzju+/MHPNVq3zh/4ILTCtL9+7OD7putwnl9erRxgKcIQI6AAC/xrLMVJbcXBPSA+H7783Nn2+84Qv/55xjDn/efLOzD1K6XCaMV+2Wu912VwSENAI6AACnU1ho+sxLSwPz+jk50pw50rJlvvCfmmrGJfbq5eyebQ59AgHh4P/qAQCwUaAvGjp4UMrIkF591Tf9pVkzc/Pn7bebg5VO5HKZeeVVhz7ZLQf8joAOAMCxKipMK0t+fmD6zA8flp59Vlq0yHwRIEkpKdLw4VK/fs692j4qSqpTx+yW16ljdzVAWCOgAwAgmTB+5IgJ0IG4aCg/X3rhBfNXUZFZq19fGjpUuusuc3GPE7ndvt1yp37xAIQZAjoAAFUXDQWiz7yoSHrpJen5581BU8mE3SFDpHvuMYcqnSg21nwBQRsLEHQEdABA5PJ4pJ9+kkpK/N9n7vFIr7xi+sxzc81aYqIJ5UOGmF1pp6maxlK1Y+70kY5AmCKgAwAiT3m5r8/c38G8rEx67TUzmeXgQbMWHy+lpUn33y81bOjf5/mD2+3rL09MtLsaIOIR0AEAkcPr9V005O8+8/JyM8P8mWekH380azExUv/+ps+8SRP/Pq+2XC7T916vHrPLAYchoAMAwl9lpen/PnzY/xcNeb3SW2+Z2z///W+zFh1tRiWOGCG1aOHf59WGy2W+aKgK5Rz6BByJgA4ACF+W5TsAWjVr3F8qK6XVq6WZM6Vdu8xaVJR0yy3Sgw9KZ5/t3+fVRlSUaV2pGpFIbzngaAR0AEB4Kiw0wbxq1ri/WJa0dq00Y4b09de+9R49zO2frVv793m1ER1tQjkjEoGQQkAHAISXQN0AalnSxo0mmG/b5lu/7jpp9GipbVv/Pas2XC4TxqsmsURF2V0RgBoioAMAwkNZmekxD8Rklk2bpOnTpc2bfWvduplg3rGjf591pqoOfdav79zZ6gCqhYAOAAhtgZzMsnWr2THfuNG39tvfmmDetat/n3WmoqJMIE9Odu5tpABqhIAOAAhNlmV2yw8d8v9klq++Moc/1671rXXuLI0ZI112mTMOWbrdpr88KYn+ciDMENABAKGnqMgcAC0p8e/r/utf0qxZ0rvv+tYuvNDsmF99tTOCeXS02S1PTjb/DCDs8F82ACB0lJaaYF5Y6N8+8z17zBzzt97yve7550ujRkk33OCMYB4ba/rL69XjUiEgzBHQAQDOV1Hh6zOvrPTf6/7wg7n58/XXff3rv/mNGZfYo4f9QdjlkuLjfQc/nfCFAoCAI6ADAJyrstKE8sOH/XsAdP9+ae5c6bXXpPJys3bWWdLIkdJtt9nfOuJymQuF6tc3FwwBiCgB/QxUWFio/v37a+7cuUpNTdWWLVs0bdo0FRUVqU2bNnr88ccVGxur2bNna/ny5UpKSpIk9evXT2lpaYEsDQDgZJYl5eWZcO7PG0APHZLmz5deftm0y0hSkyYmmPfta9pI7BQV5Tv4GR9vby0AbBOwgL5161ZNmjRJe/bskWTCenp6up599lm1bdtWDz/8sJYtW6a77rpLX375pZ566ildfPHFgSoHABAqCgpMn3lVgK6mJxdv1s7vjxz98QUt6+s/034ehXjkiPT889JLL5kLjCSpUSNp6FCpf3/7w3BMjC+Yx8TYW8tpbN6xX5nrvtX+3GI1bZio2685T13bNbW7rLDD+4yABfSlS5dq8uTJGjdunCRp48aN6ty5s9r+fNPapEmT5P3525Vffvml5s2bpx9++EGXXHKJxo8frzhGRgFAZCkuNjvcHk+ND4CeGM4laef3RzTr+Q+UXrxNWrDAHCyVzPST+++X0tJMG4ldju0vr1PH8Td+bt6xX/Mytyk62qV6CdE6nF+ieZnbpNs7Eh79iPcZUgAD+tSpU4/78d69e5WYmKixY8dq9+7d6tKliyZMmKCioiK1a9dOf/zjH9WqVStNmDBBzzzzjMaOHfuL18zPz1d+fv5xazk5OYH6EAAAweDxmB3zoqIznsxyYjiPLS/VtTvW68Yv35dKi8xi3brS4MHSoEFmEopdQvRiocx13yo62qX4WBMd4mOj5VGFMtd9S3D0I95nSEE8JOr1erVhwwYtWbJELVq00MSJE5WRkaH09HTNnz//6K8bMmSIHnnkkZMG9IULF2r27NnBKhkAEEjl5ebwZ36+3yazRFeU6+pvPtTN21YryVNgFhMSpHvukYYMMbvVdnC5fG0s9eo5uo3lVPbnFqtewvGxIS7GrQO5xTZVFJ54nyEFMaCnpKSoU6dOatmypSSpR48eWrRokX788Ud99NFH+sMf/iBJsixL0ac4PT9o0CD16dPnuLWcnBwOlAJAKPF6fSMT/TSZxe2t0JX/+li/3/qOGhQfkSSVu6O1vs3vdP2zU02/uR2ioswUlnr1Qn5MYtOGiTqcX3J0Z1eSSsu9atKQKTP+xPsMKYgBvVu3bpo1a5b27dun5s2ba+3atWrfvr3i4+P117/+VZdeeqlSU1O1ePFi3XDDDSd9jaSkpKOTXgAAIcayzG754cP+m8xSUSG9+aamrfh/Sj5y0CxFubXhgiv19443qUnb3+h6O8K5221aWOrVk8LkTNXt15yneZnb5FGF4mLcKi33qqLC0u3XnGd3aWGF9xlSEAN68+bNNWXKFA0fPlylpaVq166dxo8fr4SEBE2ZMkUjRoxQeXm5unTposGDBwerLABAMBQWmj5zj8c/r1dZKb39tjRrlrRnj5IlVbqi9NF5l+ntTjfrUL1Gx09xCQaXy4Tx5GSzW273JUd+1rVdU+n2jspc960O5BarCdNFAoL3GZLksix/3pUcfNnZ2erevbuysrKUmppqdzkAgGN5PGYyS3HxGR8APY5lSe+/L82cKe3cadZcLqlXL+mhh8wtoMEWFWWmsCQl2TsVBkDY4CZRAID/lZX5DoD6K5h/8IE0Y4a0fbtv/aabpPR06fzza/+MmgrDNhYAzkBABwD4TwAOgOrjj00w37LFt3bttdLo0VK7dv55RnW5XOa20apgHmZtLACcgYAOAKg9y5Ly8kyfeUWFf17zH/+Qpk+XPvvMt3bllSaYd+rkn2dUl8tlprEkJ5s2lhCexgLA+QjoAIDaKSgwwby01D+v98UXZsf8ww99a127mmD+29/65xnVFR3tm11OGwuAICGgAwDOTFGRbzKLP/rMv/7aHP7MyvKtdewojRkjXXFF8Hatq3bLqw59RkUF57kA8DMCOgCgZjweE8yLivwTzHftkmbPNmMTq7Rta3bMr702eMHc7TahPCmJ3XIAtiKgAwCqp7TUTGYpKPBPMP/+exPMV6wwc80lqXVrM5XlppuCt3Ptdkv165v+8lPcZA0AwcRnIgDA6Xk8ZipLYaEvSNfGvn3SnDnS8uW+A6Vnn23mmPfqFbzJKFXTWJKSmMYCwFEI6ACAk/N4zI55YaF/dswPHJAyMqRXX5XKy81a8+bSgw9Kt90mxcTU/hm/pmpMYv365uAn/eUAHIiADgA4XkmJCeb+6jHPzZWefVZavNiEfklq3FgaPlzq188E5kA7dkxi3bqBfx4A1AIBHQBg+PvwZ36+9Pzz0sKFUnGxWWvQQBo6VBowQEpIqP0zfo3bbXbKk5Kk+PjAPw8A/KBaAf3999/X9ddff9zaG2+8odtuuy0QNQEAgsnfrSyFhdJLL5lwnp9v1pKSpPvuk+6+O/A72C6XaZepmsjCwU8AIea0n7XWrFmjiooKPfHEE6o85mBQRUWFZs2aRUAHgFDm76ksJSXSyy9L8+eb15XMHPF77zV/JSXV/hmnExV1/PxybvsEEKJOG9B37NihTz75RIcOHdJLL73k+03R0br33nsDXRsAIBDKynzB3B9TWcrKpKVLpblzpYMHzVp8vNktv+8+qWHD2j/jdLjtE0CYcVnWr2+bLF68WGlpacGop8ays7PVvXt3ZWVlKTU11e5yAMC5ysvNuMS8PP8E8/Jy6fXXpWeeMaMTJdNaMmCA6TNv3Lj2zzgdxiQCCFPVaszr27evVqxYodzcXB2b5wcPHhywwgAAflJRYUL5kSOS11v71/N6pZUrzSVD339v1qKjpb59pREjzOjEQIqLY0wigLBWrYD+n//5n9q3b58uuOACuejpA4DQ4PWaYH74sH+CeWWl9M470qxZ0u7dZi0qSrr1VjPLvGXL2j/jVFwuE8wbNDCHTPl/EYAwVq2AvnPnTr377ruKYqcCAJzPsnzBvOpCoNq+XlaWNHOm9M03vvWePc3tn+eeW/tnnIrLZcYx1q/P/HIAEaNaAb1Ro0aqqKhQbDAukwAAnBnLMgc/Dx82E1r88XobNkjTp0tffulbv+EGKT1datOm9s84FZfLBPLkZDOZBQAiyGkD+oIFCyRJjRs31sCBA9W9e3fFHHMVMz3oAOAQVcG86qbO2vrsMxPM//EP39rvfieNHi116OCfZ5xMVJTpLU9O5mIhABHrtAF9586dkqS6deuqbt26+u6774JSFACgmoqLze2fJSX+mWX++efSjBnSRx/51i67zATzLl1q//qnUhXM69dnVCKAiHfagD5t2rRg1QEAqAmPxwTzoiL/BPPt202P+bp1vrWLLzbB/PLLa//6pxIVZVpZGjQgmAPAz6rVg37dddcdN73F5XIpISFB559/viZMmKAmTZoErEAAwDH8ffvnzp1mKsvq1b619u1NML/qqsBNS3G5fDvmtLIAwHGqFdCvv/56FRUVKS0tTVFRUVq2bJmKiorUpk0b/elPf9LcuXMDXScARLayMjPHPD/fP5cMffedmWO+apUv6F9wgQnm3bsHNpjXqWNuFyWYA8BJVSugb968WZmZmUd/PGnSJP3hD3/QtGnTtHz58oAVBwARz9/BPDtbevpp6Y03fK93zjlmKkuPHoG7+MflMoG8YUMT0AEAp1StgF5UVKTCwkLV/XkGbWFhoTz+mhQAAPil8nITzPPy/BPMc3KkOXOk5ct9s9FTU80c8969zU2ggRIXZ4I5FwwBQLVU6zNy37591a9fP918882yLEurV6/WHXfcoZdeeknnBvKCCgCINGVlJpTn5/vn9s+ffpIyMqRXXjGvLUnNmkkjR0p9+kiBvN8iOtoc/kxODtzOPACEoWoF9KFDh+rCCy/U+vXrFR0drUcffVSXXXaZvvzyS/Xp0yfQNQJA+PN4TDAvKPDPjvnhw9Jzz0mLFpkRjJKUkiINGybdeWdgJ6a43ebwZ/365p8BADVy2oC+a9cutW7dWtu3b1eDBg102223Hf257du366KLLgp0fQAQ3oqLTSuLv8YlFhRIL7wgLVhgXlMyQfmBB6S0NCkhofbPOJWoKLNb3qBBYFtmACDMnfYz6BNPPKF58+YpPT39Fz/ncrmUlZUVsMIAIGxZllRYaHbM/XXBUHGx2S1/9lnzupIZYzh4sDRokOn/DpSqS4YaNAhsywwARIjTBvR58+ZJktasWROUYgAgrFVWmh3uI0dMP7g/grnHI736qukzP3TIrCUmSvfcIw0ZYna0A4VLhgAgIKo9xeXJJ5/Url27NGPGDD311FMaP3686jAqCwB+XUWF2dXOyzP/7A9lZdKyZWYyy4EDZi0uzrSxPPCAmZoSKARzAAioagX0//3f/1WTJk106NAhxcXFqbCwUH/605/05JNPBro+AAhdVcH8yBH/TGSpes033pCeeUb64QezFhMj9etnDoA2beqf55wMwRwAgqJaAX3Hjh2aNm2a1q9fr4SEBP3tb39Tr169Al0bAIQmr9eEcn8Gc69Xevttc/vnnj1mze2Wbr9dGjFCOuss/zznZKp6zOvXJ5gDQBBUK6BHnTC/1uv1/mINACKe1+vbMfdXK0tlpfTee9LMmdK335o1l0u65RbpwQelVq3885yTIZgDgC2qFdAvueQS/fWvf5XH49GHH36oRYsW6be//W2gawOA0FBZaS4Wys31XzC3LGndOhPMv/rKt37TTdKoUdJ55/nnOScTFSUlJZlgzlQWAAi6agX0//qv/1JGRobq1aun6dOnq1u3bnrwwQcDXRsAOJtlmWB++LDvlk5/vOZHH0kzZkhbt/rWr71WGj1aatfOP885mao55snJBHMAsFG1Avq2bdu0adMmeTweSWbs4po1a7Ry5cqAFgcAjlUVzEtL/feamzdL06dLmzb51rp1M8G8Y0f/PedEbrcvmMfEBO45AIBqqVZAf/TRR9WvXz+1a9dOLpcr0DUBgHMVFppg7vH4Z465JG3bZnbMN2zwrV1yiTRmjNS1q3+ecTJut2ljSU7m5k8AcJBqfUaOjY3VvffeG+BSAMDBCgvN4U9/3fwpSV9/bYL5sZfBdepkdsyvuMIcBg0EgjkAOFq1PjOfe+65+uKLL9ShQ4dA1wMAzlJUZHbM/RnMd+2SZs2S/v5331q7dmbH/OqrAxfMq3rM69enlQUAHOy0Ab13796SzE2iAwYMUMuWLRV9zG4LPegAwlZxsZnK4s9g/u9/mznmK1eayS+SmcYyapR0ww0mQAcCU1kAIKScNqA/+uijwaoDAJyhuNjsmBcX+y+Y//ijufkzM9N3cVGrVtJDD0k9e5qWk0BwuczNnw0bMsccAELIaQM6s84BRIxABPMDB6R586QlS6TycrPWooW5YOi22wLb/x0fLzVqJNWpE7hnAAACgtNBACJbIIJ5bq40f760eLFvDGOTJtLw4dIddwS2zSQ62uyYJycHrpcdABBQBHQAkSkQU1ny8qTnn5defNEEfsmE5aFDpQEDzK52oERFmR7zBg0C1zIDAAgKAjqAyGFZvmDuzznmhYUmlD//vFRQYNaSk6X77pPuvjuwbSb0mQNA2CGgAwh/lZUmOB854t+bP0tKTBvL/PnmtSUTxgcPlu69V6pXz3/POpHL5eszT0wM3HMAAEEX0IBeWFio/v37a+7cuUpNTdWWLVs0bdo0FRUVqU2bNnr88ccVe0wv5rp16zRlyhStOfbSDgABtXnHfmWu+1b7c4vVtGGibr/mPHVt19TusvyjokLKzzetJ1WHNP2htNQc/Jw3T/rpJ7OWkGB2y++7z7SZnOCtDbuUtSlbnrIKxcdGq/slqerVrfWZPT821uyY16tHn/kxwvrPMoCIEqChu9LWrVs1YMAA7dmzR5IJ6+np6ZoyZYpWrVolSVq2bNnRX//TTz/p//7v/wJVDoCT2Lxjv+ZlbtPh/BLVS4jW4fwSzcvcps079ttdWu2UlUkHD0p795oA7a9wXl5ugvmNN0pTp5rXjo2VBg2S3n9f+q//OmU4f3vjHpWWV8gdJZWWV+jtjXv01oZdNXu+2y2lpEhnn23mmhPOjwrbP8sAIlLAAvrSpUs1efJkNWnSRJK0ceNGde7cWW3btpUkTZo0STfccMPRXz9p0iQ99NBDp33N/Px8ZWdnH/dXTk5OoD4EIOxlrvtW0dEuxcdGy+Uyf4+Odilz3bd2l3ZmPB4pJ8dcCHT4sG/meG1VVEivvy7dfLP0pz+ZZ8TEmIOf770nPfKICc6nkLUpW3JJ7qgouVxRckdFSa6f16vD7TY75q1amb8H6kKjEBZ2f5YBRLSAtbhMnTr1uB/v3btXiYmJGjt2rHbv3q0uXbpowoQJkqQXX3xRF154oTp16nTa11y4cKFmz54dqJKBiLM/t1j1Eo7/NBAX49aB3GKbKjpDgRiVKJne9b//XZo1S/ruO7MWFWVmmI8cKbVsWa2X8ZSZnfNjRbnM+mlxA2i1hc2fZQBQEA+Jer1ebdiwQUuWLFGLFi00ceJEZWRk6KabbtLq1av1wgsv/Opu+KBBg9SnT5/j1nJycpSWlhbI0oGw1bRhog7nlyg+1vepoLTcqyYNQ+TQYSBGJUrmtbKypBkzpJ07zZrLZW79fOgh6ZxzavRy8bHRpr3lmI6USkvHve/HqZrM0qBBYEczhpGQ/7MMAMcIWkBPSUlRp06d1PLnHacePXpo0aJFsixLBw8eVN++fVVeXq4DBw7orrvu0ssvv/yL10hKSlJSUlKwSgbC3u3XnKd5mdvkUYXiYtwqLfeqosLS7decZ3dpp1Y1KvHwYXNY09/B/IMPpJkzpS+/9K3feKOUni5dcMEZvWz3S1L19sY98lZWKsplwrkss/4LCQlMZjkDIflnGQBOIWgBvVu3bpo1a5b27dun5s2ba+3atWrfvr1GjRqlUaNGSZKys7N1zz33nDScA/C/ru2aSrd3VOa6b3Ugt1hNnDz5IlCjEqt88ok0fbq0ZYtv7ZprpFGjpPbta/XSVdNaTjvFJTraBHMOf56RkPqzDAC/ImgBvXnz5poyZYqGDx+u0tJStWvXTuPHjw/W4wGcQtd2TZ0dYrxe36jEsjL/v/4//2laWT75xLd2+eXS6NHSxRf77TG9urU++VhFbgD1G8f/WQaAanJZlj+/Pxx82dnZ6t69u7KyspSaepJvFwMITRUVJpTn5Zl/9rcvvzTB/IMPfGv/8R8mmF96qf+fdyKXy1xq1KgRN4ACAI7DTaIAnKW83BfM/TUm8VjffGN6zN9/37d20UXSmDFSt26Bby+pugG0YUMT0AEAOAEBHYAzlJaaUJ6fb/rN/W33bmn2bOntt30HS9u0MT3m3bsHp+87Ls53AygAAKdAQAdgr5ISc/CzqCgwwfz776VnnpHeeMP3+ueea6ay3HxzcC79iY42PebJyVwyBAD4VQR0AMFXNSoxL8//M8yr5OSYYL58ua+HvWVLM8e8d+/gHMiMijKhvEEDE9IBAKgG/o8BIHiOnchSXh6YYH7woDRvnrRkiW/qS7Nm0oMPSn36SDEx/n/miaoOgDZsyEVDAIAaI6ADCLzSUhPM8/MDc/BTknJzpeeekxYtkjwes9a4sTRsmNSvX/AmpcTFmcksdesG53kAgLBDQAcQOMXFZrc8UP3lkgn9CxZICxea50impeSBB6S77jI3cwaD2212zOvX56IhAECtENAB+JdlmRs/8/LMTnagrlooKpJeesnsmufnm7WkJGnIEGngwODtYLtcZipLo0bBaZ8BAIQ9AjoA//B6TTA/ciQwN35W8XikV14xfeaHD5u1xERp0CBp8GBzKDNY4uOllBTzfAAA/ISADqB2An3jZ5WyMum116Q5c8xBUMkE5LQ06f77TXtJsNDOAgAIIAI6gDMTjIOfkpn28sYbZmTijz+atZgYqX9/aehQqUmTwD37RLSzAACCgIAOoGZKSsxueWFh4A5+Sib0v/WWuf3z3/82a9HR0u23SyNGSC1aBO7ZJxMXZ9pZ6tQJ7nMBABGHgA7g1wXjYqEqlZXSu+9Ks2ZJu3aZtago6ZZbzCzzs88O3LNPxu02U2EaNKCdBQAQFAR0AKdWdfAzL8/0gAcymFuWtHatNGOG9PXXvvXf/97c/tm6deCefTIul5kE06iRFBsb3GcDACIaAR3AL5WV+YJ5IA9+SiaYb9xogvm2bb717t2lUaOktm0D+/yTiY017SxcNgQAsAEBHYBPsPrLq2zaJE2fLm3e7Fv73e9MMO/YMfDPP1FUlJnM0rCh+WcAAGxAQAciXTD7y6t8/rnZMf/oI9/ab38rjR4tde0a+OefTGKi2TWPj7fn+QAA/IyADkSqYPaXV/nqK2nmTNNrXqVzZ2nMGOmyy+w5hOl2mz7z5GQOgQIAHIGADkSasjIzuzwvL7Dzy4/1r3+ZqSzvvutbu/BCE8yvusqeYMxMcwCAQxHQgUgR7P5ySdqzx8wxf+st3w79+eebVpbrr7dvx5pDoAAAByOgA+HMjv5yScrONjd/vvGGb5f+N7+R0tOlHj1MW4kdXC7fTHO7agAA4FcQ0IFw5PX62ljKy4MXzPfvl+bOlV57zTxXks46y8wxv+UWcxOoXRISzK55QoJ9NQAAUA0EdCCc2NFfLkmHDknz50svvyyVlpq1pk2lESOkvn3tveiHm0ABACGGgA6Eg+JiE8qLioLXXy5JR45Izz8vvfSSqUEyhy6HDZP695fi4oJXy4lcLjM6sXFjbgIFAIQUAjoQqiorff3lHk/w2lgk89wXXpAWLDD/LJkLfu67T7r7bhOM7RQdbdpZkpLsrQMAgDNAQAdCTXm5r42loiK4zy4ulhYtkp57zuyeS2YSyuDB0r332j8VxeUyobxRI3v73QEAqAX+DwaEiuJiE8yDOSaxSmmp9Oqr0rx5pt9cMrvk99xjwnn9+sGt52Ti4syueZ06dlcCAECtENABJ6u67TM/34TkYLaxSObQ6fLl0pw5ZkKLZILwgAHS0KFmp9puUVG+Q6BRUXZXAwBArRHQAScqLjY75QUFwZ3GUqWiQlqxwlwy9MMPZi0mRurXzxwAbdo0+DWdTEKCOQQaH293JQAA+A0BHXCKsjJfKC8rC/5uuWRaZ95+W5o1y9wCKpkxhX36SCNHmpnmTuB2Sw0bmtYaRicCAMIMAR2wU2WlGY1YUGB2zYPdW17FsqT33pNmzpT+9S+z5nJJPXua2z9/8xt76jqRy2V6zFNSGJ0IAAhbBHTADh6Pr7fcjhaWKpYlffCBNGOGtH27b/2mm0wwP/98+2o7UWys2TVndCIAIMwR0IFgqagwLSx2Hfg8lmVJn3wiTZ8uff65b/3aa6VRo6QLL7Srsl9yuaTkZHMg1e22uxoAAAKOgA4EkmWZ1pWCAnvGI57M5s1mx/yzz3xrV14pjR4tdepkX10nEx9v2lnsvvgIAIAgIqADgVBebkK5nQc+T/TFFyaYf/ihb61rVxPMf/tb++o6GUYnAgAiGAEd8KeSEnPDZlGRM3bLJenrr83hz6ws31rHjtKYMdIVVzhvCkpiotk1Z3QiACBCEdABfygslPLyTDuLE3bLJWnXLjPH/O23fWtt25od82uvdV4wd7tNME9Kcl5tAAAEEQEdOFOWZYL5kSNmKotTgvm//y09/bS5aKhqF791azOV5aabnNcy4nJJdeuacB4TY3c1AADYjoAO1NSx/eWlpXZX47NvnzRnjrR8uZkYI0lnny099JDUq5czJ6DExprpLPXq2V0JAACOQUAHqsOyfBcKOam/XJIOHJAyMqRXXzVfPEhSixbm5s/bbnPmrrTLZW4BbdjQmV84AABgIwI6cDplZb7d8vJy57SxSFJurvTss9LixabFRpIaN5ZGjJDuuMO5N23Gx5s6ExLsrgQAAEcioAMnqqw0u+T5+WYqi5N2yyVT1/PPSwsXmkOpkhlHOGyYNGCAc6efuN2+0YkcAgUA4JQI6EAVj8e3W17Vw+0khYXSiy9KCxaYkC6ZiSf33ScNHCjVqWNvfaficvlGJ8bF2V0NAACOR0BHZPN6faHcSZNYjlVSIr38sukzP3LErCUmSoMHS/fea0K6U0VHm3YWDoECAFBtBHREpuJiE8oLC01Id6KyMmnJEmnePOngQbMWHy/dfbfZNW/Y0N76TsflkpKTzYQWDoECAFAjBHREjmPHI5aVOXO3XDJ1vv669MwzZnSiZCaxDBggDR1qdqSdjEOgAADUCgEd4a3qMqGCArNr7rQDn8fyeqWVK83tn99/b9aio6W+fc1klubN7a3v13AIFAAAvyCgIzyVlppgnp9vDnw6dbdcMl80vPOONGuWtHu3WYuKMjPMR46UWra0tbxfVXUItHFj5452BAAghAQ0oBcWFqp///6aO3euUlNTtWXLFk2bNk1FRUVq06aNHn/8ccXGxuq9997TzJkzVVlZqQ4dOmjKlCmK5X/0qCmv1zce0eNx9m65ZL5oyMqSZs6UvvnGrLlcUs+e0oMPSueeG5Qy3tqwS1mbsuUpq1B8bLS6X5KqXt1aV+83R0eb6SxOPqgKAECIiQrUC2/dulUDBgzQnj17JJmwnp6erilTpmjVqlWSpGXLlqm4uFhTpkzRggULtGrVKpWWlur1118PVFkIR8XF5jbNPXuknBznt7JYlvTBB9If/mCCeFU4v+EG6c03pSefDGo4f3vjHpWWV8gdJZWWV+jtjXv01oZdp/+NVYdAzz6bcA4AgJ8FbAd96dKlmjx5ssaNGydJ2rhxozp37qy2bdtKkiZNmiSv16vExEStWbNGMTExKikp0aFDh5R0iv/h5+fnK79q/vPPcnJyAvUhwMnKy30tLE4+8HmiTz+Vpk+X/vlP39rVV0ujRkkXXRT0crI2ZUsuyR1lvlZ3uyRvZaWyNmWfehc9Ls7smjt17joAACEuYAF96tSpx/147969SkxM1NixY7V792516dJFEyZMkCTFxMRo/fr1GjdunJo0aaJu3bqd9DUXLlyo2bNnB6pkOF3Vgc/CQtPK4uRd8hN9/rkJ5h9/7Fu77DJp9GipSxe7qpKnzOycHyvKZdZ/oeoQaP36pkceAAAERNAOiXq9Xm3YsEFLlixRixYtNHHiRGVkZCg9PV2SdPXVV+vTTz/VU089pT//+c968sknf/EagwYNUp8+fY5by8nJUVpaWlA+BtjE4zGBPC/P9JmHym65JG3fLs2YIa1f71vr0sUE88sus6+un8XHRpv2lmOGrlRaZv0ol0uqW9fMNOdsCAAAARe0gJ6SkqJOnTqp5c8TKXr06KFFixbpyJEj+vLLL4/umvfu3Vtjx4496WskJSWdsv0FYcbr9Y1HLCkJrVAuSTt3mqksq1f71tq3l8aMkX73O8eMIex+Sare3rhH3spKRblMOJdl1iWZdpZGjUxABwAAQRG071N369ZN27dv176fL15Zu3at2rdvL8uy9Mc//lE//vijJOmdd95RFxu/5Q+bHXvgc/9+8+NQCufffSf9539Kt9ziC+cXXCA9/bS0fLl01VWOCeeS1Ktba/3+yt8oLiZa3kopLiZav7/yN+p19QUmmLdsSTgHACDIgraD3rx5c02ZMkXDhw9XaWmp2rVrp/HjxyshIUF/+ctfNGzYMLlcLp133nn6n//5n2CVBScIlRs+Tyc724TwN980u/+SdM455vDnzTc7ume7V7fWvgOhtLMAAGA7l2WFYhryyc7OVvfu3ZWVlaXU1FS7y0F1hfKBz2Pt3y/NmSMtW2a+0JCks86S0tOl3r3NnPBQQTsLAACOEELpAWHB4/GNRwy1A5/H+uknKSNDeuUVs+svSc2aSSNGSLffHlq7z263mczSoIGjd/oBAIgUBHQE3rE3fIbigc9jHT4sPfectGiR+VgkMxN82DDpzjvNLnSocLmkxERTfyjVDQBAmCOgI3BKSny95VV92aGqoEB64QVpwQLzxYZkdp0feEBKS5MSEuysruaio00wZyoSAACOQ0CHf1WNR8zLk0pLQ3u3XDJhfNEis2uel2fW6tWTBg+WBg0KvX5tl8uE8kaNQqs/HgCACML/oeEfHo/ZZa7qLQ91Ho/pL8/IkHJzzVpionTPPdKQIVJysr31nYm4OLNrXqeO3ZUAAIDTIKDjzFVW+g58hnpveZWyMum118xkloMHzVpcnGljeeABqWFDe+s7E1FR5gAoh0ABAAgJBHTUXHm5CeV5eVJFhd3V+EdFhfTGG9Izz0g//GDWYmKkfv3MAdCmTW0t74xwCBQAgJBEQEf1lZSYUF5YGLpzy0/k9UqrVkmzZ0t795o1t1vq29eMTGzRwt76zlRsrOkzr1fP7koAAEANEdBxepWV5qDkkSOmLzsc2lgk83G99540c6b07bdmLSrKXC704INSq1b21nemXC4zXaZhQ/OFBgAACDkEdJxcWZk59BlObSyS+QJj3Tppxgxpxw7f+s03S6NGSa1b21ZarcXHS40bh97IRwAAcBwCOo5XdaFQUVH4tLFIJph/9JEJ5lu3+tavu04aPVpq29a+2mqr6hBow4ZmBx0AAIQ0AjpMH3bViMRwmF1+ok2bTDDftMm31q2bCeYdO9pXlz9UHQKNj7e7EgAA4CcE9EhWWuprYwmH2eUn2rrVBPONG31rl1wijRkjde1qW1l+4XabQ6DJyeyaAwAQZgjokcayfG0sxcXh1cZSZccOE8zXrvWtde5sdswvvzy0A63LZW4vbdTITGoBAABhh4AeKbxeE8rz880B0HBrY5HMNJZZs6R33vGtXXihCeZXXx3awVwygbxhQykpye5KAABAABHQw53H4+svD8c2FsnML589W1q50veFx/nnm6ks118f+rdnulymlaVRI0YnAgAQAQjo4ciyzGVCVW0s4bhbLpkbP595Rnr9dd8XH61aSenp0u9/Hx5hNi7OjE5MTLS7EgAAECQE9HBSUeFrYykvD99gvn+/NG+etHSp+Tgl6ayzzAVDt94qRYfBH2u321w41KBB6H8HAAAA1EgYJBnI4zGhvKAgfNtYJOnQIWn+fOnll80EGklq0kQaMUL6wx/C49CkyyXVqWNGJ4bDxwMAAGqMgB6qvF7TxlJQIJWUhO9uuWTGQD7/vPTii6ZlRzKHJYcNk/r3D58Z4LGxps+8Xj27KwEAADYioIeSqhGJhYXm7+G8Wy6Zj3PhQmnBAvOFiGQOS95/v5SWZnaaw0HVTaC0swAAABHQQ0NpqW+3PJx7y6sUF0uLF0vPPisdOWLW6tSRBg+W7r03fHaYXS4pIYGbQAEAwHEI6E4VKZNYjlVaKr36qjkAeuiQWUtIkO6+W7rvPrPDHC6io303gQIAAByDgO40ZWVmpzxSdssl8zFnZpqRifv3m7XYWGnAAGnoULPDHC6qZpo3bBge02YAAIDfkRCcwLLMLnl+vuktr6y0u6LgqKiQVqwwlwz98INZi4kxE1lGjJCaNrW3Pn+LjzdfbDDTHAAAnAYB3U4VFb5bPsvKImO3XDJfgPz979LMmdKePWbN7TYzzB98UEpNtbU8v3O7fYdAXS67qwEAAA5HQLdDSYkJ5YWF4T+J5ViWJWVlSTNmSDt3mjWXS+rZU3roIemcc+ytz9+YaQ4AAM4AAT1YKit9u+UeT+TslkvmY/3gAxPMt2/3rd90k5SeLp1/vn21BQozzQEAwBkioAdaaakJ5nl5kbVbXuXjj00w37LFt3bNNdKoUVL79raVFTBRUVL9+qadxe22uxoAABCCCOiBEIkjEk/0j3+YYP7pp761K66QRo+WOne2rayASkiQGjdmpjkAAKgVAro/lZebUB5JIxJP9MUXJph/+KFv7T/+wwTzSy+1r65AcrvN2MT69TkECgAAao2A7g9FRSaUFxZGzojEE33zjZnK8v77vrUOHUww79YtPIMrh0ABAEAAENBrq6JC2rcvcoP57t1mjvnbb/u+Y9Cmjekx7949PIO5ZC4ZSkmRkpLsrgQAAIQZAjrOzPffS08/Lb35pu+Lk3PPNVNZbr7ZHJYMRy6XVLeu6TXnJlAAABAAJAzUzL590pw50vLl5rsHktSypZlj3rt3eE8uYdccAAAEAQEd1XPggJSRIb36qjkAK0nNm0sjR0p9+kgxMfbWF0hVveaNG4f3xwkAAByBgI7Ty82Vnn1WWrzYXLAkmaA6fLjUr1/4H450u82ueXKy3ZUAAIAIQUDHyeXnS88/Ly1caGa5S+bynaFDpQEDzMzvcMaEFgAAYBMCOo5XWCi99JIJ5/n5Zi0pSRoyRBo40ByQDHfR0ea7BPXq2V0JAACIQAR0GCUl0iuvmD7zw4fNWmKidO+90uDBkXEw0uUyH2ejRkxoAQAAtiGFRLqyMmnpUmnuXOngQbMWHy+lpUn3329uyIwEcXEmmEfCdwgAAICjEdAjVXm59MYbZpb5vn1mLSZG6t9fGjbMtHhEgqgoqX5984VIuM5uBwAAIYWAHmm8Xumtt8ztn//+t1mLjpb69jWTWVq0sLe+YEpIMF+IxMfbXQkAAMBRBPRIUVkprV4tzZwp7dpl1qKipFtvlR580Fw2FCncbrNjXr++6TsHAABwEAJ6uLMsae1aacYM6euvfes9e5pg3rq1fbUFG6MTAQBACCCghyvLkjZuNMF82zbf+vXXS+npUtu29tVmh5gYE8wZnQgAAByOgB6OPvvMBPPNm31rV10ljRoldehgX112cLnMLaANGzI6EQAAhISAjq0oLCxUr169lJ2dLUnasmWL+vXrp549e+rhhx9WWVmZJOn999/XrbfeqltuuUUjR45UXl5eIMsKX59/bmaWDxzoC+e//a308svS/PmRF87j46WzzpKaNCGcAwCAkBGwgL5161YNGDBAe/bskWTCenp6uqZMmaJVq1ZJkpYtW6bCwkL9+c9/VkZGhlasWKE2bdpo1qxZgSrLbzbv2K9H5mzUwzM+0KzXPtcXu36yr5ivvjKjEe+8U/roI7N28cXSCy+YW0H/4z/sq80OUVFmpnlqqrlsCQAAIIQEbFtx6dKlmjx5ssaNGydJ2rhxozp37qy2P/c+T5o0SV6vV+Xl5Zo8ebKaNm0qSWrTpo1WrlwZqLL8YvOO/ZqXuU3R0S41rhurgsOlWrL6G+lGqUPrlOAV8q9/SbNmSe++61tr314aPdq0tETihJLERNNrzuhEAAAQogIW0KdOnXrcj/fu3avExESNHTtWu3fvVpcuXTRhwgTFxcXphhtukCR5PB5lZGRo4MCBJ33N/Px85efnH7eWk5MTmA/gNDLXfavoaJfiY6PlcrkUF+1Wqdul1Z/uDU5A37PHBPNVq8xhUEk6/3wTzK+/PjKDeXS02TVPTra7EgAAgFoJWmOu1+vVhg0btGTJErVo0UITJ05URkaG0tPTJUkFBQV68MEH1bZtW/Xp0+ekr7Fw4ULNnj07WCWf0v7cYtVLOP6ti42JUm6eJ7APzs6WnnnG3ADq9Zq13/zGTGXp0cPM9440LpeZzNKokZnUAgAAEOKCFtBTUlLUqVMntfz5QpwePXpo0aJFkqQDBw7ovvvu02WXXaZHHnnklK8xaNCgX4T3nJwcpaWlBa7wk2jaMFGH80sUH+t7+8rKK9UwOUBtFfv3S3PnSq+9JpWXm7WzzpIeeki65ZbIPQAZG2vaWerWtbsSAAAAvwlasuvWrZtmzZqlffv2qXnz5lq7dq3at28vr9er4cOHq0ePHho5cuRpXyMpKUlJSUlBqvjUbr/mPM3L3CaPKlQvNkqlFV55vZZuvLSVfx906JCUkSG98opUWmrWmjaVRoyQ+vaN3Mt2XC6pQQPzVyR+1wAAAIS1oAX05s2ba8qUKRo+fLhKS0vVrl07jR8/XmvWrNFXX30lr9erd38+7HjRRRf9oofdSbq2ayrd3lGZ675VUXGpGteJU+8rz/Ff//mRI9Jzz5kJLCUlZi0lRRo6VOrfX4qL889zQlFCgtS4MYdAAQBA2HJZVtUpw9CUnZ2t7t27KysrS6mpqcEvoKLCHNqsrKz9axUUmNGIL7wgFRaatfr1pfvvl9LSIntkoNttLhuqXz8yD8ECAICIEaHNyw5TXCwtWiQ9+6xUdUlT3brSkCHSoEGR3WPtckl16pjvIERqSw8AAIgoBHQ7lZaa/vKMDNNvLpld8nvuMTeC1q9va3m2i4427Sz16tldCQAAQNAQ0O1QViYtW2Yms+zfb9bi4qS77pIeeMCMDIxkLpeZZ96wYeROqAEAABGL9BNMFRXSm29KTz8t/fCDWYuJkfr1k4YNMxNaIl1cnNk1j+R+ewAAENEI6MHg9Upvvy3Nnm0OlErm0GOfPtLIkWameaSLivKNToyKsrsaAAAA2xDQA8mypPfek2bOlP71L7Pmckm9epnbP1v5eW56qEpMNIdAGZ0IAABAQA8Iy5LWr5dmzJC++sq3ftNN0qhR0nnn2Vebk7jdJpgnJTE6EQAA4GcEdH+yLOnjj00w//xz3/q110qjR0vt2tlWmqO4XGZ0ZEqK6cEHAADAUQR0f9m82QTzzz7zrV1xhTRmjNSpk21lOU5srJlSw+hEAACAkyKg19amTdL48dKHH/rWunY1wfySS2wry3FcLjPXvWFD09oCAACAkyKg18Yzz0gPPuj7cceOJphfcQU91cdidCIAAEC1EdBrY+9e8/d27UyP+TXXEMyPxehEAACAGiOg18Zjj0mDBnHb5ckkJJhdc0YnAgAA1AjbmrXhdksXXMDu8LHcbhPMU1MJ5wAAAGeArV/4R9XoxEaNzKQWAAAAnBECOmqP0YkAAAB+Q0DHmWN0IgAAgN8R0HFmOAQKAAAQEAR01IzbbXbM69dnpCQAAEAAENBRPRwCBQAACAoCOn4dh0ABAACChoCOU3O5pORkE845BAoAABAUBHScXFycOQSamGh3JQAAABGFgI7juVxSgwbmICg3pAIAAAQdAR0+8fFm1zwhwe5KAAAAIhYBHWanvGrXnNGJAAAAtiKgR7r4eKlJEy4cAgAAcAgCeqRi1xwAAMCRCOiRiF1zAAAAxyKgRxJ2zQEAAByPgB4p2DUHAAAICQT0cMeuOQAAQEghoIczds0BAABCDgE9HLFrDgAAELII6OEmIcHcBsquOQAAQEgioIcLds0BAADCAgE9HLBrDgAAEDYI6KHM7Ta75g0asGsOAAAQJgjoocjlkhITpZQUKS7O7moAAADgRwT0UBMdbYJ5UpLdlQAAACAACOihwuUyobxRIxPSAQAAEJZIeqEgLs7smtepY3clAAAACDACupNFRUn165vRiVFRdlcDAACAICCgOxWjEwEAACISAd1p3G6zY16/PqMTAQAAIhAB3SlcLtNjnpIixcbaXQ0AAABsQkB3AkYnAgAA4GcEdDu5XFJysmlpYXQiAAAAJAV0NEhhYaF69eql7OxsSdKWLVvUr18/9ezZUw8//LDKysqO+/Xjxo1TZmZmIEtyjrg46ayzpCZNCOcAAAA4KmABfevWrRowYID27NkjyYT19PR0TZkyRatWrZIkLVu2TJK0f/9+DR8+XO+++26gynGOqChz2VDLllJiot3VAAAAwGECtnW7dOlSTZ48WePGjZMkbdy4UZ07d1bbtm0lSZMmTZLX65UkrVy5Ut27d1f9+vVP+5r5+fnKz88/bi0nJ8f/xQdKYqLpNWd0IgAAAE4hYAF96tSpx/147969SkxM1NixY7V792516dJFEyZMkCTdf//9kqR//OMfp33NhQsXavbs2YEpOJDcbrNrnpzM6EQAAACcVtCan71erzZs2KAlS5aoRYsWmjhxojIyMpSenl7t1xg0aJD69Olz3FpOTo7S0tL8Xa5/uFxS3bpm1zwmxu5qAAAAEAKCFtBTUlLUqVMntWzZUpLUo0cPLVq0qEavkZSUpKRQGUUYG2t2zevVs7sSAAAAhJCATnE5Vrdu3bR9+3bt27dPkrR27Vq1b98+WI8PHpdLatDAHAIlnAMAAKCGgraD3rx5c02ZMkXDhw9XaWmp2rVrp/Hjxwfr8cERHy81biwlJNhdCQAAAEKUy7Isy+4iaiM7O1vdu3dXVlaWUlNTg19ARYX0/ffmAGiDBhwCBQAAQK1wQ05tRUVJLVqYi4cAAACAWgpaD3rYiooinAMAAMBvCOgAAACAgxDQAQAAAAchoAMAAAAOQkAHAAAAHISADgAAADgIAR0AAABwEAI6AAAA4CAEdAAAAMBBCOgAAACAgxDQAQAAAAchoAMAAAAOQkAHAAAAHISADgAAADgIAR0AAABwEAI6AAAA4CAEdAAAAMBBCOgAAACAg0TbXUBteb1eSVJOTo7NlQAAAADV16xZM0VH/zKOh3xAP3jwoCQpLS3N5koAAACA6svKylJqauov1l2WZVk21OM3Ho9HX375pRo3biy32x3UZ+fk5CgtLU2LFy9Ws2bNgvrsSML7HHi8x8HB+xx4vMeBx3scHLzPgeeE9zhsd9Dj4+PVtWtXW2to1qzZSb/6gX/xPgce73Fw8D4HHu9x4PEeBwfvc+A58T3mkCgAAADgIAR0AAAAwEEI6AAAAICDENBrISkpSQ899JCSkpLsLiWs8T4HHu9xcPA+Bx7vceDxHgcH73PgOfk9DvkpLgAAAEA4YQcdAAAAcBACOgAAAOAgBPRqKCwsVK9evZSdnS1J2rJli/r166eePXvq4YcfVllZ2XG/fty4ccrMzLSj1JBW3ff5/fff16233qpbbrlFI0eOVF5enp1lh5TqvsfvvfeeevfurZ49e2rChAm/+DOO06vp54x169bpuuuus6PUkFXd93j27Nm69tprdeutt+rWW2/V4sWL7Sw7pFT3Pd69e7cGDhyoW265Rffddx+fk2uoOu/zjh07jv4ZvvXWW/W73/1OvXr1srny0FHdP8vbt29X3759dcstt2jYsGHKz8+3r2gLp/X5559bvXr1stq3b299//33VkFBgXXllVdaO3bssCzLssaOHWstXrzYsizLysnJsYYNG2Z17NjRWr58uZ1lh5zqvs9V6zk5OZZlWdb06dOtv/zlL3aWHjKq+x4XFRVZ3bp1sw4ePGhZlmWNGTPGevXVV+0sPaTU5HOGZVnWwYMHrZtvvtm69tpr7So55NTkPR42bJj1z3/+085yQ1J13+PKykrrxhtvtNavX29ZlmX99a9/tZ544gk7Sw8pNf18YVmWVVxcbPXs2dPatGmTHSWHnJq8xwMGDLDWrVtnWZZlTZs2zXrqqadsq5sd9F+xdOlSTZ48WU2aNJEkbdy4UZ07d1bbtm0lSZMmTdINN9wgSVq5cqW6d++uHj162FZvqKru+1xeXq7JkyeradOmkqQ2bdpo3759ttUdSqr7HicmJmrNmjVKSUlRSUmJDh065MgT7k5Vk88ZVT9+6KGHbKk1VNXkPf7yyy81b9489e7dW1OmTFFpaaltdYeS6r7H27dvV2Jioq666ipJ0vDhw5WWlmZb3aGmpp8vJGnevHm65JJLbL9FPVTU5D2urKxUUVGRJKmkpETx8fH2FC0p2rYnh4ipU6ce9+O9e/cqMTFRY8eO1e7du9WlSxdNmDBBknT//fdLkv7xj38Evc5QV933OS4u7uh/SB6PRxkZGRo4cKAdJYecmvxZjomJ0fr16zVu3Dg1adJE3bp1s6PkkFST9/nFF1/UhRdeqE6dOtlRasiq7ntcVFSkdu3a6Y9//KNatWqlCRMm6JlnntHYsWNtqjx0VPc93rRpk1JSUvTII49ox44dOvfcc/Xoo4/aVHXoqcnnC0kqKCjQ0qVLtXLlymCXGrJq8h5PmDBBQ4YM0WOPPaaEhAQtXbrUjpIl0YNeY16vVxs2bNDDDz+szMxMlZSUKCMjw+6yws6vvc8FBQUaOnSo2rZtqz59+thYaej6tff46quv1qeffqprr71Wf/7zn+0rNMSd6n3euXOnVq9erZEjR9pdYsg71Xtcp04dzZ8/X61bt1Z0dLSGDBmi9evX211uSDrVe1xRUaHPPvtMAwYM0Ouvv66WLVvq8ccft7vckPVrn5dXrFih66+/Xo0aNbKxytB2qvfY4/Fo4sSJeuGFF7RhwwbdddddGj9+vG11EtBrKCUlRZ06dVLLli3ldrvVo0cPbdu2ze6yws7p3ucDBw7orrvuUps2bX7xlTGq71Tv8ZEjR7Rhw4ajv65379765ptvbKw0tJ3qfX7nnXd08OBB9e3bV0OHDj365xo1d6r3+Mcff9SyZcuO/jrLshQdzTeOz8Sp3uPGjRurVatW6tChgySpV69e/D+xFn4tY7z//vv6/e9/b2OFoe9U7/HOnTsVFxenjh07SpLuvPNOffbZZ7bVSUCvoW7dumn79u1H+57Xrl2r9u3b21xV+DnV++z1ejV8+HD16NFDEydOlMvlsrnS0HWq99iyLP3xj3/Ujz/+KEl655131KVLFztLDWmnep9HjRqld999V2+++aYyMjLUpEkTvfzyyzZXG5pO9R7Hx8frr3/9q77//ntZlqXFixf/op8X1XOq9/jiiy9Wbm6uvv76a0nSmjVr+H9iLZwuY1iWpe3bt+viiy+2s8SQd6r3uFWrVsrJydHu3bslSVlZWUe/8LQDWwk11Lx5c02ZMkXDhw9XaWmp2rVrZ+u3QMLVqd7nNWvW6KuvvpLX69W7774rSbrooovYST8Dp3qPExIS9Je//EXDhg2Ty+XSeeedp//5n/+xu9yQxeeMwDvdn+UpU6ZoxIgRKi8vV5cuXTR48GC7yw1Jp3qP4+Pj9fTTT2vSpEkqKSlRs2bN9MQTT9hdbsg63eeL3NxcxcTEKC4uzuYqQ9vpPl9MmzZNY8aMkWVZatSokR577DHb6nRZlmXZ9nQAAAAAx6HFBQAAAHAQAjoAAADgIAR0AAAAwEEI6AAAAICDENABAAAAByGgA4DNPv30U/Xq1atGv+eBBx7Qt99+e9pfM2HCBD333HMn/bnZs2fr/fffr9Ezg6mgoED33HOP3WUAgC2Ygw4AIWj+/Pm1+v2ffvqpzjvvPD9V4395eXn64osv7C4DAGxBQAcAByguLtbYsWO1e/dulZaW6n//93/VsWNH/e1vf9OmTZvk9Xp14YUXatKkSapbt66uu+46zZgxQx06dFBGRoaWLVumOnXqqGvXrsrKytKaNWskSVu2bFH//v31008/6fzzz9eTTz6p119/XV9++aWeeOIJud3u096ueeGFF+qBBx7Qhx9+qOLiYj388MO68cYbVVxcrD//+c/au3evjhw5ojp16uhvf/ubzj33XA0cOFDJycnavXu3BgwYoNtuu01Tp07Vzp07VV5erssvv1zjxo1TdHS0OnTooKFDh2rjxo06cOCA7r//ft1111367//+b3k8Ht16663KzMzU008/rffee08xMTFq0KCBpk2bpiZNmgTrXw8ABBUtLgDgADk5Obr33nv15ptvqn///po1a5YyMjLkdruVmZmpFStWqEmTJvrb3/523O/78MMPlZmZqWXLlikzM1NFRUXH/fz+/fu1YMECvfvuu9q/f79Wr16ttLQ0XXTRRRo3btxpw7kkeb1eJSQkKDMzU9OnT9cjjzyi3NxcffDBB0pKStKSJUv07rvv6qKLLtLixYuP/r6kpCS9/fbbGjhwoB577DG1b99emZmZeuONN3T48GEtWLBAklRWVqYGDRro1Vdf1cyZMzVt2jSVlpZq2rRpio+P15tvvqkDBw5o4cKFWr58uTIzM3XllVdq27ZtfnrnAcB52EEHAAdo2bKlOnXqJElq27atli9frnXr1qmgoEAfffSRJKm8vFyNGjU67vetX79eN998s5KSkiRJaWlp+uSTT47+/PXXX6+EhARJ0vnnn6/c3Nwa13b33XcfreuCCy7Qpk2bdPPNN6tly5Z66aWXtHfvXn322We6+OKLj/6erl27Hv3ndevW6YsvvtCyZcskSR6P57jX7969uySpffv2KisrU3Fx8XE/37RpU7Vt21Z9+vTRVVddpauuukqXX355jT8OAAgVBHQAcICYmJij/+xyuWRZliorK/XII4/o6quvliQVFRWptLT0uN8XHR0ty7KO/tjtdv/i50983Zo69jUrKyvldrv18ssva+nSpUpLS1Pv3r1Vv359ZWdnH/11iYmJx/2eGTNmqHXr1pKk/Px8uVyuoz8fFxd3tD5Jv6gxKipKixYt0hdffKGPP/5Yjz32mH73u99p3LhxNf5YACAU0OICAA7VrVs3LV68WGVlZaqsrNSjjz6qp5566rhfc/XVV2v16tUqKCiQpKO71L/G7XaroqKiWr/2jTfekCRt375d3333nS655BJt2LBBffr00R133KFzzjlHa9askdfrPeXH8cILL8iyLJWVlWnEiBFatGjRaZ8ZHR0tr9cry7L09ddfq1evXmrdurWGDRume++9lwOkAMIaO+gA4FAjR47U//3f/6lPnz7yer1q166dJkyYcNyvufzyy9WvXz/deeedio+P1/nnn3+0peV0rrvuOj311FMqLy9Xnz59Tvtr//nPf2rp0qWqrKzU//t//0/JyckaMmSI/vSnPx39gqBz587auXPnSX//xIkTNXXqVPXu3Vvl5eW64oordP/995/2mY0bN1bHjh3Vs2dPLV68WD169FDfvn2VmJio+Ph4TZo06Vc/RgAIVS7rTL7fCQBwhC+++EJbtmw5OjN8wYIF2rp1q6ZPn+6X12/Tpo0+/vhjNWzY0C+vBwD4deygA0AIO+ecczR//nwtXbpULpdLzZs311/+8pdq//5nn31WK1euPOnP3Xffff4qEwBQA+ygAwAAAA7CIVEAAADAQQjoAAAAgIMQ0AEAAAAHIaADAAAADkJABwAAAByEgA4AAAA4yP8H9uhy2PtwVk4AAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot regression line with CI 95%\n",
"sns.lmplot(x='height_parents', y='height', data=df, order=1, line_kws={'color':'red'}, height=7, aspect=1.5, ci=95);"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Confidence intervals in small samples\n",
"\n",
"The procedure we just used is fine when samples are large, because the central limit theorem tells us that the sampling distribution will be normal. However, for small samples, the sampling distribution is not normal – it has a t-distribution. The t-distribution is a family of probability distributions that change shape as the sample size gets bigger (when the sample is very big, it has the shape of a normal distribution). \n",
"\n",
"To construct a confidence interval in a small sample we use the same principle as before, but instead of using the value for z we use the value for t:\n",
"\n",
"lower boundary of confidence intervall = $\\bar{X} - (t_{n-1} \\times SE)$\n",
"\n",
"upper boundary of confidence intervall = $\\bar{X} + (t_{n-1} \\times SE)$\n",
"\n",
"The (n − 1) in the equations is the degrees of freedom and tells us which of the t-distributions to use. For a 95% confidence interval, we can calculate the value of t for a two-tailed test with probability of 0.05, for the appropriate degrees of freedom."
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.093024054408263\n",
"Lower boundary of CI (t-statistics) 164.30416372335924\n",
"Upper boundary of CI (t-statistics) 165.69583627664076\n"
]
}
],
"source": [
"# calculate t-statistic\n",
"# 95% confidence interval, two tailed test, \n",
"# p<0.05 (we need to take 0.025 at each side), n=20, df=19\n",
"t = stats.t.ppf(1-0.025, 19)\n",
"print(t)\n",
"# lower boundary\n",
"lb_t = (df.height.mean() - (t*se))\n",
"# upper boundary\n",
"up_t = (df.height.mean() + (t*se))\n",
"print('Lower boundary of CI (t-statistics)', lb_t)\n",
"print('Upper boundary of CI (t-statistics)', up_t)"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEJCAYAAACaFuz/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAo4UlEQVR4nO3deVhU56EG8Hc2ZthGUDYFXHDDBddoRFSiMca4FrVRQxKatElsmuZq7pPUx2i9N6Y217ZJm3DTW/OkiU3tYowmpk2iRtxwX+IGuCIKwiCKMMwAs577Bzo6IoiGM99w5v09jw9ymOG8H8vr8ZtzvqOSJEkCEREpllp0ACIikheLnohI4Vj0REQKx6InIlI4Fj0RkcL5XdE7nU6UlJTA6XSKjkJEpAha0QFuZzKZ8PDDD2PLli1ISEgQHYcE2zM7E+76eqgNBqT+c7XQLI8v+hfqbC4E6zVYs3yK0CyAf31tyL/53RE90a3c9fVeb0Wqs7m83ormT18b8m8seiIihWPRExEpHIueiEjhWPRERArHoiciUjhZiz4nJwczZszAY489hjfffFPOXRERURNkK/ri4mIsXboU77//PjZs2ID8/Hxs375drt0REVETZLtgavPmzZg0aRLi4uIAAO+88w70er3XY8xmM8xms9c2k8kkVyQiooAkW9FfuHABOp0O8+bNQ1lZGR566CHMnz/f6zGrVq1Cdna2XBFIwRxON3TawHqJKRDHTK1DtqJ3uVw4ePAgPvnkE4SEhOCnP/0p1q9fjxkzZngek5WVhYyMDK/nmUwmZGZmyhWLFEKnVWPR+7uE7V/Evpe/mObzfZIyyFb0UVFRSE1NRfv27QEA48ePx7Fjx7yK3mg0wmg0yhWBiIgg44uxY8eORW5uLsxmM1wuF3bu3Il+/frJtTsiImqCbEf0AwcOxE9+8hM88cQTcDgcSEtLw8yZM+XaHRERNUHWZYpnzZqFWbNmybkLIiK6C76ET0SkcCx6IiKFY9ETESkci56ISOFY9ERECseiJyJSOBY9EZHCseiJiBSORU9EpHAseiIihWPRExEpHIueiEjhWPRERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoVj0RMRKRyLnohI4Vj0REQKx6InIlI4Fj0RkcKx6ImIFI5FT0SkcFo5P/lTTz2FyspKaLUNu3njjTcwcOBAOXdJRES3ka3oJUlCUVERtm7d6il6IiLyPdmmbgoLCwEAzz77LKZNm4a//vWvcu2KiIiaIduhttlsRmpqKpYsWQKHw4Gnn34a3bp1Q1pamtdjzGaz1/NMJpNckYiIApJsRT948GAMHjzY8/6sWbOwfft2r6JftWoVsrOz5YpARESQsegPHjwIh8OB1NRUAA1z9rfP1WdlZSEjI8Nrm8lkQmZmplyxiIgCjmxz9DU1NVixYgVsNhssFgvWr1+PRx55xOsxRqMRCQkJXn/i4uLkikREFJBkO6IfO3Ysjh49ih/84Adwu9144oknvKZyiIjIN2Q973H+/PmYP3++nLsgIqK74JWxREQKx6InIlI4Fj0RkcKx6ImIFI5FT0SkcCx6IiKFY9ETESkci56ISOFY9ERECseiJyJSOBY9EZHCseiJiBSORU9EpHAseiIihWPRExEpHIueiEjhWPRERArHoiciUjgWPRGRwrHoiYgUjkVPRKRwLHoiIoVj0RMRKRyLnohI4Vj0REQKx6InIlI42Yv+f/7nf7Bw4UK5d0NERE2Qtej37NmD9evXy7kLIiK6C61cn7iqqgrvvPMO5s2bh5MnT97xMWazGWaz2WubyWSSKxIRUUCSreh/+ctfYsGCBSgrK2vyMatWrUJ2drZcEQKGw+mGTsuXW5QuEL/PgThmOchS9J9++ik6duyI1NRUrFu3rsnHZWVlISMjw2ubyWRCZmamHLEUS6dVY9H7u0THkMXkW/5+6xiXv5jm+zCC3f59buproySB+H2WgyxF/9VXX6GiogLTp09HdXU1amtrsXz5cixatMjrcUajEUajUY4IRER0nSxF/9FHH3n+vm7dOuzfv79RyRMRkW9w8ouISOFkezH2hhkzZmDGjBly74aIiJrAI3oiIoVj0RMRKRyLnohI4Vj0REQKx6InIlI4Fj0RkcKx6ImIFI5FT0SkcCx6IiKFY9ETESlci4r+TguSvfzyy60ehoiIWl+za90sXboU5eXlOHToECorKz3bnU4niouLZQ9HRETfX7NFP2vWLJw5cwanTp3Co48+6tmu0WgwaNAgubMREVEraLboU1JSkJKSgpEjRyIuLs5XmYiIqBW1aJnisrIyvPrqq6iuroYkSZ7tX375pWzBiIiodbSo6H/5y19ixowZ6Nu3L1QqldyZiIioFbWo6LVaLZ555hm5sxARkQxadHplz549cerUKbmzEBGRDFp0RF9cXIyZM2eiU6dO0Ov1nu2coyci8n8tKvoFCxbInYOIiGTSoqLv1auX3DmIiEgmLSr6ESNGQKVSQZIkz1k30dHR2LFjh6zhiIjo+2tR0Z88edLzd7vdjn/96184f/68bKGI6PtRud3QOergVmvg1BlExyHBWlT0twoKCsKMGTMwY8YM/Od//qccmUhJJDeSig4jvjQfBpsFkkqNekMYTDE9cK7bMEhqjeiEiqJ11GNA3mbEVBRC7XbDfr3kSzsm42Tv0XCr7/lXnhSgRd/1qqoqz98lScKJEydgNpvlykQK0q9gK3SOeuT1GYt6QzggSTDYrOhccgwpeZtxLGWi6IiK0r8gB+Ux3XEkZSISS05AUqtRHt0dPQr3oV9+Do73nyA6Iglwz3P0ANChQwe8/vrrd33eH/7wB2zcuBEqlQqzZs3iRVcBKOrqRWwf7f19rw2NRGVkPNJ3rRKUSrnCLJU4MmASAOBCl8FI27MaFxMHIq/vw0jP/VhsOBLmnufoW2r//v3Yu3cvNmzYAKfTiUmTJiE9PR1JSUn3/Lmo7ZJUaujsdXAEBXtt1znq4eZyGq1O43ZAb7PApg+Doa4GKjQcnGmcdkh3eS4pV4uK3u1248MPP8SOHTvgdDqRlpaGefPmQatt+unDhw/HX/7yF2i1WpSXl8PlciEkJKTVglPbUNh1KEbv+SvKY7qjzhAOADDUWxBTUYgz3UcITqc8FxIHIW3v33GlQ2d0uFqMMz1GIKS2CiMOrMG5bsNFxyNBWlT0v/vd73Dy5ElkZWXB7Xbjn//8J1asWHHHO0/dSqfT4d1338Wf//xnTJw4EbGxsV4fN5vNjeb6TSbTPQ6B/FlJQn9UtYtDbMU5hNRVA5KEeoMRhwZPQ014tOh4ilPUZTDM4VFoZy5HcXx/XIuMh8bpwL4HfghraKToeCRIi4p+586d+Oyzz6DT6QAADz30EKZNm3bXogcabjn43HPPYd68eVizZg1mz57t+diqVauQnZ19n9HvzOF0Q6flrXD9iSU8CpbwKABAiPUawi1XYNcF3+VZdL9cai2KE1Lg1OoRc/kcIqtKUdUurk0WfaD9Pss13hYVvSRJnpIHGk6xvPX9Ozl37hzsdjv69OmD4OBgTJgwodHCaFlZWcjIyPDaZjKZkJmZ2dL8jei0aix6f9d9P78tWv5imugITWpXbcKg41/DrgvGuW7DMPD4N6gJj0ZobQ6+GzAJle0TRUdUlKTzB9Gl+Cj2DvshEi7loXPxMVyOTkLPc3sRbrmKs21suizQfp/l+l1uUdEnJydj+fLlePLJJ6FSqfDJJ5/cdVmEkpISvPvuu/j73/8OANiyZQtmzpzp9Rij0Qij0Xif0akt6HdyKwp6pyOktgpDjv4Lux+cC7MxBmGWKxhwYjN2j5grOqKiJF46jh0jn4JLG4SES/nYO+xx2PUh0DjtGLX3b22u6Kl1tKjoly5dijfffBNz5syB2+3G6NGjsWTJkmafk56ejmPHjuEHP/gBNBoNJkyYgMmTJ7dKaGo71C4nLkcnAZKEpPMHYTbGAAAsYVFQu12C0ymPS62DSxsEAHBqg2DXN5wA4dIGQQLPcgpUzRa93W7HkiVLMH78eLz11lsAgOeffx4ajQZhYWF3/eQ///nP8fOf/7x1klKbJKnUCK+pQE14NHJTn/Rsj6gqhcTTK1vdtchOGHTsK5zqMRKlHZORfGoHSuL7IuFSPqrbxYiOR4I0O+v/7rvvwmKxYMiQIZ5ty5Ytg9lsxnvvvSd7OGr7TvYajeEHPwMkt+foMrb8LIYdWo/85HTB6ZSnoNcY2HUGjNqzGv0KcpBUdBCj9qxGSF018pLHio5HgjR7RL9t2zasXbsWBsPNRZFiY2OxYsUKzJ49m+vU011d7dAZOenPASr1LdsSkZP+PFza5l/Qp3vn1miR32cc8vuMQ5C9Fiq3G/agYK4pFOCaPaLX6XReJX9DWFgYgoKCZAtFynJ7yTi1eri0OhjN5YISBQZ7UAhshjDP1z/mcqHgRCRKs0WvVqthsVgabbdYLHA6nbKFosAw4MQm0RECSpfio6IjkCDNFv2UKVOwePFi1NbWerbV1tZi8eLFmDCBq+DR95M78inREQLKgaEZd38QKVKzRZ+VlYXw8HCkpaXh8ccfx6xZs5CWlgaj0Yif/exnvspICsMpBPmEWK95/m40l6Pn2T3oXrgPodZKgalItGZfjFWr1Vi2bBnmzZuHvLw8qNVqDBgwADExPE2LWuZO8/B9Tu9AvSEMgASzMbbxk+i+DTn6b+SOfBKx5WeQkr8FpXG9oAIw4sCnyEseC1Mc7/8ciFp0wVR8fDzi4+PlzkIK9ODBz6Byu68vU9ywUK6+3oqhRzYAUGHrmB8LzadU3c8fwJ5hj8Ma1h4AUNj1AQw7vJ5FH6B4XzGSVe6ITAw6/jVK4vuhOCEFADBq9yecn5eZWnJ7LWJWF2zklbEBLHCWhSMh6kLaYd+wWYioKsOA4xuhdjkBFo5sQmuvoX/et3BqdOhccgwAoHE60K3oEGx63g8iULHoSXZutRbH+0/AtchOSN3/T2jcDtGRFGvXg3NRbYxBbXA7hNdcAQB0vXgYHU2ncLzveMHpSBRO3ZDPFCekoNoYi/jSfNFRFOvG2v/FiQM82851G45zSQ8KTEWi8YieZKVyu5B0/gD0NgvULgfiS/MRWVWKvgU50DjtouMpjs5eh/75WxBcVw2dvQ5Dv/sCD29biaGHv4DeZhUdjwRh0ZOsBpzYhIhqE9wqDfqc2gG9zYoz3UdA43YhJf9b0fEUZ+CJjXCptXDoDOh7ahusIRHYN2wWqtvF8krkAMaiJ1m1M1/G4UFT4QgKRmRVKY6mPIaK6CQc7zse7cyXRcdTnOA6MwqS0+HU6hFecwUne6fDEtYBZ7uPQHC9+e6fgBSJRU+ycms00DpsAACbPtRzsxG128XT/WTgVmtgqKsB0HBKZZC9YfkSnb3u+lUMFIj4YizJqji+P0bu+wcKuw5FZWQCHvjuc5hie6Kj6RRKO/YWHU9xznUbhrR9f0NxfH9YQtvjwQOfoiKqK2IrzuN8lyF3/wSkSCx6ktWFzoNg04civjQfodYqqCQXOppOoyy2Fy50HiQ6nuKY4nqhLtiITmUnEWa9Bps+DEH2OhT0GoPLMUmi45EgLHqSnSm2J0yxPUXHCBjV7eJQ3S5OdAzyI5yjJyJSOB7Rk6y6FR1q9uPnuw71UZLA0Lcgp9mP5/cZ56Mk5E9Y9CSr8JoKdCw/g7K4Xmh02gdPuml1Tq0e3S4cRmHXoeAXmG5g0ZOsjqVMRHB9DSo6dEFZx2TRcRTvdM80hFkrYQ8K4Yvd5ME5epLdiT4Po31VqegYASMv+SHo7HWiY5AfYdGT7Kxh7ZHHuWGfsRnCcbZHqugY5EdY9ERECifrHH12dja+/vprAEB6ejpee+01OXdHRER3INsR/e7du5Gbm4v169fj888/R15eHjZv3izX7oiIqAmyHdFHR0dj4cKFCAoKAgB0794dpaXeL8iZzWaYzd4r6plMJrkiEQUMraMeTp3B85YCm2xH9D179sSgQYMAAEVFRfj666+Rnp7u9ZhVq1bh4Ycf9vqTmZkpVyQSaOTev3u9JXmNOLDW6y0FNtnPoz9z5gxeeOEFvPbaa+jatavXx7KyspCRkeG1zWQysewV6ObyxE7BSQINFycmmYv+0KFDePnll7Fo0SJMnjy50ceNRiOMRqOcEYiIAp5sRV9WVoaf/exneOedd5CaynN6iYhEka3oP/zwQ9hsNrz11luebXPmzMHcuXPl2iUREd2BbEW/ePFiLF68WK5PT21MZWT89bcJgpPIy+5wQaUCdFqN0Bylcb2uv+VdvIiLmpGP5PcZ6/VWKS6Wm9E51gi704ULZWbYnW5AAgx6DTrHhgsr/MKk4V5vKbCx6Im+B5u94WyisitWRITrER0RAgC4Wl2HkssWdOvUTmQ8IgBc64aoVdgdLk/JA0CHdsFwON0CExHdxKIn35Aan8+tddQLCNK6XG4JtfUOBOk0qLffvEbA7nAJve+Hvr6m0bYwy1UBScgfsOjJJ0btWd1o28j9/xSQpHW1Cw1C2RUramrtMF21AgCu1dTjbEkVYiKCfZ5HZ6+Dzl6H4YfWQ+uo97wfZLNi6JENPs9D/oFz9CSrBw98inbV5dC4HZjwbbZnuwpuVBvjBCZrHR2jwgAAbkmC8/pUTXCQFknx7WAI8v2v1+BjXyHq6gUAwISc9z3bJZUaZbE9fZ6H/AOLnmR1aPA06Bz1GHBiE471f9Sz3a1SwaYPE5is9dTWO6AP0iBIp4HZakNtvRPBeq2Qot//wEwAwIATG72+3hTYWPQku7rgdjg8cEqj7TpHPRxBvp/eaE0VVbW4Wl2PpE7tcM1ch0pzPcJDg3D5Wi1sDhdiIkPu/klaUailEtaw9ijqPAhGc3mjj5uNsT7NQ/6BRU+yGrH/U+SOfBKPbP0jGl6dvPVFWRW+enSBoGSt45rZhp6JEdCo1bhW01D4Wq0arsgQnCup8nnR9zm9HQeHZGDokS/v8FEVto75sU/zkH9g0ZOsckc+CQD46tFXBCeRh0oFaNQN5zRo1GpotTf+LuaUm4NDGlaD3TrmJ0L2T/6JRU8+oXY5EHv5HIJuO6XyQudBYgK1klCDDsXlNYhtH4J2YUEou2pFZLge12psCNaL+/XS26xIuJQH3W1f75O9xwhKRCKx6Mknhh3+HFqnHXXBN5ellqBq80Uf1yEUpkorzpZUweVumJa6Wl2H8JAgJESLe7H5gcOfo94QDmsIr8wlFj35iMFmxfZRPxIdo9Wp1Sp0igpDp6gwOF1uSJIErUYNlUrg1VIAVJIbhwZPE5qB/AcvmCKfMIdFQW+zio4hK61GDZ1WI7zkAcBsjEFYzRXRMchP8IiefKIsrhfScz9GTVgU3Oqbxxf7hv1QYKrvr87W/K0RRc3TV0bEY/SeT2DTh0JS3VxBk2fdBCYWPflEr3N7cDZpOGqDlTVnXHK5BjaHC1rNnf9znNylvY8TNehedBBHBkyCNThCyP7Jv7DoySecGh0Kuw0THaPVJXVqh3OXqpEYGy70LJvbObR6lPGmI3Qd5+jJJ6526IwuF49Ab7N4FtrS2etEx/reNBq158wbf3K1QyL6nNqOiKpSGM3lnj8UmPznEIQUrVvRIajdLvQryLlla9u/MhYAjKFBMIYGiY7hpVPZSQBAXPmZW7byythAxaInn/jmkf8QHSGg8MpYuhWLnmQVX5qPS536olvRoTt+/HzXoT5OpGxJ5w+gsNsw9PX6n9NN+X3G+TgR+QMWPckqpLYKABDOc7p9wqnVAwAcura9Kii1LhY9yepMj5EAgGMpXBvdFy4mDgAAnOmRKjgJ+RMWPclq9K5VaO7mqTvTnvZdGBmVVljQKTrM81aUR799D1IzX+9N41/yYRryFyx6klVegMwJW+sdXm9F2ZGWJXT/5J9kLXqLxYI5c+bg//7v/5CQkCDnrshPVbZPFB0hoNy6OijRDbJdMHX06FHMnTsXRUVFcu2CiIhaQLaiX7NmDZYuXYqYmBi5dkFERC0g29TNr371q7s+xmw2w2w2e20zmUxyRSKBep/eiVO9RouOIRv19VsHqgXdQvB2A49/jaMpj4mOQX5C6Iuxq1atQnZ2tsgI5CMxFYWKLvru8RFeb0UzmisASWq4qS0FPKFFn5WVhYyMDK9tJpMJmZmZghKRXGqDIzD84GeojOgEl/bmujC8MlYe9fpQjNm1ClURHeHU6DzbeWVsYBJa9EajEUYjzxIIBA6dAQ4AIXW3TNXxYFM2VRGdUCU6BPkNnkdPPnHjylitox5OnUFwGuU70yMVapcDobVVqAmLgtrthPuWI3sKLLKvR5+Tk8Nz6Amh1msYk/sx0netgr7egvTcjxFqqRQdq1XZHS7U2Zxef0SJqCrD2J1/xrDDn8Ngs+Dh7R8g8lqpsDwkFm88Qj7RryAH+cljYQsKgc0QhqLOg5CSv1l0rFZTXmnF6eJruGAye/0Rpc+pHdj3wCzYdQbUG8JxJGUi+p7cKiwPicWpG/KJIEcdrkR1AU43vH+h8yAklhwXG6oVXauxoXfnSOi0mrs/2Ac0bgcsYR0871dEJ6H3mV0CE5FIPKInn5CggtrlxI1XYPU2K1SSW2yoVqTTqv2m5AHArVJD66jHja93qFVZ02R0b3hETz5xMXEAhh/6DEH2WvQ+vROdyk7hnIJuFh4WHISyq1YYQ4K8LpoSdcPws0kjkLp/DfQ2KwYd/Teir17A8b7jhWQh8Vj05BPFCSmwhkQipqIQasmN4/0eaZjKUYhrNfUAgGqLzWt7cpf2IuLgckwSLKGRiLp6ESrJjbPdR3hN5VBgYdGTTww6+m+YYnvidI9URZ7mJ6rQm6OWXNC4nZBUKrhVnKUNZCx68onL0UnoWH4a/QpycC2iE0yxPVEe093rKtm2zOlyo6rGBpdbur5Fgs3hRufYcCF5OhcfRa+zu1ER1Q0qyY0ehfuR12csyuJ6C8lDYrHoySdKO/VBaac+ULld6Gg6jd5ncjEgbxO+eeQ/REdrFRdNNVCrgXq7C2HBOljqHAg1iPufS7eiQ9iZ+iRshoZ/aAx1Zgw/vJ5FH6BY9OQT7SuLEXX1IqKuXoDBZsHV9p1R0UE5c/QOpwu9u7THpQoL2hsNiG0fggumGmF5nFq9p+QBoD7YCJeav+6Bit958okRBz6FTR+KM0kjUJyQAkmtrDljrbZhPHqdBvV2JyLDDZAk6S7Pks+VDl3QP/9bFCUOgqRSI6E0H9aQCBjN5QAAszFWWDbyPRY9+cSWh55H9JUiRF+5gO5FB2EOi8KVqC640HmQ6GitQqtRo+JaLUIMOpRfq4VGrYJbYNF3Mp0EAERfKfLaPvTIlwBU2Drmx74PRcKw6MknbPowlMT3x+XoJMRUnEfS+QPocKZEMUUfHxWGKosNocE6BNdqUV5Zi7j2ocLybB3zE2H7Jv/Doief6HVmF6KvFMFgs+BydBIKeqfjSofOomO1Gq1WjfZGA+ptTsS1D4EUGSL0blMapx09z+1F9JUiSCo1ymOScC5pONycpw9I/K6TT2hcDuQnP4RrEZ0Uedej2noHLphqoFIB3ePb4UxxFbp0NAo782ZA3mZIKhXykx8CIKFzyXH0LdiKE/0eEZKHxGLRk08U9B6DpKJD6H1mF1SSGxUduuBc0oOKeVG27KoV3ToZUVxeA51Wg4SYcJRdsaJHQoSQPEbzZWwf/Yzn/avtOyN91yohWUg8ZfyWkd/rfWYXOly9iPNdhuB816GIrCpF8ukdomO1GrdbgiHo5nGTMTRI6Fk3Nn0odPY6z/salwN2XbCwPCQWj+jJJ2IqziM3NROSumGFx8vR3TB69ycowENig7USlUoFl+vmapw2u7ibjgBAnSEco/b8Faa4XnCr1Ii7fA62oBD0LcgBwHvHBhoWPfmI5Cl5AHCrtYpafyUmMgSFpdVwON24WG6GpdaB+OgwYXlqQyJQGxLheb+UV8QGNBY9+YQ5PAZ9Tm7Dhc6DIAHoevEIasKjRcdqNcbQIOiDNLDU2iGhofhvncrxtTM9UoXtm/wPi558Iq/vOPQt2IqR+/4BQEJFh67ISx4rOtb35rxlukajVqFdmN7rY1qNb//XMmL/Gki3nNUkqVRw6IJxObobLnXq69Ms5D9Y9OQTTq0ex1Imem0Ls1yBI6htv0BYUNT8nZtSukf5KEmDojtcgKa3WdH1wncIstfhfNehPs1D/oFFT8KM3PsPbBr/kugY30tEuB619Q4YQ/SINOqFTtcAgCmu1x23X+rUB6n717DoAxSLnoRRQdzph60lMSYcbreEaqsNZVescLklRIbrERGmh8bH0zbNceoMXlM6FFhY9CSMBGUUj1qtQmS4AZHhBjicLlyrsaGwtBp6nQad44yi4zWQJKjdyrkZO90bFj1RK3K6JDhdbrhcElwa3/+P5daLpDzbHPXodvE7XIvo6PM85B9Y9CSrR799745H7ipI0LjEXlTUWuxOF6pqbKiqsQEqIDJMj+4J7aDTau7+5Fb2yNY/AlABnmkxFWxBwaiI6nZ93RsKRLIW/Zdffok//vGPcDqdyMrKQmZmppy7Iz+0Iy1LdARZFV6qhs3hQruwICTGhiNYL/bY6atHXxG6f/JPsv1UlpeX45133sG6desQFBSEOXPm4MEHH0SPHj3k2iX5obpgP5mjlom13gGVCrhmtuGa2dbo4/2SOghIReRNtqLfvXs3RowYgYiICADAo48+im+++QYvvXTzdDqz2Qyz2ez1vEuXLgEATCbTfe/bar5y389ti0pKShQ75it2u+fvt45RxJgdtTfPmbde/7FNbH9jy53n41sz4+1jbuproyRK/tm+k5KSku/9OeLi4qDVele7SpJpib0//elPqK2txYIFCwAAn376KY4dO4Zly5Z5HvPee+8hOztbjt0TEQWkLVu2ICEhwWubbEf0brcbqlsvxZYkr/cBICsrCxkZGV7b7HY7iouL0bVrV2g09/ZilslkQmZmJlavXo24uLj7D+8HlDIWpYwD4Fj8lVLG0lrjuNNzZSv6uLg4HDx40PN+RUUFYmJivB5jNBphNDaew01KSvre+779X7S2SiljUco4AI7FXyllLHKMQ7ZL90aOHIk9e/agsrISdXV12LRpE8aMGSPX7oiIqAmyHdHHxsZiwYIFePrpp+FwODBr1iwMGDBArt0REVETZD3pd+rUqZg6daqcuyAiorvwn1WXWoHRaMRLL710x3n/tkYpY1HKOACOxV8pZSxyjkO20yuJiMg/KOqInoiIGmPRExEpXJsseovFgilTpnguF/7uu+/w+OOPY/LkyXjllVdgv35p+Lfffovp06dj2rRpePHFF1FdXS0y9h21dCybN2/G1KlTMXnyZCxcuNCz3V+0dBw3bNu2DePGjRMR9a5aOpbs7GyMHTsW06dPx/Tp07F69WqRse+opWMpLCzEU089hWnTpuHHP/5xm/1dKSgo8Hw/pk+fjtGjR2PKlCmCk3tr6fckLy8PM2fOxLRp0/DCCy80Wi7mnkhtzJEjR6QpU6ZI/fr1k4qLi6WamhopLS1NKigokCRJkhYsWCCtXr3as91kMkmSJEm///3vpWXLlomM3khLx2K1WqVRo0ZJFRUVkiRJ0vz586V//OMfIqN7aek4bqioqJAmTpwojR07VlTkJt3LWF544QXp8OHDIuM2q6Vjcbvd0oQJE6Tt27dLkiRJv/nNb6QVK1aIjN7Ivf6MSZIk1dbWSpMnT5YOHDggIvId3cs45s6dK23btk2SJEn69a9/Lb399tv3vd82d0S/Zs0aLF261HOV7a5duzBo0CAkJycDABYvXoxHHnkEDocDS5cuRWxsLACgd+/eKCsrE5b7Tlo6lpCQEOTk5CAqKgp1dXW4evWqX51h0NJx3LB48WKvxe38yb2M5cSJE/jTn/6EqVOn4o033oDN1nj1SpFaOpa8vDyEhIR4LmicN2+e3y0pfq8/Y0DDelvDhg3DAw884PO8TbmXcbjdblitVgBAXV0dDAbDfe+3zd145Fe/+pXX+xcuXEBISAgWLFiAwsJCDBkyBAsXLoRer/d8werr67Fy5Uo89dRTIiI3qaVjAQCdToft27fjtddeQ0xMDEaNGiUi8h3dyzj+8pe/oG/fvhg4cKCIqHfV0rFYrVb06dMHr776Krp06YKFCxfi/fff9yzi5w9aOpYDBw4gKioKixYtQkFBAZKSkrBkyRJBqe/sXn7GAKCmpgZr1qzBl19+6euozbqXcSxcuBDPPvssli9fjuDgYKxZs+a+99vmjuhv53K5kJubi1deeQXr1q1DXV0dVq5c6fl4TU0Nnn/+eSQnJzdaQM3f3G0s6enp2LdvH8aOHYv/+q//Ehf0Lpoax+nTp7Fp0ya8+OKLoiO2WFNjCQ0NxQcffIDu3btDq9Xi2Wefxfbt20XHbVZTY3E6ndi/fz/mzp2L9evXIzExEW+99ZbouM262+/Khg0bMH78eHTo4N/3A2hqHPX19Xj99dfx8ccfIzc3F0888QR+8Ytf3Pd+2nzRR0VFYeDAgUhMTIRGo8Fjjz2GY8eOAQAuX76MJ554Ar179270L6k/amosVVVVyM3N9Txu6tSpOHXqlMCkzWtqHN988w0qKiowc+ZMPP/8857vjz9raiylpaVYu3at53GSJDVaA9zfNDWW6OhodOnSBSkpKQCAKVOmeH6H/FVzv/dAw4kYkyZNEpiwZZoax+nTp6HX6z3LxsyePRv79++/7/20+aIfNWoU8vLyPPPvW7duRb9+/eByuTBv3jw89thjeP311xstkeyPmhqLJEl49dVXUVpaCgD45ptvMGTIEJFRm9XUOF5++WVs3LgRX3zxBVauXImYmBj87W9/E5y2eU2NxWAw4De/+Q2Ki4shSRJWr17daI7Y3zQ1lsGDB6OyshInT54EAOTk5KBfv34io95VU2MBGv7RzcvLw+DBg0VGbJGmxtGlSxeYTCYUFhYCaFhj/sY/xPfDvw9BWqBjx4544403MG/ePNhsNvTp0we/+MUvkJOTg/z8fLhcLmzcuBEA0L9/f78+sm9qLMHBwVi2bBleeOEFqFQq9OjRA//93/8tOm6TmhpHW9Tc9+SNN97AT3/6UzgcDgwZMgTPPPOM6LjNamosBoMB//u//4vFixejrq4OcXFxWLFihei4zWruZ6yyshI6nQ56vV5wyrtr7ufr17/+NebPnw9JktChQwcsX778vvfDJRCIiBSuzU/dEBFR81j0REQKx6InIlI4Fj0RkcKx6ImIFI5FTwFn375997yi4XPPPYezZ882+5iFCxfiww8/vOPHsrOz8e23397TPolaS5s/j57IFz744IPv9fx9+/ahR48erZSG6N6w6Ckg1dbWehaSstlsePPNNzFgwAD89re/xYEDB+ByudC3b18sXrwYYWFhGDduHP7whz8gJSUFK1euxNq1axEaGooHHngAW7ZsQU5ODoCGtcXnzJmDK1euoGfPnvjd736H9evX48SJE1ixYgU0Go3fX0FLysOpGwpIJpMJP/rRj/DFF19gzpw5eO+997By5UpoNBqsW7cOGzZsQExMDH772996PW/nzp1Yt24d1q5di3Xr1nmWkb2hvLwcH330ETZu3Ijy8nJs2rQJmZmZ6N+/P1577TWWPAnBI3oKSImJiZ6lkpOTk/HZZ59h27ZtqKmpwe7duwEADoej0eqH27dvx8SJEz33A8jMzMTevXs9Hx8/fjyCg4MBAD179kRlZaUvhkPULBY9BSSdTuf5u0qlgiRJcLvdWLRoEdLT0wEAVqu10c1EtFotbl01RKPRNPr47Z+XSDRO3RBdN2rUKKxevRp2ux1utxtLlizB22+/7fWY9PR0bNq0CTU1NQDgtVRxczQaDZxOZ6tnJmoJFj3RdS+++CLi4+ORkZGBSZMmQZIkr7sWAUBqaioef/xxzJ49GzNmzEBNTY1nqqY548aNw9tvv43169fLFZ+oSVy9kugeHD9+HN999x2efvppAMBHH32Eo0eP4ve//73YYETNYNET3QOLxYJFixahsLAQKpUKHTt2xLJlyzw3oSfyRyx6IiKF4xw9EZHCseiJiBSORU9EpHAseiIihWPRExEpHIueiEjh/h9TA7ofH2ygdgAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# draw limits of confidence intervall for t-statistic\n",
"plt = sns.histplot(x=\"height\", data=df, bins=5);\n",
"# draw a vertical line to mark the mean \n",
"plt.axvline(165, 0, 1, linewidth=3, color='b');\n",
"# add text\n",
"plt.text(165.1, 0.1,'Mean = 165', rotation=90, color='b');\n",
"# draw a vertical line to mark the lower limit of the confidence intervall\n",
"plt.axvline(164.304164, 0, 1, linewidth=3, color='r');\n",
"# add text\n",
"plt.text(164, 0.15,'Lower limit = 164.30 ', rotation=90, color='r');\n",
"# draw a vertical line to mark the upper limit of the confidence intervall\n",
"plt.axvline(165.695836, 0, 1, linewidth=3, color='r');\n",
"plt.text(165.8, 0.15,'Upper limit = 165.69', rotation=90, color='r');"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEJCAYAAACaFuz/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAW9ElEQVR4nO3de3BU5R3G8Sck4Sbd6RQIocBgESoXAWFsaww2QhG5BDAhwy2FVFohojLEGTATgmkJRhoFFFJmxKE0WmpFTCo4lWskclEu1kKJgLYUJxSWYjOwCQm57ekfyI5rsptNyGHDm+9nhmH33bP7/n7Z3SdnTs6+G2JZliUAgLHaBLsAAIC9CHoAMBxBDwCGI+gBwHAEPQAYrsUFfU1Njc6dO6eamppglwIARggLdgHf5nQ69bOf/Ux79uxRz549g10OguyjaYlyX7umNu3bK+qtTUGtZWrae6qorFWHdqHanBUb1FqklvWzQcvW4vbogW9yX7vm9X8wVVTWev0fbC3pZ4OWjaAHAMMR9ABgOIIeAAxH0AOA4Qh6ADCcrUFfUFCg+Ph4jRs3TsuXL7dzKgCAD7YFfXFxsTIyMrRu3Tpt3bpVn332mQoLC+2aDgDgg20fmNq1a5fGjx+vyMhISdLq1avVrl07r21cLpdcLpfXmNPptKskAGiVbAv6L7/8UuHh4UpOTtaFCxf00EMPaeHChV7b5ObmKicnx64SYLDqGrfCw1rXn5haY89oHrYFfW1trY4ePao33nhDHTt21BNPPKH8/HzFx8d7tklKSlJcXJzX/ZxOpxITE+0qC4YID2ujtHUHgjZ/MObOmh99y+eEGWwL+i5duigqKkrf+973JEmjR4/W8ePHvYLe4XDI4XDYVQIAQDb+MXbkyJHav3+/XC6XamtrtW/fPg0aNMiu6QAAPti2Rz906FD96le/0syZM1VdXa3o6GhNmTLFrukAAD7YukxxQkKCEhIS7JwCANAA/oQPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMBxBDwCGI+gBwHAEPQAYjqAHAMMR9ABgOIIeAAxH0AOA4Qh6ADAcQQ8AhiPoAcBwBD0AGI6gBwDDEfQAYDiCHgAMR9ADgOEIegAwHEEPAIYLs/PBZ82apZKSEoWFXZ9m2bJlGjp0qJ1TAgC+xbagtyxLZ8+e1QcffOAJegDArWfboZszZ85IkubMmaNJkybpj3/8o11TAQD8sG1X2+VyKSoqSkuXLlV1dbVmz56tH/zgB4qOjvbaxuVyed3P6XTaVRIAtEq2Bf2wYcM0bNgwz/WEhAQVFhZ6BX1ubq5ycnLsKgEAIBuD/ujRo6qurlZUVJSk68fsv32sPikpSXFxcV5jTqdTiYmJdpUFAK2ObcfoS0tLlZ2drcrKSpWVlSk/P18PP/yw1zYOh0M9e/b0+hcZGWlXSQDQKtm2Rz9y5EgdO3ZMjz76qNxut2bOnOl1KAcAcGvYet7jwoULtXDhQjunAAA0gE/GAoDhCHoAMBxBDwCGI+gBwHAEPQAYjqAHAMMR9ABgOIIeAAxH0AOA4Qh6ADAcQQ8AhiPoAcBwBD0AGI6gBwDDEfQAYDiCHgAMR9ADgOEIegAwHEEPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMJztQf/b3/5Wqampdk8DAPDB1qD/6KOPlJ+fb+cUAIAGhNn1wJcvX9bq1auVnJysU6dO1buNy+WSy+XyGnM6nXaVBACtkm1B/9xzzyklJUUXLlzwuU1ubq5ycnLsKqHVqK5xKzyMP7eYrjU+z62xZzvYEvRvv/22unfvrqioKOXl5fncLikpSXFxcV5jTqdTiYmJdpRlrPCwNkpbdyDYZdhiQj1jaesOKGt+9C2vJdj8BZ6pz39rfJ7tYEvQ//Wvf9WlS5c0efJkXblyReXl5crKylJaWprXdg6HQw6Hw44SAABfsyXoN27c6Lmcl5enw4cP1wl5AMCtwcEvADCcbX+MvSE+Pl7x8fF2TwMA8IE9egAwHEEPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMBxBDwCGCyjo61uQbMGCBc1eDACg+fld6yYjI0MXL17UJ598opKSEs94TU2NiouLbS8OAHDz/AZ9QkKCvvjiC50+fVqPPPKIZzw0NFT33nuv3bUBAJqB36AfPHiwBg8erAceeECRkZG3qiYAQDMKaJniCxcuaNGiRbpy5Yosy/KMb9u2zbbCAADNI6Cgf+655xQfH6+BAwcqJCTE7poAAM0ooKAPCwvTY489ZnctAAAbBHR6Zb9+/XT69Gm7awEA2CCgPfri4mJNmTJF3//+99WuXTvPOMfoAaDlCyjoU1JS7K4DAGCTgIL+hz/8od11AABsElDQ33///QoJCZFlWZ6zbrp27aoPP/zQ1uIAADcvoKA/deqU53JVVZXee+89/fvf/7atKABA82n06pVt27ZVfHy8Dhw4YEc9AIBmFtAe/eXLlz2XLcvSiRMn5HK57KoJANCMGn2MXpI6d+6sJUuWNHi/V155RTt27FBISIgSEhL40BUABEGjj9EH6vDhw/r444+1detW1dTUaPz48YqJiVGfPn0a/VgAgKYLKOjdbrc2bNigDz/8UDU1NYqOjlZycrLCwnzf/cc//rFef/11hYWF6eLFi6qtrVXHjh2brXAAQGACCvqVK1fq1KlTSkpKktvt1ltvvaXs7Ox6v3nqm8LDw7VmzRr9/ve/19ixY9WtWzev210uV51j/U6ns5EtAAD8CSjo9+3bp3feeUfh4eGSpIceekiTJk1qMOil6185+Pjjjys5OVmbN2/WtGnTPLfl5uYqJyeniaXXr7rGrfAwvgoXMEFrez/b1W9AQW9ZlifkpeunWH7zen3+9a9/qaqqSgMGDFCHDh00ZsyYOgujJSUlKS4uzmvM6XQqMTEx0PrrCA9ro7R1revUz6z50cEuAbBFa3s/2/VeDijo+/fvr6ysLP385z9XSEiI3njjjQaXRTh37pzWrFmjN998U5K0Z88eTZkyxWsbh8Mhh8PRxNIBAIEIKOgzMjK0fPlyTZ8+XW63Ww8++KCWLl3q9z4xMTE6fvy4Hn30UYWGhmrMmDGaMGFCsxQNAAic36CvqqrS0qVLNXr0aK1YsUKSNHfuXIWGhqpTp04NPvjTTz+tp59+unkqBQA0id+j/mvWrFFZWZmGDx/uGcvMzJTL5dLatWttLw4AcPP8Bv3evXu1cuVKde7c2TPWrVs3ZWdna/fu3bYXBwC4eX6DPjw8XO3bt68z3qlTJ7Vt29a2ogAAzcdv0Ldp00ZlZWV1xsvKylRTU2NbUQCA5uM36GNjY5Wenq7y8nLPWHl5udLT0zVmzBjbiwMA3Dy/QZ+UlKTvfOc7io6O1tSpU5WQkKDo6Gg5HA49+eSTt6pGAMBN8Ht6ZZs2bZSZmank5GQVFRWpTZs2GjJkiCIiIm5VfQCAmxTQB6Z69OihHj162F0LAMAGrWe1IABopQh6ADAcQQ8AhiPoAcBwBD0AGI6gBwDDEfQAYDiCHgAMR9ADgOEIegAwHEEPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMFxAXyXYVDk5OXr//fclSTExMVq8eLGd0wEA6mHbHv3Bgwe1f/9+5efn6y9/+YuKioq0a9cuu6YDAPhg2x59165dlZqaqrZt20qS7rrrLp0/f95rG5fLJZfL5TXmdDrtKgkAWiXbgr5fv36ey2fPntX777+vN99802ub3Nxc5eTk2FUCAEA2H6OXpC+++ELz5s3T4sWLdeedd3rdlpSUpLi4OK8xp9OpxMREu8sCgFbD1qD/5JNPtGDBAqWlpWnChAl1bnc4HHI4HHaWAACtnm1Bf+HCBT355JNavXq1oqKi7JoGANAA24J+w4YNqqys1IoVKzxj06dP14wZM+yaEgBQD9uCPj09Xenp6XY9PAAgQHwyFgAMR9ADgOEIegAwHEEPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMBxBDwCGI+gBwHAEPQAYjqAHAMMR9ABgOIIeAAxH0AOA4Qh6ADAcQQ8AhiPoAcBwBD0AGI6gBwDDEfQAYDiCHgAMZ2vQl5WVKTY2VufOnbNzGgCAH7YF/bFjxzRjxgydPXvWrikAAAGwLeg3b96sjIwMRURE2DUFACAAYXY98PPPP9/gNi6XSy6Xy2vM6XTaVRIAtEq2BX0gcnNzlZOTE8wSAMB4QQ36pKQkxcXFeY05nU4lJiYGqSIAME9Qg97hcMjhcASzBAAwHufRA4DhbN+jLygosHsKAIAf7NEDgOEIegAwHEEPAIYj6AHAcAQ9ABiOoAcAwxH0AGA4gh4ADEfQA4DhCHoAMBxBDwCGI+gBwHAEPQAYjqAHAMMR9ABgOIIeAAxH0AOA4Qh6ADAcQQ8AhiPoAcBwBD0AGI6gBwDDEfQAYDiCHgAMR9ADgOEIegAwnK1Bv23bNo0fP15jxozRpk2b7JwKAOBDmF0PfPHiRa1evVp5eXlq27atpk+frp/85Cfq27evXVMCAOphW9AfPHhQ999/v7773e9Kkh555BFt375dTz31lGcbl8sll8vldb///Oc/kiSn09nkua+6vmryfW9H586dM7bnr6qqPJfPnTsn6frzG4yeq8tLPJevuvxsaJMb/d/wzZ+Nqc+/ya/t+nz7OW6KyMhIhYV5R3uIZVnWTT9yPV599VWVl5crJSVFkvT222/r+PHjyszM9Gyzdu1a5eTk2DE9ALRKe/bsUc+ePb3GbNujd7vdCgkJ8Vy3LMvruiQlJSUpLi7Oa6yqqkrFxcW68847FRoa2qg5nU6nEhMTtWnTJkVGRja9+BbAlF5M6UOil5bKlF6aq4/67mtb0EdGRuro0aOe65cuXVJERITXNg6HQw6Ho859+/Tpc9Nzf/s32u3KlF5M6UOil5bKlF7s6MO2s24eeOABffTRRyopKVFFRYV27typn/70p3ZNBwDwwbY9+m7duiklJUWzZ89WdXW1EhISNGTIELumAwD4YFvQS9LEiRM1ceJEO6cAADTAqE/GOhwOPfXUU/Ue97/dmNKLKX1I9NJSmdKLnX3YdnolAKBlMGqPHgBQF0EPAIa7LYO+rKxMsbGxno8Lf/rpp5o6daomTJigZ555RlVffzR89+7dmjx5siZNmqT58+frypUrwSy7XoH2smvXLk2cOFETJkxQamqqZ7ylCLSPG/bu3atRo0YFo9QGBdpLTk6ORo4cqcmTJ2vy5MktcuG+QHs5c+aMZs2apUmTJumXv/zlbfteOXnypOf5mDx5sh588EHFxsYGuXJvgT4nRUVFmjJliiZNmqR58+bVWS6mUazbzN///ncrNjbWGjRokFVcXGyVlpZa0dHR1smTJy3LsqyUlBRr06ZNnnGn02lZlmW9/PLLVmZmZjBLryPQXq5evWqNGDHCunTpkmVZlrVw4ULrz3/+czBL9xJoHzdcunTJGjt2rDVy5MhglexTY3qZN2+e9be//S2Y5foVaC9ut9saM2aMVVhYaFmWZb344otWdnZ2MEuvo7GvMcuyrPLycmvChAnWkSNHglFyvRrTx4wZM6y9e/dalmVZL7zwgrVq1aomz3vb7dFv3rxZGRkZnk/ZHjhwQPfee6/69+8vSUpPT9fDDz+s6upqZWRkqFu3bpKku+++WxcuXAha3fUJtJeOHTuqoKBAXbp0UUVFhf73v/+1qDMMAu3jhvT0dK/F7VqSxvRy4sQJvfrqq5o4caKWLVumysrKoNVdn0B7KSoqUseOHT0faExOTlZiYmLQ6q5PY19j0vX1tn70ox/pvvvuu+X1+tKYPtxut65evSpJqqioUPv27Zs8r63n0dvh+eef97r+5ZdfqmPHjkpJSdGZM2c0fPhwpaamql27dp4f2LVr17R+/XrNmjUrGCX7FGgvkhQeHq7CwkItXrxYERERGjFiRDBKrldj+nj99dc1cOBADR06NBilNijQXq5evaoBAwZo0aJF6t27t1JTU7Vu3TrPIn4tQaC9HDlyRF26dFFaWppOnjypPn36aOnSpUGqun6NeY1JUmlpqTZv3qxt27bd6lL9akwfqampmjNnjrKystShQwdt3ry5yfPednv031ZbW6v9+/frmWeeUV5enioqKrR+/XrP7aWlpZo7d6769+9fZwG1lqahXmJiYnTo0CGNHDlSv/71r4NXaAN89fH5559r586dmj9/frBLDJivXu644w699tpruuuuuxQWFqY5c+aosLAw2OX65auXmpoaHT58WDNmzFB+fr569eqlFStWBLtcvxp6r2zdulWjR49W586dg1hlw3z1ce3aNS1ZskR/+MMftH//fs2cOVPPPvtsk+e57YO+S5cuGjp0qHr16qXQ0FCNGzdOx48flyT997//1cyZM3X33XfX+U3aEvnq5fLly9q/f79nu4kTJ+r06dNBrNQ/X31s375dly5d0pQpUzR37lzP89OS+erl/Pnz2rJli2c7y7LqrAHe0vjqpWvXrurdu7cGDx4sSYqNjfW8h1oqf+976fqJGOPHjw9ihYHx1cfnn3+udu3aeZaNmTZtmg4fPtzkeW77oB8xYoSKioo8x98/+OADDRo0SLW1tUpOTta4ceO0ZMmSOkskt0S+erEsS4sWLdL58+clSdu3b9fw4cODWapfvvpYsGCBduzYoXfffVfr169XRESE/vSnPwW5Wv989dK+fXu9+OKLKi4ulmVZ2rRpU51jxC2Nr16GDRumkpISnTp1SpJUUFCgQYMGBbPUBvnqRbr+S7eoqEjDhg0LZokB8dVH79695XQ6debMGUnX15i/8Yu4KVr2LkgAunfvrmXLlik5OVmVlZUaMGCAnn32WRUUFOizzz5TbW2tduzYIUm65557WvSeva9eOnTooMzMTM2bN08hISHq27evfvOb3wS7XJ989XE78vecLFu2TE888YSqq6s1fPhwPfbYY8Eu1y9fvbRv316/+93vlJ6eroqKCkVGRio7OzvY5frl7zVWUlKi8PBwtWvXLshVNszf6+uFF17QwoULZVmWOnfurKysrCbPwxIIAGC42/7QDQDAP4IeAAxH0AOA4Qh6ADAcQQ8AhiPo0eocOnSo0SsaPv744/rnP//pd5vU1FRt2LCh3ttycnK0e/fuRs0JNJfb/jx64FZ47bXXbur+hw4dUt++fZupGqBxCHq0SuXl5Z6FpCorK7V8+XINGTJEL730ko4cOaLa2loNHDhQ6enp6tSpk0aNGqVXXnlFgwcP1vr167Vlyxbdcccduu+++7Rnzx4VFBRIur62+PTp0/XVV1+pX79+WrlypfLz83XixAllZ2crNDS0xX+CFubh0A1aJafTqV/84hd69913NX36dK1du1br169XaGio8vLytHXrVkVEROill17yut++ffuUl5enLVu2KC8vz7OM7A0XL17Uxo0btWPHDl28eFE7d+5UYmKi7rnnHi1evJiQR1CwR49WqVevXp6lkvv376933nlHe/fuVWlpqQ4ePChJqq6urrP6YWFhocaOHev5PoDExER9/PHHnttHjx6tDh06SJL69eunkpKSW9EO4BdBj1YpPDzcczkkJESWZcntdistLU0xMTGSpKtXr9b5MpGwsDB9c9WQ0NDQOrd/+3GBYOPQDfC1ESNGaNOmTaqqqpLb7dbSpUu1atUqr21iYmK0c+dOlZaWSpLXUsX+hIaGqqamptlrBgJB0ANfmz9/vnr06KG4uDiNHz9elmV5fWuRJEVFRWnq1KmaNm2a4uPjVVpa6jlU48+oUaO0atUq5efn21U+4BOrVwKN8I9//EOffvqpZs+eLUnauHGjjh07ppdffjm4hQF+EPRAI5SVlSktLU1nzpxRSEiIunfvrszMTM+X0AMtEUEPAIbjGD0AGI6gBwDDEfQAYDiCHgAMR9ADgOEIegAw3P8BWvFI3xSMj/oAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# compare CI z-statistic with t-statistic\n",
"# draw limits of confidence intervall\n",
"plt = sns.histplot(x=\"height\", data=df, bins=5);\n",
"# draw a vertical line to mark the mean \n",
"plt.axvline(165, 0, 1, linewidth=3, color='b');\n",
"# add text\n",
"# draw vertical lines to mark the lower/upper limit of the confidence intervall (z)\n",
"plt.axvline(164.348388, 0, 1, linewidth=3, color='w');\n",
"plt.axvline(165.651612, 0, 1, linewidth=3, color='w');\n",
"# draw vertical lines to mark the lower/upper limit of the confidence intervall (t)\n",
"plt.axvline(164.304164, 0, 1, linewidth=3, color='r');\n",
"plt.axvline(165.695836, 0, 1, linewidth=3, color='r');"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Additional measures"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"When fitting models, it is possible to increase the likelihood by adding parameters, but doing so may result in overfitting. \n",
"\n",
"Both the Bayesian information criterion (**BIC**) (also called the Schwarz criterion SBC or SBIC) and Akaike information criterion (**AIC**) attempt to resolve this problem by introducing a penalty term for the number of parameters in the model; the penalty term is larger in BIC than in AIC. \n",
"\n",
"Both measures are an estimator of the **relative quality** of statistical models for a given set of data. Hence, they are used to select the best performing model. \n",
"\n",
"For both measures, **lower** values are better"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Bayesian information criterion (BIC) "
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"41.980585438104086"
]
},
"execution_count": 67,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# BIC\n",
"lm.bic"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Akaike information criterion (AIC) "
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"39.989120890996105"
]
},
"execution_count": 68,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# AIC\n",
"lm.aic"
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"interpreter": {
"hash": "463226f144cc21b006ce6927bfc93dd00694e52c8bc6857abb6e555b983749e9"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": true,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}