python business analytics

answer question in the ipynb file

{
“cells”: [
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “2481e090b06e15168f0f8cff06553adc”,
“grade”: false,
“grade_id”: “cell-80d2c074a80faa82”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“## Homework 9 : Linear Regression \n”,
“\n”,
“#### Due Date: Saturday November 7, 2020 at 11:59PM EST\n”,
“\n”,
“We want to get practice with linear regression models. Along the way, we will need to apply our experience with data processing. \n”,
“\n”,
“The questions guide you step-by-step through the assignment. Please post queries to the #homework-9 channel on Slack.\n”,
“\n”,
“#### Collaboration Policy\n”,
“\n”,
“Data analysis is a collaborative activity. While you may discuss the homework with classmates, you should answer the questions by yourself. If you discuss the assignments with other students, then please **include their names** below.”
]
},
{
“cell_type”: “markdown”,
“metadata”: {},
“source”: [
“**Name:** *list name here*”
]
},
{
“cell_type”: “markdown”,
“metadata”: {},
“source”: [
“**NetId:** *list netid here*”
]
},
{
“cell_type”: “markdown”,
“metadata”: {},
“source”: [
“**Collaborators:** *list names here*”
]
},
{
“attachments”: {},
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “75d9ffd06410619f4a51c29cbd99afbf”,
“grade”: false,
“grade_id”: “cell-946f429ec7438c5b”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“### Rubric\n”,
“\n”,
“Question | Points\n”,
“— | —\n”,
“Question 1.1 | 2\n”,
“Question 1.2 | 1\n”,
“Question 2.1 | 1\n”,
“Question 3.1 | 1\n”,
“Question 3.2 | 1\n”,
“Question 3.3 | 1\n”,
“Question 4.1 | 1\n”,
“Question 4.2 | 1\n”,
“Question 4.3 | 1\n”,
“Question 5.1 | 1\n”,
“Question 5.2 | 1\n”,
“Question 6.1 | 1\n”,
“Question 6.2 | 1\n”,
“Total | 14”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“ExecuteTime”: {
“end_time”: “2018-09-12T05:50:17.975391Z”,
“start_time”: “2018-09-12T05:50:17.067217Z”
},
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “5b6e6cb3e79c22ec8ff30611b5799d6b”,
“grade”: false,
“grade_id”: “cell-b3036c1461e914e2”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# import some packages\n”,
“\n”,
“import numpy as np\n”,
“import pandas as pd\n”,
“\n”,
“import matplotlib.pyplot as plt\n”,
“import seaborn as sns\n”,
“\n”,
“from sklearn.linear_model import LinearRegression\n”,
“from sklearn.model_selection import train_test_split\n”,
“\n”,
“# change some settings\n”,
“\n”,
“import os, sys\n”,
“\n”,
“pd.set_option(‘display.max_rows’, 10)\n”,
“pd.set_option(‘display.max_columns’, 15)\n”,
“plt.rcParams[‘figure.figsize’] = (9,7)\n”,
“\n”,
“# specify location of dataset\n”,
“\n”,
“path_home = os.environ[\”HOME\”] + \”/shared/homework-9\”\n”,
“path_data = f'{path_home}/kenya.csv'”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “4559f53abbe7ebc2d17b29af8f613304”,
“grade”: true,
“grade_id”: “cell-6c405db462cad8c1”,
“locked”: true,
“points”: 0,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert ‘pandas’ in sys.modules and \”pd\” in locals()\n”,
“assert ‘numpy’ in sys.modules and \”np\” in locals()\n”,
“assert ‘matplotlib’ in sys.modules and \”plt\” in locals()\n”,
“assert ‘seaborn’ in sys.modules and \”sns\” in locals()\n”,
“assert ‘sklearn’ in sys.modules and \”LinearRegression\” in locals()”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “9bb877bcb7f4401309799a3dbc272e57”,
“grade”: false,
“grade_id”: “cell-368e3a1e41126a75”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Dr. Elisabeth Svendsen founded The Donkey Sanctuary in 1969. The charity helps to protect donkeys and mules in developing countries from inhumane treatment. Donkeys and mules are important in rural area where animals help to transport crops, water and building materials. \n”,
“\n”,
“The Donkey Sanctuary funded Kate Milner in 2010 to research veterinary clinics in Kenya. Veterinarians need to prescribe medication to ailing donkeys and mules. However dosage depends on the weight of the animal. Since reliable weighing equipment is unavailable in rural areas, Kate Milner developed a model to predict weight from readily available measurements like height and length. \n”,
“\n”,
“Please check the [research report](https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9713.2014.00768.x) for information.\n”,
“\n”,
“### Question 0\n”,
“\n”,
“Before we process the data in Question 1, we need to inspect the dataset. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {},
“outputs”: [],
“source”: [
“donkeys = pd.read_csv(path_data)\n”,
“donkeys”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “46a7a3f2ed2d9ee12a644f5a6e553fa6”,
“grade”: false,
“grade_id”: “cell-72fc0aea58c929a1”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“We should check the size of the dataset to understand possible challenges surrounding large datasets.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “8821836cb79917e692b6d17bddc224b8”,
“grade”: false,
“grade_id”: “cell-cabc36fe40f51963”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys.shape”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “23f9d03e02b4ea24c95135ae3e117eea”,
“grade”: false,
“grade_id”: “cell-d050b68e296063eb”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“The dataset is small with 544 rows and 8 columns. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “80a6efe5f22c7bb8fd1a5d9da4fd0d7a”,
“grade”: false,
“grade_id”: “cell-24130b780ac9f9b5”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys.columns”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “79255da12af69cc0d3cfc522ccab8f49”,
“grade”: false,
“grade_id”: “cell-c406c5dd1251c26a”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“The attributes of the records will help us to determine features for the model. We should understand the column headers.\n”,
“\n”,
“- `BCS` \n”,
” * Veterinarians make a general assessment of physical health called Body Condition Score\n”,
“- `Girth`\n”,
” * Alongside length and height, donkeys have girth meaning the size of the torso\n”,
“- `WeightAlt`\n”,
” * Note that researchers weighed 31 donkeys among the 544 donkeys twice to assess the accuracy of the scale.\n”,
“\n”,
“We have five numerical variables and three categorical variables\n”,
“\n”,
“- Numerical: `Length`, `Girth`, `Height`, `Weight`, `WeightAlt`\n”,
“- Categorical: `BCS`, `Age`, `Sex`\n”,
“\n”,
“Note that we want to determine `Weight` from some combination of the other attributes.\n”,
“\n”,
“### Question 1\n”,
“\n”,
“We will process the data with attention to data cleaning. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “5d3f8357d91638ee5561a197eea17e1e”,
“grade”: false,
“grade_id”: “cell-d025d44f4a114f88”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“difference = donkeys[‘WeightAlt’] – donkeys[‘Weight’]\n”,
“plt.hist(difference.dropna())\n”,
“plt.xlabel(\”Difference in Weight Measurements\”)\n”,
“plt.ylabel(\”Frequency\”);”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “399a6569f6e4c0367451edd90f69ca57”,
“grade”: false,
“grade_id”: “cell-ca77a81172774858”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“By examining `WeightAlt`, we can ensure that the scale was accurate. We took the difference between the two weighings. The histogram shows that the measurements are within 1 kg of each other. Therefore the measurements seem reasonable.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “b8e9de51a220aabb5cd769263caeb9d8”,
“grade”: false,
“grade_id”: “cell-1641f0e813f1b01b”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys.quantile([0.005, 0.995])”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “7cb354e6ad277d82d93216d77c53c128”,
“grade”: false,
“grade_id”: “cell-e0055dabd275cc7f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“We can look for unusual values that might indicate errors in the dataset. We use the `quantile` function to calculate \n”,
“\n”,
“- 0.5\\% quantile \n”,
“- 99.5\\% quantile\n”,
“\n”,
“We should treat values outside of the range like outliers.\n”,
“\n”,
“#### Question 1.1\n”,
“\n”,
“We can determine the donkeys with unusual length.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “ceedb032c75905261b20afa6e8f920b7”,
“grade”: false,
“grade_id”: “cell-23b592953821d59a”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys.loc[(donkeys[‘Length’] < 71.145) | (donkeys['Length'] > 111), ‘Length’]”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “28a71920d409aa06e618cb0f52ff4f5c”,
“grade”: false,
“grade_id”: “cell-745c5af619e74874”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Determine the donkeys with unusual girth.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “a858d927d10ab16e6d7e04eba1d6e2a9”,
“grade”: false,
“grade_id”: “cell-bd6a1f39e8846585”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“unusual_girth = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “59e2b327897b83f4ce5fe3fbccc222b7”,
“grade”: true,
“grade_id”: “cell-0ade5cdad015cc70”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert len(unusual_girth) == 4\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “ceb4f2c9bb88c95fc5a2d797c04753a4”,
“grade”: false,
“grade_id”: “cell-0fafe6ac50b2571f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Determine the donkeys with unusual height.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “7f490280a9700d81820c64b558b60263”,
“grade”: false,
“grade_id”: “cell-9d01bd9197e84cf9”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“unusual_height = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “429183423653a3b7f4b4375b67ffe5b7”,
“grade”: true,
“grade_id”: “cell-96d6dec0c7aed760”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert len(unusual_girth) == 4\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “f26d252f1752c298069aeafc4905d8d8”,
“grade”: false,
“grade_id”: “cell-c607afd65342f079”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“While many of the unusual height, length or girth are close to the 0.5\\% quantile or 99.5\\%, we should treat donkey 8 like an outlier.\n”,
“\n”,
“#### Question 1.2\n”,
“\n”,
“\n”,
“We can determine the donkeys with unusual Body Condition Score.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “f143c39de80a4cd67e27bec43076f982”,
“grade”: false,
“grade_id”: “cell-472b2d0de5989c96”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys[(donkeys[‘BCS’] < 1.5) | (donkeys['BCS'] > 4)][‘BCS’]”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “f4a5bddd731f407238a9df3860673df0”,
“grade”: false,
“grade_id”: “cell-0084b1c47f8780fb”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Remember that `BCS` is an indication of the health of a donkey\n”,
“\n”,
“- `BCS` of 1 represents an emaciated donkey \n”,
“- `BCS` of 4.5 represent an obese donkey\n”,
“\n”,
“Based on `BCS`, we should remove donkey 291 and donkey 445.\n”,
“\n”,
“We can determine the donkeys with unusual weight.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “64280ce15497606fdf6f66674ef78b85”,
“grade”: false,
“grade_id”: “cell-7c5f397d553daf34”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys[(donkeys[‘Weight’] < 71.715) | (donkeys['Weight'] > 214)][‘Weight’]”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “c05084f781b3f8caae577e13ba62f2c1”,
“grade”: false,
“grade_id”: “cell-ede688a7f865264e”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Based on weight, we should remove donkey 8, donkey 26, donkey 291 and donkey 523. Combining the observations about outliers, we can filter the table\n”,
“\n”,
“- Keep donkeys with `BCS` in the range 1.5 and 4\n”,
“- Keep donkeys with `Weight` between 71 and 214 \n”,
“\n”,
“Define `donkeys_clean` to be the table consisting of records with `BCS` and `Weight` in the specified ranges.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “37929c0df24477de9d551ba9921aa63c”,
“grade”: false,
“grade_id”: “cell-3bdb5082194dff83”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys_clean = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “388e59e11d5ed7a5f040cb3789f881ea”,
“grade”: true,
“grade_id”: “cell-0c901994714c952e”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert donkeys_clean.shape == (539, 8)\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “b95ff4fdd4f46823bc51b75afb8f1671”,
“grade”: false,
“grade_id”: “cell-68adfc4f282b91ef”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“### Question 2\n”,
“\n”,
“Before we explore the data in Question 3, we need to divide the dataset \n”,
“\n”,
“- 80% of the dataset to train the model\n”,
“- 20% of the dataset to test the model\n”,
“\n”,
“We want to avoid examining the testing set until Question 6.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “7fcdd3bdd2aa30c9e85d22469c0fbbdd”,
“grade”: false,
“grade_id”: “cell-3a7c7e538952c932”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“X_train, X_test, y_train, y_test = train_test_split(donkeys_clean.drop([‘Weight’, \”WeightAlt\”], axis=1),\n”,
” donkeys_clean[‘Weight’],\n”,
” test_size=0.2,\n”,
” random_state=42)\n”,
“X_train.shape, X_test.shape”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “366b865ae4196e19dd24af499483c8c5”,
“grade”: false,
“grade_id”: “cell-dee94535b1f52da7”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Here we use the `train_test_split` function from the `sklearn` package. Note that we have a seed `42` to ensure reproducible random split.\n”,
“\n”,
“#### Question 2.1 \n”,
“\n”,
“We will need to compare the observed weights in `y_test` and the predicted weights from `X_test`. We will use the mean-square-error. Note that 0 is the smallest possible value of the mean-square-error. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “c00fa765a6368defb180d2fe52558fab”,
“grade”: false,
“grade_id”: “cell-1a72e99c15c2d3cd”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“predictions = y_test.values\n”,
“observations = y_test.values\n”,
“\n”,
“mse = np.sum((predictions – observations)**2)\n”,
“mse”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “41101aa92756c889add40318e68fc6ff”,
“grade”: false,
“grade_id”: “cell-abc5b2b92fc4b145”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Write a function `mse_test_set` with input \n”,
“\n”,
“- array of observations \n”,
“- array of predictions \n”,
“\n”,
“and output the mean-square-error.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “c7704eb95969a1fb79bbd70bf5f0b06f”,
“grade”: false,
“grade_id”: “cell-fcd90abd14c19e8d”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“def mse_test_set(predictions, observations):\n”,
” # YOUR CODE HERE\n”,
” raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “f5d9c8e2d78d2f73736ceef1761d22a2”,
“grade”: true,
“grade_id”: “cell-c2020339c3779020”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert mse_test_set(y_test, y_test) == 0”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “7af5633ed12e53f0cd5fa5fa370a2a3a”,
“grade”: false,
“grade_id”: “cell-662d6d707da4fedf”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“### Question 3\n”,
“\n”,
“Before we determine models in Question 4 and Question 5, we need to explore the data. We will use charts to understand attributes of the dataset. The attributes will inform the features of the model.\n”,
“\n”,
“#### Question 3.1\n”,
“\n”,
“We can examine the categorical variables with boxplots.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “3e171168698ed90ee18c6d5e287ac1b1”,
“grade”: false,
“grade_id”: “cell-5ca0f26c53f79a79”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“sns.boxplot(x=X_train[‘BCS’], y=y_train);”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “de672c8a8bdcffddd300b5d130da0925”,
“grade”: false,
“grade_id”: “cell-9dbf0322574534b1”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“The median weight increases with `BCS`. We learn that BCS and weight have an association.\n”,
“\n”,
“We can generate box-plots for gender and weight. Note that we have three genders\n”,
“\n”,
“- female \n”,
“- stallion \n”,
“- gelding \n”,
“\n”,
“We should compare the median weights to check the association between weight and gender.\n”,
“\n”,
“**True or False** Based on median weights, we do not observe an association between weight and gender.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “fecbf521ad445daf9c323e67173a2826”,
“grade”: false,
“grade_id”: “cell-71786fc59b449a1e”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q3_1 = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “1b2238b6c75e0f0f64932edf8488d040”,
“grade”: true,
“grade_id”: “cell-5eff8223340ae0f1”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert q3_1 in [True, False]\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “13c62719a7833ab620ab789a831a22db”,
“grade”: false,
“grade_id”: “cell-43bb5fb438cf49fe”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“#### Question 3.2 \n”,
“\n”,
“We can generate a strip-plot to understand possible association between age and weight. Note that a strip-plot consists of side-by-side scatter-plots for different values of a categorical variable.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “497552f47ed4522f54304e0a987ecd10”,
“grade”: false,
“grade_id”: “cell-1e80a0502310fc30”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“sns.stripplot(x=X_train[\”Age\”], y=y_train, order=[‘<2', '2-5', '5-10', '10-15', '15-20', '>20′]);”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “412aeb6f79da2d097eb580b4f766b5c9”,
“grade”: false,
“grade_id”: “cell-89d8d7f6c41ddcb1”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“We observe related ranges of weights for some ages. Which of the following ages have the most related ranges of weights. \n”,
“\n”,
“1. ‘<2', '10-15', '15-20', '>20’\n”,
“1. ‘2-5’, ‘5-10′, ’10-15’, ’15-20’\n”,
“1. ‘<2', '2-5', '5-10', '10-15'\n", "1. '5-10', '10-15', '15-20', '>20’\n”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “65b309a565f5f34b552a3f1ae80e3702”,
“grade”: false,
“grade_id”: “cell-9631bda7f7d9ab6e”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q3_2 = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “9e20240e3baee02101dc8aa3e6948223”,
“grade”: true,
“grade_id”: “cell-48d1e833c9cc0822”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert q3_2 in [1,2,3,4]\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “28b077ef2f67c9268cf6688f87ec715f”,
“grade”: false,
“grade_id”: “cell-85065ddaa0e832af”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“#### Question 3.3\n”,
“\n”,
“We can determine the association between weight, length, girth, and height.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “711168b54175ca23839c5a77588742ec”,
“grade”: false,
“grade_id”: “cell-205dfb7ea112d66f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“donkeys[[‘Length’, ‘Girth’, ‘Height’, ‘Weight’]].corr()”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “ce2f3a88365282b580242e56d5d82ceb”,
“grade”: false,
“grade_id”: “cell-7c8fccff6286ecbe”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Which attribute among length, girth and height is most associated to weight?\n”,
“\n”,
“1. length \n”,
“1. girth \n”,
“1. height ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “508ae10bdee107f931e5eec682784871”,
“grade”: false,
“grade_id”: “cell-2145989f74b73c68”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q3_3 = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “1f07039ef22aff46e179bc3bc895c6d8”,
“grade”: true,
“grade_id”: “cell-20c47e592e1ba732”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert q3_3 in [1,2,3]\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “7cc4d3b8dd56e2424b5ff3e22454b483”,
“grade”: false,
“grade_id”: “cell-89869c2c71d66a8f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“### Question 4\n”,
“\n”,
“Before we use the categorical variables in Question 5, we want to determine a linear regression model with the numerical variables.\n”,
“\n”,
“#### Question 4.1\n”,
“\n”,
“We can fit a line to the scatter-plot of length and weight.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “6e6e27672fc6ed661e84124d63778828”,
“grade”: false,
“grade_id”: “cell-649d36c55d99e86c”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“sns.regplot(x = X_train[\”Length\”], y = y_train, fit_reg=True)\n”,
“plt.title(\”Linear Regression Model\”);”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “80f934eee1e087b057004568816edfa9”,
“grade”: false,
“grade_id”: “cell-976748d5faa94a66”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Using `LinearRegression` from the `sklearn` package, we can determine the corresponding predictions on the test set.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “6ba2f654026fbd164874ba45ef3f07e2”,
“grade”: false,
“grade_id”: “cell-658761f5228a55a6”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“model_length = LinearRegression()\n”,
“model_length.fit(X_train[[‘Length’]], y_train)\n”,
“predictions_length = model_length.predict(X_test[[‘Length’]])”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “330ce749c1ca7e6c004278000ce37b01”,
“grade”: false,
“grade_id”: “cell-558fdf17abda63f2”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Having calculated the predictions, we can evaluate the differences between predictions and observations.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “6b7a6e096f91c6d4924be345db019012”,
“grade”: false,
“grade_id”: “cell-f274d757188597e3”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“mse_test_set(predictions_length, y_test)”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “494606f1ea67d06f3ed68d6fa9f68428”,
“grade”: false,
“grade_id”: “cell-3853223cec07b87f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Adapt the code to calculate the mean-square-error for the linear regression model between girth and weight.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “2ad0e7ca072d3eab6959adece7d08538”,
“grade”: false,
“grade_id”: “cell-f623f34d3b928124”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“model_girth = …\n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()\n”,
“\n”,
“predictions_girth = …\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()\n”,
“\n”,
“mse_test_set(predictions_girth, y_test)”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “91a7b4287aecc68740650dd411a90451”,
“grade”: false,
“grade_id”: “cell-e4fce634eee4d30c”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“Adapt the code to calculate the mean-square-error for the linear regression model between height and weight.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “f9e46c744665d8c50b3a5509d22f6637”,
“grade”: false,
“grade_id”: “cell-aff42116758d273f”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“model_height = …\n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()\n”,
“\n”,
“predictions_height = …\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()\n”,
“\n”,
“mse_test_set(predictions_height, y_test)”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “08145e4772f13364307fb37f9e5c5a71”,
“grade”: false,
“grade_id”: “cell-71098c093908bf13”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“What is the lowest mean-square-error among the three models?”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “e0d944e7a748e57e260c1a54c754bb2a”,
“grade”: false,
“grade_id”: “cell-d17214f09163291b”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q4_1 = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “eae4f4dcf6176481c558caad20eb45c4”,
“grade”: true,
“grade_id”: “cell-6bbfc7f108bf53b2”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert 10000 < q4_1 < 50000\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "2d3179c84c8e5c2dfc219cb6d37554cf", "grade": false, "grade_id": "cell-aa7313b4cb873fee", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "#### Question 4.2\n", "\n", "Is your answer to Question 4.1 consistent with your answer to Question 3.3?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "20dd9b6b5693b4329cbb0cb26c5e9e46", "grade": false, "grade_id": "cell-64f18e7ac1d872ef", "locked": false, "schema_version": 3, "solution": true, "task": false } }, "outputs": [], "source": [ "q4_2 = ... \n", "\n", "# YOUR CODE HERE\n", "raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "43a607e66d3d67c7368b42b250524ad9", "grade": true, "grade_id": "cell-dc7cb31ed26e4615", "locked": true, "points": 1, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "# TEST \n", "\n", "assert q4_2 in [True, False]\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "166d40aba5e880745dbeb4bb5d259480", "grade": false, "grade_id": "cell-405229e69acdc039", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "#### Question 4.3 \n", "\n", "We can determine a linear regression model from girth, length and weight." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "6901cafb500a8189298bdb5c0bf63865", "grade": false, "grade_id": "cell-4e5ed16ac6406529", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "model_length_girth = LinearRegression()\n", "model_length_girth.fit(X_train[['Girth', 'Length']], y_train);" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "befbe9fa92b3773a3fa582c6164d7a25", "grade": false, "grade_id": "cell-538d26d506f91966", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "Since we use two features to predict weight, we have an improvement over the models based on one feature.\n", "\n", "Determine the mean-square-error of the model based on girth and length." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "5461df5a51a07f32c1d2ee6f8fc1610e", "grade": false, "grade_id": "cell-cf021cdd0512a2fa", "locked": false, "schema_version": 3, "solution": true, "task": false } }, "outputs": [], "source": [ "q4_3 = ...\n", "\n", "# YOUR CODE HERE\n", "raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "b1cd9e73b2cd40788d641fd959f3db48", "grade": true, "grade_id": "cell-1fbd6299f0a4bced", "locked": true, "points": 1, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "# TEST \n", "\n", "assert 5000 < q4_3 < 10000\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "45d6d689d3f479d6fba59588756ebc68", "grade": false, "grade_id": "cell-88f44c92865e7212", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "### Question 5 \n", "\n", "We want to incorporate categorical variables in the model. Based on Question 3.1, we will omit gender from the model. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "32b16afa1f614066c6dfa8119234d3a5", "grade": false, "grade_id": "cell-75ecc522f021507d", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "X_train = pd.get_dummies(X_train, columns = [\"BCS\", \"Age\"])\n", "X_train" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "fc78ec22cbdd98f1a367294cf3337419", "grade": false, "grade_id": "cell-bbb6299555f63f76", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "Before we incorporate `BCS` and `Age` into the model, we need to encode the categorical data as numerical data. We will use the one-hot encoding discussed in Week 11 of class. Here we add a column for each value of variable\n", "\n", "- 1 indicates the presence of a value \n", "- 0 indicates the absence of a value \n", "\n", "Based on Question 3.2, we will group the ranges of ages\n", "\n", "> ‘5-10′, ’10-15′, ’15-20’, ‘>20’\n”,
“\n”,
“We add a column to the training set.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “10fdc73e5319a12e2bc44ba8715575eb”,
“grade”: false,
“grade_id”: “cell-006bf9b7583317b2”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“X_train[‘Age_>5’] = (X_train[‘Age_5-10’] | X_train[‘Age_10-15’] | X_train[‘Age_15-20’] | X_train[‘Age_>20’]) \n”,
“X_train.drop([‘Age_5-10’, ‘Age_10-15’, ‘Age_15-20’, ‘Age_>20’], axis=1, inplace=True)\n”,
“X_train”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “ad76ffd55f2d209d48179f6dcab54328”,
“grade”: false,
“grade_id”: “cell-77ce599719a370ca”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“#### Question 5.1\n”,
“\n”,
“We have three features for age in the training set\n”,
“\n”,
“- ‘Age_2-5’\n”,
“- ‘Age_<2'\n", "- 'Age_>5’\n”,
“\n”,
“Determine a linear regression model that predicts weight from age. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “100389a2c1702966132389cd90f694c5”,
“grade”: false,
“grade_id”: “cell-7017b26e122a62c4”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“model_age = …\n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “77e11329c52cecfb223abff7d0fb9105”,
“grade”: false,
“grade_id”: “cell-f4d7d33a2a146ec1”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“We can check the mean-square-error on the training set.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “2f8787ed2d591706e3c928f6247d81ee”,
“grade”: false,
“grade_id”: “cell-d05ddb094019a85b”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“predictions_age = model_age.predict(X_train[[‘Age_2-5’, ‘Age_<2', 'Age_>5′]])\n”,
“mse_test_set(predictions_age, y_train)”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “12eeb79869fffa2b181d80462f9db0fd”,
“grade”: false,
“grade_id”: “cell-c8f5e97e95ea489f”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“**True or False** The mean-square-error for the model based on age is larger than the mean-square-errors for the models in Question 4.1 based on length, girth or height? ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “ffd763e688bdfc3d8d1b35802fa71cce”,
“grade”: false,
“grade_id”: “cell-3ef1aa2c3f904d3f”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q5_1 = … \n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “2437bb20384faaf5bedbd092ee5642e9”,
“grade”: true,
“grade_id”: “cell-6ef621fd7be382c6”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert q5_1 in [True, False]\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “3c38b9ef754ab97c550910e72cb8c584”,
“grade”: false,
“grade_id”: “cell-0b1bbc9312b6d003”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“#### Question 5.2\n”,
“\n”,
“\n”,
“\n”,
“We can combine the numerical features and categorical features in a model.\n”,
“\n”,
“- length, height, girth \n”,
“- BCS, age\n”,
“\n”,
“Determine a model to predict weight from these five features. ”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “523a5700274e4374b0653da6d2089c56”,
“grade”: false,
“grade_id”: “cell-95a397cc61808055”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“model = …\n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “f723e6619f5e2937aec39fbb0e47900a”,
“grade”: true,
“grade_id”: “cell-a0cf5e221cd7918c”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert len(model.coef_) == 12\n”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “56040187e38fc65e3469c65e13c5616a”,
“grade”: false,
“grade_id”: “cell-e46bbba143cea891”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“### Question 6\n”,
“\n”,
“Before we can evaluate the model on the testing set, we need to generate the same features.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “3a2588c2173ea0a91dea75344a3f06e5”,
“grade”: false,
“grade_id”: “cell-f750ab609ecae492”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“X_test = pd.get_dummies(X_test, columns = [\”BCS\”, \”Age\”])\n”,
“X_test[‘Age_>5’] = (X_test[‘Age_5-10’] | X_test[‘Age_10-15’] | X_test[‘Age_15-20’] | X_test[‘Age_>20’]) \n”,
“X_test.drop([‘Age_5-10’, ‘Age_10-15’, ‘Age_15-20’, ‘Age_>20’], axis=1, inplace=True)\n”,
“X_test”
]
},
{
“cell_type”: “markdown”,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “markdown”,
“checksum”: “9bf1869e010cbf3804059595577c7378”,
“grade”: false,
“grade_id”: “cell-429ed377783ffcd6”,
“locked”: true,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“source”: [
“#### Question 6.1\n”,
“\n”,
“Using the model from Question 5.2, calculate the mean-square-error on the testing set.”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “831588174faced6b8bca949382f882ce”,
“grade”: false,
“grade_id”: “cell-14bd771947f4cf51”,
“locked”: false,
“schema_version”: 3,
“solution”: true,
“task”: false
}
},
“outputs”: [],
“source”: [
“q6_1 = …\n”,
“\n”,
“# YOUR CODE HERE\n”,
“raise NotImplementedError()”
]
},
{
“cell_type”: “code”,
“execution_count”: null,
“metadata”: {
“deletable”: false,
“editable”: false,
“nbgrader”: {
“cell_type”: “code”,
“checksum”: “ef141e00cc7cc79fc092da2f5842a329”,
“grade”: true,
“grade_id”: “cell-6b421170aeeedc50”,
“locked”: true,
“points”: 1,
“schema_version”: 3,
“solution”: false,
“task”: false
}
},
“outputs”: [],
“source”: [
“# TEST \n”,
“\n”,
“assert 5000 < q6_1 < 10000\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "2229767133751b3b4e35cd97c4bddf58", "grade": false, "grade_id": "cell-f35318898b6a659c", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "#### Question 6.2\n", "\n", "With these predictions, we can calculate the relative differences between observations and predictions\n", "\n", "$\\displaystyle\\frac{\\text{Observed Weight} - \\text{Predicted Weight}}{\\text{Observed Weight}}$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "1185e3c19bf02052363e117aa0e2c874", "grade": false, "grade_id": "cell-7497b8cb7bcc8277", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "relative_difference = (y_test.values - model.predict(X_test.drop(columns = \"Sex\"))) / y_test.values" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "eda7cb904662918388326d4d64169a58", "grade": false, "grade_id": "cell-fafbd08f7eec4bc8", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "Now we can generate a scatter-plot." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "966f6b170ca3e3db1a87ce3f3714000d", "grade": false, "grade_id": "cell-20f4572766e2d12a", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "plt.scatter(np.arange(len(relative_difference)), relative_difference, s=15)\n", "plt.axhline(0)\n", "\n", "plt.title('Relative Difference of Observation and Prediction')\n", "plt.xlabel('Index of row in data')\n", "plt.ylabel('Relative Difference');" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "4c38d907263ed0a0fee5673cd006b6ab", "grade": false, "grade_id": "cell-3f4529ac296ab5cc", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "We learn that the relative difference are between\n", "\n", "1. -5\\% and 5\\%\n", "1. -10\\% and 10\\%\n", "1. -15\\% and 15\\%" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "354be641823b4ccf7ded2de719a4d8cd", "grade": false, "grade_id": "cell-1214003cd7ee2e78", "locked": false, "schema_version": 3, "solution": true, "task": false } }, "outputs": [], "source": [ "q6_2 = ... \n", "\n", "# YOUR CODE HERE\n", "raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "024b72c297fb43cbb2d9d02832712f67", "grade": true, "grade_id": "cell-d5c7cd24963c3ec5", "locked": true, "points": 1, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "# TEST \n", "\n", "assert q6_2 in [1,2,3]\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:mg-gy-8413]", "language": "python", "name": "conda-env-mg-gy-8413-py”
},
“language_info”: {
“codemirror_mode”: {
“name”: “ipython”,
“version”: 3
},
“file_extension”: “.py”,
“mimetype”: “text/x-python“,
“name”: “python”,
“nbconvert_exporter”: “python”,
“pygments_lexer”: “ipython3”,
“version”: “3.7.7”
}
},
“nbformat”: 4,
“nbformat_minor”: 4
}

BCS,Age,Sex,Length,Girth,Height,Weight,WeightAlt
3,<2,stallion,78,90,90,77,NA 2.5,<2,stallion,91,97,94,100,NA 1.5,<2,stallion,74,93,95,74,NA 3,<2,female,87,109,96,116,NA 2.5,<2,female,79,98,91,91,NA 1.5,<2,female,86,102,98,105,NA 2.5,<2,stallion,83,106,96,108,NA 2,<2,stallion,77,95,89,86,NA 3,<2,stallion,46,66,71,27,NA 3,<2,stallion,92,110,99,141,NA 3,<2,female,86,105,92,100,NA 3,<2,female,81,106,92,95,NA 2,<2,female,84,106,96,115,NA 2,<2,stallion,85,101,96,106,NA 2.5,<2,female,87,110,97,112,NA 2.5,<2,stallion,92,100,98,117,NA 2.5,<2,stallion,79,100,90,94,NA 2.5,<2,female,81,109,96,107,NA 2.5,<2,female,83,104,94,102,103 2.5,<2,female,87,112,98,127,NA 2.5,<2,female,82,95,93,90,NA 3,<2,female,78,105,96,108,NA 3,<2,stallion,68,92,86,72,NA 2.5,<2,female,85,100,96,102,NA 2.5,<2,female,79,95,92,86,NA 2.5,<2,female,88,108,106,118,NA 3,<2,stallion,69,90,91,65,NA 2.5,<2,stallion,84,95,96,90,NA 3,<2,female,88,104,97,118,NA 2.5,<2,female,88,108,98,113,NA 3,<2,stallion,85,106,98,117,NA 3,<2,stallion,95,115,99,124,NA 2.5,<2,stallion,72,97,94,89,NA 3,<2,stallion,98,110,101,130,NA 3,<2,female,95,115,106,150,NA 2.5,<2,stallion,80,98,97,87,NA 2.5,<2,female,87,100,98,96,NA 2.5,<2,female,87,108,100,117,NA 3,<2,female,85,96,93,94,NA 2.5,<2,female,85,108,93,105,NA 3,<2,female,80,101,92,94,NA 2.5,<2,female,89,103,100,107,NA 2.5,<2,female,83,108,97,117,NA 2.5,<2,stallion,87,104,96,106,NA 3,<2,female,80,94,89,75,NA 3,<2,female,83,104,96,105,NA 2.5,<2,female,86,108,98,114,NA 3,<2,female,85,105,91,115,NA 2.5,<2,stallion,85,106,99,107,NA 2.5,<2,female,82,102,96,98,98 2,<2,stallion,74,90,89,71,NA 3,<2,stallion,93,112,101,142,NA 3,<2,stallion,96,111,104,146,NA 2,2-5,stallion,79,108,97,106,NA 3,10-15,female,91,121,100,162,NA 3,10-15,female,94,120,101,164,NA 3,10-15,stallion,91,114,99,133,NA 3,10-15,female,90,116,97,152,NA 2.5,5-10,stallion,86,112,98,129,NA 2.5,2-5,female,87,101,93,110,NA 3,10-15,female,98,127,101,181,NA 3.5,2-5,female,101,118,104,160,NA 3,10-15,female,97,123,101,172,NA 3,10-15,female,100,117,97,164,NA 3,10-15,female,111,127,106,192,NA 3,15-20,female,97,117,99,163,NA 3,10-15,female,91,120,102,169,NA 2.5,5-10,female,95,114,100,144,NA 3,10-15,female,98,119,97,165,NA 3,10-15,female,100,114,100,160,NA 3,10-15,stallion,92,113,98,145,NA 2.5,2-5,stallion,82,100,95,110,NA 3,10-15,gelding,96,117,98,154,NA 2.5,2-5,female,90,112,104,133,NA 3,10-15,stallion,92,117,102,152,NA 3,2-5,stallion,86,114,105,145,NA 2.5,2-5,stallion,84,114,98,130,NA 3,5-10,female,103,122,108,183,NA 3,15-20,female,94,122,107,173,NA 3,10-15,female,96,124,106,161,NA 3,10-15,gelding,90,119,101,172,NA 3,5-10,gelding,93,117,101,151,NA 3,15-20,gelding,102,123,102,170,NA 3,15-20,female,97,116,99,159,NA 3,2-5,stallion,92,112,98,142,NA 2.5,>20,gelding,90,114,104,136,NA
3,10-15,female,105,130,107,185,NA
3,5-10,female,91,111,99,143,NA
2.5,2-5,stallion,86,104,94,107,NA
3,5-10,female,91,123,97,173,NA
3,10-15,stallion,94,120,107,161,NA
3,10-15,female,104,122,103,176,NA
3,10-15,female,99,120,100,169,NA
3.5,2-5,female,101,123,98,184,NA
3.5,5-10,female,101,122,101,166,NA
3,5-10,gelding,96,118,101,152,NA
2,2-5,stallion,85,101,97,98,NA
2.5,5-10,stallion,94,108,96,131,NA
2,15-20,stallion,102,106,97,136,NA
2.5,2-5,stallion,89,107,100,109,NA
3.5,2-5,female,107,120,100,165,NA
3,2-5,stallion,90,107,98,138,NA
3,>20,female,96,112,99,137,NA
3.5,10-15,female,102,122,102,177,NA
3,5-10,female,90,113,99,142,NA
3.5,5-10,female,86,120,99,160,NA
2.5,5-10,female,89,110,102,125,NA
3,>20,gelding,92,118,101,158,158
3,10-15,female,106,121,104,178,NA
3,>20,stallion,97,118,103,157,NA
3,>20,stallion,101,117,103,164,NA
3,10-15,female,90,120,104,162,NA
3,10-15,stallion,95,118,107,155,NA
3.5,10-15,female,99,118,102,166,NA
3,10-15,female,102,120,103,170,170
2,>20,stallion,93,115,103,141,NA
3,5-10,female,97,112,102,142,NA
3,10-15,female,103,120,102,165,NA
3,2-5,stallion,94,113,98,146,146
3,>20,female,89,129,102,163,NA
3,10-15,female,96,116,97,170,169
3,10-15,gelding,106,113,103,159,NA
3,10-15,gelding,97,116,103,154,NA
3,10-15,female,106,119,103,165,165
3,10-15,female,93,125,107,167,NA
3,10-15,female,95,121,101,157,157
2,2-5,stallion,93,106,97,114,114
3,10-15,female,102,119,98,181,NA
3,10-15,stallion,106,119,102,170,NA
3,10-15,stallion,102,119,104,168,NA
3,2-5,stallion,93,110,100,146,NA
3,10-15,stallion,98,120,102,163,NA
4,10-15,stallion,97,123,102,190,NA
3,2-5,female,91,116,104,138,NA
3,10-15,stallion,90,121,106,162,NA
3,10-15,gelding,108,123,102,171,NA
2.5,10-15,female,98,120,98,167,NA
2.5,>20,gelding,95,113,97,139,NA
3,10-15,female,105,122,105,188,NA
2.5,10-15,gelding,93,115,106,152,NA
3,>20,female,103,121,106,174,NA
2,15-20,stallion,91,110,99,125,NA
3.5,10-15,female,99,119,105,179,NA
3,5-10,female,97,116,102,167,NA
3.5,2-5,stallion,95,115,101,149,NA
3,10-15,gelding,98,113,95,144,NA
3,5-10,female,105,116,104,158,NA
2.5,5-10,female,93,114,97,144,NA
3,5-10,stallion,99,116,101,145,NA
3,10-15,stallion,90,113,102,140,NA
2.5,10-15,stallion,99,116,105,168,NA
2.5,2-5,female,95,116,102,152,151
3,10-15,stallion,98,115,103,161,160
3,5-10,stallion,91,111,100,155,NA
3.5,10-15,female,102,121,97,179,NA
2.5,5-10,stallion,96,115,98,143,NA
3,5-10,female,98,114,100,150,NA
3.5,10-15,stallion,93,123,102,170,NA
3,10-15,stallion,99,116,105,136,NA
3.5,2-5,female,98,124,105,167,NA
3,10-15,female,99,121,104,169,NA
2.5,10-15,stallion,89,114,102,141,NA
2,10-15,gelding,93,115,100,154,NA
1.5,5-10,female,92,113,97,131,NA
2.5,2-5,female,97,115,106,155,NA
3,10-15,stallion,105,119,102,157,NA
3,10-15,gelding,89,119,103,165,NA
2,2-5,stallion,76,98,98,92,NA
3,15-20,gelding,100,121,101,171,NA
2.5,10-15,stallion,95,119,104,156,NA
3,10-15,female,109,121,104,182,NA
2.5,10-15,stallion,96,115,101,157,NA
3.5,10-15,female,102,125,101,194,NA
3,10-15,gelding,103,124,107,174,NA
3,10-15,female,95,116,99,160,NA
4,10-15,female,96,121,99,178,NA
3,5-10,female,96,117,101,152,NA
3.5,5-10,female,94,118,99,157,NA
3,10-15,female,107,114,98,170,NA
3,10-15,female,97,114,99,142,NA
3,10-15,female,105,118,104,156,NA
3,10-15,stallion,96,120,104,167,NA
3,2-5,stallion,94,114,102,151,NA
3,10-15,stallion,91,116,105,158,NA
2.5,10-15,stallion,97,119,99,149,NA
3,10-15,stallion,91,118,102,146,NA
3,2-5,stallion,99,118,102,139,NA
2,2-5,stallion,93,107,101,133,NA
3,10-15,stallion,103,116,99,144,NA
3,10-15,female,98,124,98,178,NA
2.5,10-15,gelding,93,117,101,159,NA
1.5,>20,stallion,88,107,95,114,NA
3,10-15,stallion,99,110,96,142,NA
3,15-20,gelding,100,122,102,175,NA
2.5,10-15,stallion,92,115,98,145,NA
3,2-5,stallion,96,112,98,150,NA
3,10-15,gelding,101,116,100,155,NA
3,10-15,gelding,85,113,98,140,NA
3,2-5,stallion,85,114,98,140,NA
2.5,10-15,gelding,89,121,106,158,NA
3,10-15,stallion,92,116,105,160,NA
3,10-15,stallion,96,112,100,160,NA
3.5,10-15,female,97,122,103,188,NA
3.5,15-20,gelding,101,122,101,172,NA
2.5,2-5,female,92,117,102,145,NA
3,10-15,gelding,100,116,105,154,NA
3.5,10-15,female,96,116,99,158,NA
2,15-20,female,100,123,103,158,NA
3,10-15,female,102,122,101,165,NA
2.5,10-15,female,94,121,99,149,NA
2.5,10-15,female,91,114,97,128,NA
3.5,5-10,stallion,102,118,103,167,NA
1.5,15-20,gelding,94,111,105,126,NA
2.5,10-15,female,95,120,101,158,158
3,15-20,stallion,101,121,104,168,168
3,2-5,female,86,115,100,134,NA
3,2-5,stallion,112,123,106,186,186
2.5,2-5,stallion,90,118,99,148,NA
3,10-15,stallion,97,120,103,162,NA
2.5,15-20,stallion,89,110,94,134,NA
2.5,10-15,female,98,114,101,136,NA
3,10-15,stallion,99,121,103,168,NA
2.5,10-15,gelding,99,124,108,174,NA
2.5,2-5,stallion,88,114,96,123,NA
3.5,5-10,female,111,128,101,196,NA
3,2-5,stallion,94,116,103,146,NA
3,2-5,stallion,96,112,100,149,149
3,10-15,gelding,84,119,98,148,NA
2.5,10-15,female,98,120,103,170,NA
3,2-5,stallion,98,115,101,155,NA
3,10-15,stallion,101,116,100,165,NA
3,10-15,female,96,120,102,166,NA
3,10-15,stallion,101,122,102,165,NA
3,10-15,stallion,101,119,101,164,NA
3,10-15,gelding,103,123,105,178,NA
3,15-20,gelding,96,122,102,170,NA
2.5,10-15,stallion,95,115,99,142,NA
3.5,15-20,female,101,120,100,173,NA
2.5,5-10,female,93,111,94,127,NA
2.5,10-15,gelding,111,132,111,204,NA
2.5,10-15,female,101,120,102,155,NA
3,10-15,stallion,99,114,103,162,162
2.5,10-15,stallion,93,116,105,137,NA
3,10-15,stallion,96,121,103,171,NA
3.5,10-15,stallion,108,128,113,212,NA
3,10-15,female,104,117,103,165,NA
2.5,5-10,female,108,117,98,157,NA
3,5-10,female,101,115,101,148,NA
3,15-20,female,98,121,103,178,NA
2,5-10,female,96,116,102,151,NA
3.5,5-10,stallion,99,114,99,150,NA
3,10-15,female,101,125,103,190,NA
3,15-20,stallion,107,130,105,178,NA
3,10-15,gelding,102,123,103,188,NA
3,10-15,gelding,97,115,100,158,NA
3.5,10-15,stallion,100,125,107,195,NA
3,10-15,stallion,89,120,99,163,NA
3,10-15,stallion,102,115,99,149,149
4,10-15,stallion,110,125,105,202,NA
2,10-15,female,88,110,101,116,NA
3,10-15,female,107,121,100,165,NA
2.5,10-15,stallion,98,117,105,145,NA
3,10-15,female,93,120,99,147,NA
3,5-10,stallion,90,118,99,144,144
2.5,2-5,female,92,105,96,121,NA
4,10-15,stallion,106,126,103,200,NA
3,>20,female,102,121,102,179,NA
2,10-15,stallion,97,112,100,138,NA
2,15-20,female,93,117,98,136,NA
2.5,>20,female,101,118,104,143,NA
2.5,2-5,female,92,113,97,140,140
3,5-10,stallion,93,116,98,149,NA
3.5,5-10,female,95,123,99,166,166
3.5,10-15,female,104,121,99,173,NA
2.5,10-15,stallion,101,120,106,146,NA
2.5,10-15,female,96,116,100,143,NA
2.5,2-5,stallion,85,105,97,114,NA
3,15-20,stallion,99,115,101,145,NA
3,10-15,stallion,102,120,104,179,NA
3,10-15,stallion,90,123,102,174,NA
3,10-15,female,103,122,100,162,NA
3,10-15,female,95,116,101,158,NA
4,10-15,stallion,105,125,110,199,NA
4,10-15,gelding,101,134,110,210,NA
3,15-20,gelding,98,120,101,171,NA
2.5,2-5,female,88,111,102,130,NA
2.5,2-5,stallion,86,109,89,125,NA
3,10-15,female,93,122,102,156,NA
3,5-10,gelding,98,128,104,177,NA
3.5,10-15,gelding,99,125,104,175,NA
3,10-15,stallion,93,112,99,144,NA
4.5,10-15,female,107,130,106,227,NA
3,2-5,stallion,96,116,103,154,NA
3,15-20,gelding,103,119,99,166,NA
3,10-15,female,101,123,108,180,NA
3,2-5,female,95,110,97,130,NA
3,5-10,female,104,117,103,155,NA
3,10-15,female,102,120,99,162,NA
3.5,10-15,stallion,100,121,103,171,NA
3,5-10,gelding,100,115,101,145,NA
3,5-10,female,100,118,104,176,NA
2,10-15,female,98,111,101,138,NA
3,10-15,female,92,120,100,149,NA
3,10-15,female,96,118,97,135,NA
3,10-15,gelding,92,120,106,156,NA
3,5-10,female,91,116,102,157,NA
3.5,10-15,stallion,91,122,106,166,NA
3,2-5,female,98,123,105,160,NA
3,2-5,female,98,109,97,126,NA
3.5,10-15,female,98,120,101,177,NA
2.5,>20,female,99,119,101,156,NA
3,5-10,stallion,88,122,102,145,NA
3,10-15,female,100,115,101,136,NA
3,10-15,female,98,112,99,152,NA
3,10-15,stallion,90,118,103,154,NA
3,2-5,stallion,87,111,98,126,126
2.5,2-5,female,93,115,103,139,NA
3.5,10-15,stallion,99,117,100,160,NA
2.5,>20,stallion,95,111,100,134,134
3,2-5,stallion,93,107,100,120,NA
3,10-15,female,107,123,102,177,NA
3,2-5,female,97,113,103,130,NA
3,10-15,female,98,126,101,163,NA
3,10-15,female,97,119,103,159,NA
2.5,10-15,female,100,124,108,153,NA
2.5,2-5,female,86,112,102,130,NA
3,>20,female,101,117,105,162,NA
3,10-15,stallion,93,123,112,168,NA
3,5-10,stallion,100,115,105,164,NA
3.5,5-10,female,98,115,103,167,NA
3,10-15,gelding,94,111,99,150,NA
2.5,5-10,stallion,88,111,101,134,NA
3.5,2-5,gelding,100,113,105,154,NA
3,2-5,stallion,94,120,107,169,NA
2.5,5-10,female,104,113,105,143,NA
3,2-5,female,94,117,107,167,NA
3,5-10,female,99,119,102,166,NA
3,2-5,female,92,107,94,122,NA
3.5,10-15,gelding,99,122,102,178,NA
2.5,>20,female,102,121,108,170,NA
3,5-10,female,102,113,102,138,NA
3,15-20,female,107,121,107,165,NA
3,>20,gelding,98,117,106,160,NA
2.5,15-20,gelding,103,116,107,155,NA
2,5-10,stallion,93,106,102,120,NA
2.5,2-5,stallion,100,112,107,149,NA
2.5,15-20,female,95,111,103,125,NA
2,15-20,stallion,92,110,102,126,NA
2.5,2-5,stallion,92,110,99,122,NA
3,10-15,female,92,116,103,137,NA
2.5,10-15,female,100,117,100,139,NA
3.5,10-15,female,103,120,98,178,NA
3,10-15,stallion,94,114,101,145,NA
3,10-15,female,92,111,101,146,NA
2.5,>20,female,103,123,105,173,NA
3.5,10-15,gelding,92,124,107,172,NA
3,5-10,female,98,113,104,144,NA
3,10-15,gelding,95,119,102,158,NA
3,10-15,stallion,86,113,100,135,NA
2.5,10-15,female,95,114,99,130,NA
3,10-15,female,95,122,109,174,NA
3,5-10,stallion,81,111,101,126,NA
3,5-10,gelding,83,115,96,129,NA
3,10-15,female,91,112,100,127,126
3,>20,female,97,118,99,161,NA
3,5-10,stallion,99,119,104,168,NA
3,10-15,gelding,101,125,107,183,NA
2.5,2-5,female,86,105,100,122,NA
4,10-15,female,96,130,105,184,NA
3.5,2-5,stallion,96,115,104,160,NA
3.5,10-15,stallion,92,115,99,148,NA
2.5,10-15,stallion,97,116,102,146,NA
2,>20,gelding,94,116,106,160,NA
2.5,>20,female,96,112,103,137,NA
3,15-20,gelding,101,123,110,184,183
3,5-10,female,90,113,104,141,NA
3,5-10,stallion,93,116,100,142,NA
2.5,10-15,female,97,113,99,147,NA
2,>20,female,97,114,104,146,NA
2,2-5,stallion,97,111,102,132,NA
3,2-5,stallion,92,111,96,142,NA
2.5,2-5,female,86,106,89,114,NA
3,5-10,stallion,89,116,103,158,NA
2,10-15,stallion,94,111,100,148,NA
3,15-20,stallion,97,118,103,143,NA
3.5,10-15,stallion,98,116,99,152,NA
2.5,2-5,female,93,106,102,113,NA
2.5,10-15,stallion,96,109,98,133,NA
3,2-5,female,106,118,105,159,NA
3.5,10-15,female,108,121,101,183,NA
3,10-15,gelding,99,123,106,171,NA
2.5,2-5,female,96,113,101,136,NA
3,>20,female,103,126,101,180,NA
3,15-20,female,100,114,100,167,NA
3,5-10,gelding,95,117,101,132,NA
3.5,5-10,gelding,106,124,106,184,NA
3,10-15,female,108,115,100,150,NA
3,>20,gelding,101,117,100,147,NA
2.5,15-20,gelding,94,117,104,135,NA
3.5,10-15,female,99,125,103,180,NA
3,5-10,stallion,94,119,105,166,NA
3,>20,female,105,117,97,160,NA
3,>20,gelding,103,126,106,175,NA
2.5,2-5,stallion,93,104,98,114,NA
2.5,10-15,female,103,119,107,166,NA
3,10-15,female,103,115,103,181,NA
3,5-10,stallion,90,115,99,129,NA
2.5,10-15,stallion,100,114,102,154,NA
3,10-15,female,104,112,100,174,NA
3,10-15,gelding,98,120,106,165,NA
2.5,10-15,female,90,116,99,141,NA
3,10-15,stallion,100,115,103,156,NA
2.5,2-5,stallion,90,113,105,142,143
2.5,2-5,female,78,107,90,96,NA
3,10-15,female,90,119,101,156,NA
3.5,>20,gelding,101,121,104,178,NA
3,5-10,female,94,109,98,141,NA
3,15-20,gelding,100,125,106,175,NA
3,2-5,stallion,100,125,110,183,NA
3,2-5,female,99,118,102,151,NA
3,10-15,female,91,127,101,161,NA
4,5-10,stallion,100,128,104,183,NA
3.5,15-20,gelding,87,122,102,163,NA
3,10-15,stallion,93,126,98,161,NA
3,15-20,stallion,90,117,103,162,NA
3,>20,gelding,98,129,106,173,NA
3,2-5,female,85,113,102,132,NA
2.5,10-15,stallion,99,114,100,143,NA
2,2-5,female,99,108,100,122,123
3,10-15,stallion,89,119,105,174,NA
2.5,2-5,stallion,99,105,99,115,NA
3,2-5,female,95,112,102,144,144
3.5,10-15,female,102,126,102,177,NA
4,10-15,female,104,118,100,175,NA
2.5,>20,gelding,95,121,102,164,NA
4,10-15,female,106,124,107,204,NA
2.5,10-15,female,87,115,98,133,NA
3,10-15,female,104,122,101,163,NA
3,2-5,stallion,97,103,94,133,NA
2.5,2-5,female,97,120,112,160,NA
2.5,10-15,female,103,111,100,140,NA
3,10-15,gelding,94,112,100,151,NA
2.5,2-5,female,85,112,98,132,NA
3,10-15,female,93,120,98,143,NA
3,10-15,stallion,94,118,105,158,NA
1,>20,female,97,109,102,115,NA
3,2-5,stallion,100,120,98,147,NA
3,10-15,female,94,113,103,146,NA
3,10-15,gelding,98,120,103,158,157
3,10-15,female,98,118,101,152,NA
2.5,10-15,female,106,125,103,165,NA
3,>20,female,94,120,104,152,NA
2.5,5-10,female,98,120,108,161,NA
2.5,2-5,stallion,93,120,100,144,NA
2.5,2-5,female,99,111,101,139,NA
3,10-15,stallion,100,120,103,174,NA
3,5-10,stallion,100,120,107,164,NA
3,>20,gelding,95,120,103,153,NA
2.5,10-15,stallion,98,115,104,153,NA
2.5,>20,female,94,115,100,142,NA
2.5,>20,female,91,113,99,150,NA
3,10-15,female,101,117,103,180,NA
3,10-15,stallion,103,120,105,173,NA
2.5,2-5,female,94,100,97,126,NA
3,10-15,gelding,99,116,103,154,NA
3,2-5,female,98,113,99,130,NA
3,5-10,female,104,117,104,171,172
3,5-10,female,101,121,101,172,NA
3,10-15,female,100,119,102,168,NA
3,10-15,gelding,107,113,109,196,NA
3,2-5,female,98,113,101,132,NA
3.5,10-15,female,105,125,105,192,NA
3,5-10,gelding,101,119,108,159,NA
3.5,10-15,gelding,98,123,106,173,NA
3.5,15-20,female,104,112,105,185,NA
3.5,10-15,female,99,120,103,181,NA
2.5,10-15,female,92,112,97,140,NA
2.5,10-15,female,103,122,104,185,NA
2.5,2-5,female,88,108,97,119,NA
3,2-5,female,92,116,103,153,NA
3,2-5,female,94,112,101,142,NA
3,10-15,gelding,104,118,106,173,NA
3,10-15,stallion,95,122,105,170,NA
3,5-10,female,97,121,102,177,NA
3.5,10-15,gelding,103,128,106,194,194
2.5,10-15,stallion,98,115,103,143,143
3,2-5,stallion,102,111,99,145,NA
2.5,2-5,stallion,97,113,102,142,NA
3,2-5,stallion,101,116,99,159,NA
3,15-20,stallion,99,118,106,184,NA
2.5,>20,stallion,98,119,103,156,NA
3,5-10,female,101,120,102,179,NA
2.5,10-15,stallion,96,116,103,157,NA
2.5,10-15,stallion,97,125,104,179,NA
3,>20,stallion,90,123,104,163,NA
3,15-20,stallion,108,123,104,177,NA
3,2-5,stallion,94,117,101,143,NA
3,5-10,stallion,101,123,105,177,NA
3,15-20,stallion,102,118,104,174,NA
2.5,10-15,female,95,116,100,143,NA
3,15-20,stallion,99,126,108,193,NA
3,5-10,stallion,97,123,102,183,NA
2.5,5-10,stallion,100,123,105,183,NA
3,10-15,female,104,118,100,181,NA
3,10-15,stallion,93,122,103,170,NA
3,10-15,female,104,122,105,189,NA
3,15-20,stallion,104,131,108,193,NA
4,10-15,female,104,127,102,191,NA
3.5,10-15,stallion,105,128,105,195,NA
3,10-15,gelding,95,118,101,163,NA
2.5,10-15,stallion,91,118,108,153,NA
3,15-20,stallion,101,121,102,183,NA
3,2-5,stallion,100,115,102,156,NA
4,10-15,gelding,106,121,106,185,NA
3,2-5,stallion,101,119,103,170,NA
3,5-10,stallion,98,119,106,170,NA
3.5,15-20,gelding,106,129,110,214,NA
3,5-10,gelding,105,121,107,179,NA
3,10-15,stallion,107,115,103,173,NA
3,2-5,female,99,114,101,147,NA
3,15-20,female,107,118,106,171,NA
3,10-15,stallion,105,122,109,189,NA
3.5,10-15,female,109,128,108,214,NA
4,10-15,gelding,108,134,116,230,NA
3,2-5,stallion,91,117,103,145,NA
3,10-15,stallion,97,116,104,162,NA
3,5-10,stallion,98,121,107,169,NA
3.5,10-15,stallion,99,124,103,178,NA
3.5,>20,stallion,99,119,105,177,NA
3,2-5,stallion,89,121,106,151,NA
2.5,10-15,stallion,97,119,109,172,NA
3,15-20,female,109,118,105,180,NA
3,15-20,gelding,107,123,105,187,NA
3,2-5,female,91,111,103,132,NA
3,10-15,stallion,97,118,107,167,NA
3,5-10,stallion,99,114,100,152,NA
3,>20,gelding,100,117,103,165,NA
4,>20,gelding,103,128,108,213,NA
3.5,2-5,stallion,100,124,102,189,NA
3,10-15,stallion,98,115,101,145,NA
3,10-15,stallion,102,126,110,183,NA
2.5,10-15,stallion,103,118,103,174,NA
3,2-5,stallion,91,112,100,139,NA
3,5-10,stallion,104,124,110,189,NA