{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ul-YGtvEgwd1" }, "source": [ "# What is Machine Learning?\n", " \n", "According to Tom Mitchell in his seminal book:\n", "\n", "> \"A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure $P$, if its performance at tasks in $T$, as measured by $P$, improves with experience $E$.\"\n", "\n", "Example: playing checkers.\n", "\n", " * $E$ = the experience of playing many games of checkers\n", " * $T$ = the task of playing checkers.\n", " * $P$ = the probability that the program will win the next game.\n", "\n", "In general, any machine learning problem can be assigned to one of two broad classifications:\n", "\n", " * Supervised learning (this module) \n", " * Unsupervised learning (next module)\n", "\n", "## Supervised Learning\n", "\n", "In supervised learning, we are given a data set and, for **this** data set (named training data set), we **know** what our correct output should look like. The objective is to learn the relationship between the input and the output.\n", "\n", "Supervised learning problems are categorized into \"regression\" and \"classification\" problems: \n", " * In a **regression** problem, we are trying to predict results within a continuous output, meaning that we are trying to map input variables to some continuous function. \n", " * In a **classification** problem, we are instead trying to predict results in a discrete output. In other words, we are trying to map input variables into discrete categories.\n", "\n", "### Example 1:\n", "\n", " * Given data about the size of houses on the real estate market, try to predict their price. Price as a function of size is a continuous output, so this is a **regression** problem.\n", "\n", " * We could turn this example into a classification problem by instead making our output about whether the house \"sells for more or less than the asking price.\" Here we are **classifying** the houses based on price into two discrete categories.\n", "\n", "### Example 2:\n", "\n", " * **Regression** - Given a picture of a person, we have to predict their age on the basis of the given picture\n", " * **Classification** - Given a patient with a tumor, we have to predict whether the tumor is malignant or benign based on an x-ray image.\n", "\n", "## Linear regression\n", "\n", "The goal of linear regression is to find a line that fits our data as best as possible. Unfornutaley, we don't have time to look at the math details, but you can find them in [this book](https://web.stanford.edu/~hastie/ElemStatLearn/). We'll try to pass the intuituin behind this method.\n", "\n", "Let's assume that we have a set of pairs $(x_i,y_i)$, where $x$ is the independent variable and $y$ the independent variable. Normaly, $x$ is a $m$-dimenstional vector, where we have $m$ variables describing the instance (e.g., the features computed from the molecules by some feature descriptor). However, for simplicity, we assume for now that $x$ is unidimensional. The independent variable is the one we want to predict (e.g., the energy of the molecule). The picture bellow illustrates this: \n", "\n", "\n", "\n", "\n", "The yellow line in the figure corresponds to a model that can be used to predict the value of $y$ from $x$. How do we measure the quality of this model? We need a way to measure the **difference** between real and predicted values (an **error measure**, also known as cost or loss function). Then, we can transform the problem of fiting the best line to the data in a problem of minimizing the error between real values and predictions. \n", "\n", "A common error measure is the **squared loss**. Let $pred_i$ be the value predicted for instance $i$. The squared loss can be computed as:\n", "\n", "$$ E = \\sum_{i=1}^n (pred_i - y_i)^2 $$\n", "\n", "This function basically compute the difference between the real value and the prediction for all the instances:\n", "\n", "![](https://media.giphy.com/media/VbnQM59vjG6fYPmBIl/giphy.gif)\n", "\n", "And then square these values and sum over all the training data:\n", "\n", "![](https://media.giphy.com/media/YlHI3bh3u6hf8oADOq/giphy.gif)\n", "\n", "The resons for squaring is three fold:\n", "\n", " 1. It eliminates the negative signs. Both positive and negative error should count for the loss, and squaring turn a negative error in a positive error\n", " 1. It amplifies the errors with greater magnitude, thus larger errors counts more than small ones.\n", " 1. The function is continue and differentiable (we will see the importance of this when studying the gradient descent method). \n", "\n", "## Least Squares \n", "\n", "Given an error (loss) function, how do we find the line that minimizes it? Recall that we can represent a line by $\\theta_0 + \\theta_1 x$ (alternative notatios are $b + mx$ or $b+ ax$), where $\\theta_0$ is the intercep and $\\theta_1$ the slope. Thus, our objective is to find the line \n", "\n", "$$ pred_i = \\theta_0 + \\theta_1 x_i$$\n", "\n", "that minimizes $E$.\n", "\n", "Let's introduce an algebra trick to facilitate the math. Let $\\mathbf{x_i} = [1, x_i]$, where we added a dummy variable which which value is allways 1, and $\\mathbf{\\theta} = [\\theta_0, \\theta_1]$. Thus, we can write the prediction line as:\n", "\n", "$$ pred_i = \\mathbf{\\theta} ^\\intercal \\mathbf{x_i} = \\begin{bmatrix}\n", " \\theta_0 \\\\\n", " \\theta_1\n", " \\end{bmatrix} [1, x_i] = \\theta_0\\cdot 1 + \\theta_1 x_i = \\theta_0 + \\theta_1 x_i$$\n", "\n", "Stacking a set of intances, we can form a matrix $X$ where each row $i$ contains a vector $\\mathbf{x_i}$. Then, we can compute all predictions at once by computing $X\\mathbf{\\theta}$. The error function can then be computed as:\n", "\n", "$$ E(\\mathbf{\\theta}) = (X\\mathbf{\\theta} -y)^\\intercal(X\\mathbf{\\theta} -y)$$\n", "\n", "To minimize the error we can compute its derivate:\n", "\n", "$$ \\frac{\\partial E(\\mathbf{\\theta})}{ \\partial \\mathbf{\\theta}} = 2X^\\intercal X \\mathbf{\\theta} - 2X^\\intercal y$$\n", "\n", "By taking $\\frac{\\partial E}{ \\partial \\mathbf{\\theta}} = 0$ and solving for $\\mathbf{\\theta}$, we get\n", "\n", "$$ \\mathbf{\\theta} = (X^\\intercal X)^{-1} \\cdot X^\\intercal y$$\n", "\n", "This is known as the normal equation of the linear regression. Although we have described it in terms of a single variable, it can be easily extended to multidimensional data.\n", "\n", "The code sequence below illustrates the use of python package sklearn to compute a linear regression for the [diabetes data set](https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html). \n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": {}, "colab_type": "code", "id": "yXrXbSLGCwua" }, "outputs": [], "source": [ "## First, we load the packages \n", "\n", "# plotting\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "# data manipulation\n", "import numpy as np\n", "\n", "# dataset and algorithm \n", "from sklearn import datasets, linear_model\n", "\n", "# quality measures\n", "from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": {}, "colab_type": "code", "id": "_OfeNTZ9EaDZ" }, "outputs": [], "source": [ "# The, we load the diabetes dataset\n", "# This is a sample data set provided by sklearn\n", "# You can use our own, as long as X is a matrix and y a vector with the same number of rows as X\n", "diabetes_X, diabetes_y = datasets.load_diabetes(return_X_y=True)\n", "\n", "# Use only one feature (this is not necessary, just for plotting)\n", "diabetes_X = diabetes_X[:, np.newaxis, 2]\n", "\n", "# Split the data into training/testing sets (we will discuss this latter on)\n", "diabetes_X_train = diabetes_X[:-20]\n", "diabetes_X_test = diabetes_X[-20:]\n", "\n", "# Split the targets into training/testing sets (we will discuss this latter on)\n", "diabetes_y_train = diabetes_y[:-20]\n", "diabetes_y_test = diabetes_y[-20:]" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": {}, "colab_type": "code", "id": "0lphca4FElXL" }, "outputs": [], "source": [ "# Create linear regression object\n", "regr = linear_model.LinearRegression()\n", "\n", "# Train the model using the training sets\n", "regr.fit(diabetes_X_train, diabetes_y_train)\n", "\n", "# Make predictions using the testing set\n", "diabetes_y_pred = regr.predict(diabetes_X_test)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "InpZNvsXejqs" }, "source": [ "One common measure of predictive performance of the algorithm is to compare the predictions, $pred_i$, to the true values $y_i$. A commonly used measure for this is the sum of the mean square-error (MSE) on the test set:\n", " \n", "$$ MSE= \\frac{1}{N_\\mathrm{test}} \\sum_{i=1}^{N_\\mathrm{test}}(y_i - pred_i)^2 $$\n", "\n", "Another common measure is the mean absolute-error (MAE)\n", "\n", "$$ MAE= \\frac{1}{N_\\mathrm{test}} \\sum_{i=1}^{N_\\mathrm{test}}|y_i - pred_i| $$\n", "\n", "A measure that is independent of the scale of the [coefficient of determination](https://pt.wikipedia.org/wiki/Coeficiente_de_determina%C3%A7%C3%A3o). The best possible possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y (the mean of y), disregarding the input features, would get a $R^2$ score of 0.0.\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 104 }, "colab_type": "code", "id": "zX2MyDz9EuqI", "outputId": "3ba050c9-66d8-4383-ec36-76d12d3f787f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Intercept: 152.92 \n", " Coefficients: 938.24 \n", "\n", "Mean squared error: 2548.07\n", "Coefficient of determination: 0.47\n" ] } ], "source": [ "# The intercept and coefficients\n", "print('Intercept: {0:.2f} \\n Coefficients: {1:.2f} \\n'.format(regr.intercept_, regr.coef_[0]))\n", "\n", "# The mean squared error\n", "print('Mean squared error: %.2f'\n", " % mean_squared_error(diabetes_y_test, diabetes_y_pred))\n", "\n", "# The coefficient of determination: 1 is perfect prediction\n", "print('Coefficient of determination: %.2f'\n", " % r2_score(diabetes_y_test, diabetes_y_pred))" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 252 }, "colab_type": "code", "id": "Cg54EcTKE5te", "outputId": "b94ffb91-4242-4a0f-8be0-426d444ed7f0" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADrCAYAAABXYUzjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAQOElEQVR4nO3dfawcVf3H8c9sH7QLtBYKaiw7g8RKLYJArcZfVHzC538MauJaY3zYGAIhklAjm2g0WWL1LyD406XGGO/8oxJNxJiUWokx0WgrJBahhMjuLRpMW0HabC992PGP4969vffuzky7s2fmzPuV9A+G0+bbXPjkm+85c8aLokgAgOmr2C4AAMqKAAYASwhgALCEAAYASwhgALCEAAYAS1amWbxhw4YoCIKMSgEAN+3fv/9IFEWXLn6eKoCDINC+ffsmVxUAlIDned3lnjOCAABLCGAAsIQABgBLCGAAsIQABgBLCGAATgvDUEEQqFKpKAgChWFou6R5qY6hAUCRhGGoRqOhXq8nSep2u2o0GpKker1uszRJdMAAHNZsNufDd6DX66nZbFqq6GwEMABnzc7Opno+bQQwAGfVarVUz6eNAAbgrFarpWq1etazarWqVqtlqaKzEcAAnFWv19Vut+X7vjzPk+/7arfbudiAkyQvzUc5t27dGnEZDwCk43ne/iiKti5+TgcMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwACc9dxz0nXXSZ5nfn3/+7YrOhsBDCCXwjBUEASqVCoKgkBhGCb+vb/8pQncV79aeuyx4fMvfSmDQs/DStsFAMBiYRiq0Wio1+tJkrrdrhqNhiSpXq8v+3tOnpRuuUX6wQ9G/7n33DPxUs+LF0VR4sVbt26N9u3bl2E5ACAFQaBut7vkue/76nQ6Zz178knpbW+Tnn9+9J935ZXS3r1SrTbhQhPyPG9/FEVbFz9nBAEgd2ZnZ2Off+97ZsywefPo8L3zTun0aenpp+2F7ziMIADkTq1WW7YD3rhxi266SXr44fG//5FHpHe+M5vaJokOGEDutFotVavVBU/+T1KkQ4f+OjJ83/1u0wlHUTHCV6IDBpBD9Xpd/b6nL3xhi06evHbs2vvuk269dUqFTRgBDCBXnnhCesMbJOlTI9esXSv94Q+DdcXFCAJALnzjG2ZTbVyofvaz0tyc9J//FD98JTpgABYdPy5t2CC99NL4dd/6lvSVr0ynpmmiAwZy6HzeAiuC3/zGdLsXXTQ+fA8eNJtqLoavRAADuTN4C6zb7SqKovm3wIoewlEkfeITJnjf+97R697xDunMGbN+06bp1WcDb8IBOZPmLbAi+Mc/pI0b49f99KfSzTdnX48NvAkHFESSt8CKYNcu0+3Ghe+RI6bbdTV8xyGAgZypjXhndtTzPDl1SrrqKhO8X/zi6HW33GJCN4qkSy6ZXn15QwADObP0LTCpWq2q1WpZqijeo4+a0F292mycjfLHP5rQvf/+6dWWZwQwkDP1el3tdlu+78vzPPm+r3a7PfIaRpvuvNME7/XXj15Tq5mzu1EkveUt06utCNiEA5DKCy9I69fHr7v3Xum227KvpwhGbcLxIgaARB56SProR+PXPfOMFASZl+MERhAARooi6YMfNGOGceH74Q9L/b5ZT/gmRwcMYIlOR7riivh1Dz1kwhfnhg4YwLx77zXdblz4vvCC6XYJ3/NDAAMld/z48LPtt98+et2OHcOzu+vWTa8+lxHAQEn9+MfDC3HGefRRE7o7d06nrjJhBgyUzKpV5kOV42zZYoJ31arp1FRWdMBACTzzzHDMMC58d+0y3e6BA4TvNBDAgMPuuMOE7mtfO37dgQMmeD//+enUBYMRBOCY06eTd6/9vglo2EEHDDjikUdMmMaF7z33DE8zEL520QEDBbdtm/TnP8evO3Kk3Fc/5hEBDBTQ889LF18cv+7aa6XHHsu+HpwbRhBAgXz3u2ZsEBe+e/aYEQPhm290wEDORZFUSdgqnTolreT/6sKgAwZy6oknTLcbF7633TbcVCN8i4UfF5AzV1xhbiOL8/TT0pVXZl4OMkQAAzlw4oS06DNwI6X4iA1yjhEEYNFgUy0ufH/4w+GYAe6gAwYsSPoCxNGjyY6boZjogBcJw1BBEKhSqSgIAoVhaLskOKLTGV6IE2fQ7RK+biOAFwjDUI1GQ91uV1EUqdvtqtFoEMI4L5/8ZLKvTPziF4wZyobP0i8QBIG63e6S577vq5NkWxr4nzRnd0+fllasyLYe2DXqs/R0wAvMzs6meg43nc8YavfuZGd3P/CBYbdL+JYXm3AL1Gq1ZTvgWq1moRrYMBhD9Xo9SZofQ0lSvV4f+fvWrJHm5uL//IMHpU2bJlIqHEAHvECr1VJ10XmgarWqVqtlqSJMW7PZnA/fgV6vp2azuWTtiy8ON9XiwnfQ7RK+WIgAXqBer6vdbsv3fXmeJ9/31W63x3Y+cEuSMdTdd5vQjfsy8M6dbKphPAJ4kXq9rk6no36/r06nQ/iWzKhxU61Wm+92l2mGz3LsmAndHTsyKDADHL20hwAGFlg6hrpKUqRutzP2973iFcNu98ILs6xwsjh6aRfH0IBFwjDU5z63WSdPXh+7du9e6V3vmkJRGeHo5XSMOobGKQjgf4Yfs4wfO7nyMUuOXtrFCAKld//9yT5muX27ex+zHDfzRvbogFFaSUN0dla6/PJsa7Gl1Wqdde5Z4ujlNNEBo1T++c/0F+K4Gr4SRy9tI4BRCh/5iAnd17xm/Lqvfa18Z3c5emkPIwg4LemYodczrxMD00QHDOf8/OfpxwyEL2ygA4Yzkna7u3dL73tftrUASRDAKLReT7rggmRryzTXRTEwgkAhNRqm440LX98v36YaioMOGIWSdMzw97/HfwIIsI0OGLn3+OPpN9UIXxQBAYzcGoTu1VePX/fVrzJmQDERwBZxD+tSg3sWknS7L71k1t99d/Z1AVkggC3hHtazffvbyT5mKQ273dWrs68LyBL3AVvCPaxG0k21PXuk97wn21qArHAfcM6U+R7Ww4elyy5Ltpa5LlzGCMKSMt7D+sY3mo43Lnxf+Uo21VAOBLAlS7895u49rINNtQMHxq979lkTus89N526ANsIYEtcv4d1z570Z3fjrooEXMMmHCYq6abaXXdJDjb7wLLYhENmhh+zTLZ2xYps6wGKghEEztkddyT7mKU0HDMQvsAQHTBSSzpm+N3vpLe/PdtagCIjgJFIp5P8ghuOjwHJMILAWNddZzreuPDdto2zu0BadMBYVtIxw7//La1fn20tgKvogDHv179Of3aX8AXOHQGM+dD90IfiVm6X7weamSnnjW3ApDGCKKm5ueSfYl+z5gKdONGTJHW7UqPRkCRn3toDbKEDLpkvf9l0u3Hhu369GTH4fjAfvgO9Xk/NZjPDKoFyoAMuiaSbagcPSps2Df+5zNdmAlmjA3bYU0+l31RbGL5SOa/NBKaFAHbQJZeY0H3968evu/32+LO7Zbo2E5g2RhCOiKJk31OTpBMnpJe/PNnawUZbs9nU7OysarWaWq0WG3DABHAdZcHNzEjbtydby1tqgB1cR+mYpJtqv/pVkvO9AGxgBlwQYRiqVrs69aYa4QvkFwFcAG99a1ef/nRdhw6N/6jaNddwIQ5QJIwgcmzY6fpj1x06JG3cmHk5ACaMDjhn9u9PfnbX8yqKIsIXKCoCOCcGobt1yT7pYndJ8iR5vAwBFBwjCIv6/eTfSFuzZq1OnDg2/8+8DAEUHx2wBbt3m243SfgONtUeeOD/5fu+PM+T7/tqt9u8DAEUHAE8RS97mQne979//Lrf/37paYZ6va5Op6N+v69Op0P4xgjDUEEQqFKpKAgChSF3GCN/GEFk7MUXpXXrkq3l+NhkhGGoRqOhXm9wh3GXO4yRS3TAGWm1TLcbF77f+Q5ndyet2WzOh+8Adxgjj+iAJyzpK8LHjkkXXphtLWXFHcYoCjrgCfjb35Kd3b344mG3S/hmhzuMURQE8Hm48UYTulu2jF+3d68J3aNHp1LWxBVtQ4s7jFEUjCBSOn1aWrUq2dp+P/lIIq+KuKHFHcYoCu4DTuhnP5M+/vH4dZ/5jPSjH2Vfz7QEQaBut7vkue/76nQ60y8IKCDuAz5HSTtYVy/EYUMLyA4z4GUcPpz+Y5Yuhq/EhhaQJQJ4gQceMKF72WXj1+3aVZ6zu2xoAdlhBKHkY4a5OfM6cZmwoQVkp7SbcP/6l/SqV8Wv27zZnPMFgHM1ahOudCOImRnT8caF78GDZsSQt/At2plcAKOVYgRx5oy0bZv0l7/Er83zXLeIZ3IBjOZ0B/z446bbXblyfPjOzNjdVEva1XLJDOAWJzvgr39d+uY3x6/ZsEGanZXWrJlOTaOk6Wo5kwu4xZkO+PhxafVq0/GOC9+dO02ne/iw/fCV0nW1nMkF3FL4AH74YRO6F10knTo1et1TT5ng3bFjerUlkaar5Uwu4JZCBnAUSTffbIL3pptGr7vxRrMBF0XS6143tfJSSdPV1ut1tdttvg0HOKJQAfzssyZ0KxXpwQdHr3vwQRO6v/2tWZtnabtavg0HuCPn8WS02yZ4L798/LqjR03wfuxj06lrEuhqgfLK9Ztwc3PxG2W33irdd9906gGAc1HI6yh/8pPR/+5Pf5Le/Obp1QIAk5brAH7Tm6S1a82n3SUpCKQnnyzfhTgA3JTrAL7mGvOyxMmT0qWX2q4GACYr1wEsSevW2a4AALJRiFMQAOAiAhgALCl1AHO3LgCbcj8Dzgp36wKwrbQdMHfrArCttAHM3boAbCttAHO3bnExu4crShvArtytW7YwGszuu92uoiian927/veGo6IoSvzrhhtuiFwyMzMT+b4feZ4X+b4fzczM2C4plZmZmaharUaS5n9Vq9Wxf4+i/5193z/r7zv45fu+7dKAkSTti5bJ1FzfhobxgiBQt9td8tz3fXU6nSXPF5/8kEzXX6TrLyuVipb7b9bzPPX7fQsVAfFG3YZW2hGEC9JuJLpw8oPZPVxCABdY2jBy4eSHK7N7QCKACy1tGLnQPfIFEbiEAC6wtGHkSvfId/HgikIEcNmOWqWRJozoHoF8yf0pCBd27gGUW2FPQbiwcw8Ay8l9ALuwcw8Ay8l9ALuwcw8Ay8l9ALuycw8Ai+U6gMMwnJ8Br1ixQpLYuS8JTr6gDHL7RYzFpx/OnDkz3/kSvm7jayUoi9weQ0t70Qzcwc8erincMTROP5QXP3uURW4DmNMP5cXPHmWR2wDm9EN58bNHWeQ2gLm3oLz42aMscrsJBwCuKNwmHAC4jgAGAEsIYACwhAAGAEsIYACwJNUpCM/zDkta+o4oAGAcP4qiSxc/TBXAAIDJYQQBAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJYQwABgCQEMAJb8F4FKY8Ec3TGwAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot outputs\n", "plt.scatter(diabetes_X_test, diabetes_y_test, color='black')\n", "plt.plot(diabetes_X_test, diabetes_y_pred, color='blue', linewidth=3)\n", "\n", "plt.xticks(())\n", "plt.yticks(())\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "QkT3wj6vTM2T" }, "source": [ "## Exercise\n", "\n", "In this exercise, we will apply a linear regression to a materials science problem. \n", "\n", "\n", "\n", "The first part (creating a representation) can be performed in different ways, as we studied in the previous module. Today we will see the second part\n", "\n", "Let's open a representation of a data set that we generated like yesterday and build a linear model for predicting some numerical property. The representation is the eigen values of the Coulomb matrix." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
feature_0feature_1feature_2feature_3feature_4feature_5feature_6feature_7feature_8feature_9...homolumogapr2zpveU0UHGCv
O3NC4H5134.38304973.27743165.56522041.20533937.97136028.56217022.14847519.5414560.3461830.246778...-0.2528-0.00770.24511012.18640.095653-435.725372-435.718189-435.717245-435.75726626.019
FONC6H4135.76212285.19293461.37565438.78915036.30910926.51595423.00382220.93152518.6641590.345873...-0.2604-0.07890.18151188.64410.089534-460.774434-460.767413-460.766468-460.80637725.848
O2N2C5H4139.21411374.41726949.82891645.67417037.00023826.23330924.53920423.07953619.0392770.343824...-0.2324-0.04680.1856945.79880.092456-452.744414-452.738132-452.737188-452.77571623.374
O3C6H4137.46688773.56536461.51838346.47332532.47677128.00542623.11813820.29464519.7725880.349248...-0.2670-0.08230.1848944.04260.090391-456.613010-456.606412-456.605468-456.64439224.902
FON3C4H4146.43197883.54877159.49413250.05566143.92324231.13762023.79579122.15140019.0936720.331518...-0.2192-0.03090.18821074.30780.090835-494.100913-494.093709-494.092765-494.13247127.708
\n", "

5 rows × 29 columns

\n", "
" ], "text/plain": [ " feature_0 feature_1 feature_2 feature_3 feature_4 feature_5 \\\n", "O3NC4H5 134.383049 73.277431 65.565220 41.205339 37.971360 28.562170 \n", "FONC6H4 135.762122 85.192934 61.375654 38.789150 36.309109 26.515954 \n", "O2N2C5H4 139.214113 74.417269 49.828916 45.674170 37.000238 26.233309 \n", "O3C6H4 137.466887 73.565364 61.518383 46.473325 32.476771 28.005426 \n", "FON3C4H4 146.431978 83.548771 59.494132 50.055661 43.923242 31.137620 \n", "\n", " feature_6 feature_7 feature_8 feature_9 ... homo lumo \\\n", "O3NC4H5 22.148475 19.541456 0.346183 0.246778 ... -0.2528 -0.0077 \n", "FONC6H4 23.003822 20.931525 18.664159 0.345873 ... -0.2604 -0.0789 \n", "O2N2C5H4 24.539204 23.079536 19.039277 0.343824 ... -0.2324 -0.0468 \n", "O3C6H4 23.118138 20.294645 19.772588 0.349248 ... -0.2670 -0.0823 \n", "FON3C4H4 23.795791 22.151400 19.093672 0.331518 ... -0.2192 -0.0309 \n", "\n", " gap r2 zpve U0 U H \\\n", "O3NC4H5 0.2451 1012.1864 0.095653 -435.725372 -435.718189 -435.717245 \n", "FONC6H4 0.1815 1188.6441 0.089534 -460.774434 -460.767413 -460.766468 \n", "O2N2C5H4 0.1856 945.7988 0.092456 -452.744414 -452.738132 -452.737188 \n", "O3C6H4 0.1848 944.0426 0.090391 -456.613010 -456.606412 -456.605468 \n", "FON3C4H4 0.1882 1074.3078 0.090835 -494.100913 -494.093709 -494.092765 \n", "\n", " G Cv \n", "O3NC4H5 -435.757266 26.019 \n", "FONC6H4 -460.806377 25.848 \n", "O2N2C5H4 -452.775716 23.374 \n", "O3C6H4 -456.644392 24.902 \n", "FON3C4H4 -494.132471 27.708 \n", "\n", "[5 rows x 29 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The features in this data set are the eingen values of the Coulomb matr\n", "\n", "import pandas as pd\n", "data = pd.read_csv(\"data.csv\",index_col=0)\n", "data.head()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Index(['feature_0', 'feature_1', 'feature_2', 'feature_3', 'feature_4',\n", " 'feature_5', 'feature_6', 'feature_7', 'feature_8', 'feature_9',\n", " 'feature_10', 'feature_11', 'feature_12'],\n", " dtype='object')\n" ] } ], "source": [ "feature_names = data.columns[0:13]\n", "print(feature_names)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "molecules_X = data[feature_names]\n", "molecules_y = data['gap']\n", "\n", "\n", "# Split the data into training/testing sets (we will discuss this latter on)\n", "molecules_X_train = molecules_X[:-2000]\n", "molecules_X_test = molecules_X[-2000:]\n", "\n", "# Split the targets into training/testing sets (we will discuss this latter on)\n", "molecules_y_train = molecules_y[:-2000]\n", "molecules_y_test = molecules_y[-2000:]" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# dataset and algorithm \n", "from sklearn import datasets, linear_model\n", "\n", "# quality measures\n", "from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\n", "\n", "# Create linear regression object\n", "regr = linear_model.LinearRegression()\n", "\n", "# Train the model using the training sets\n", "regr.fit(molecules_X_train, molecules_y_train)\n", "\n", "# Make predictions using the testing set\n", "molecules_y_pred = regr.predict(molecules_X_test)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD6CAYAAACvZ4z8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dfZAc9XnnP8+OZsWuHLPCVlJmxCKZYDCKQBvWoFhlKFQY5CiIjbEtMKrgi+/Ii6k7wFbdcuhAEFzo2HIgV+EqkDtXyodjxIu9JSKIwEZOyorlaOVdoRMgWwgQGvmMYmnxwS7SaPXcHzO96p3t7vn1TM/rPp8qlXb6bX49PfPtp5/f8yKqimEYhtG6tNV7AIZhGEZ1MaE3DMNocUzoDcMwWhwTesMwjBbHhN4wDKPFMaE3DMNocZyEXkRWiMheEdknIv0B6/9URHaLyIiI/EhELigs/7SI7Cys2ykiy5M+AcMwDCMaKRVHLyIp4GfAp4GDwA7gBlV92bfNB1X114W/VwF/rqorRKQH+KWqHhKR3wG2qGom6v0+/OEP64IFCyo5J8MwjBnHzp07/01V5wWtm+Ww/yXAPlXdDyAijwPXApNC74l8gTmAFpYP+5bvAU4TkdmqeizszRYsWMDQ0JDDsAzDMAwPEXkzbJ2L0GeAt3yvDwKXBrzJV4DbgXYgyEVzHTAcJPIicjNwM0B3d7fDkAzDMAxXXHz0ErBsmr9HVR9W1XOA/wysm3IAkUXAfwP+JOgNVPVRVe1V1d558wKfPAzDMIwycRH6g8BZvtfzgUMR2z8O9HkvRGQ+8D3gj1T1tXIGaRiGYZSPi9DvAM4VkYUi0g5cD2zybyAi5/pergR+XljeBWwG7lDVbckM2TAMw4hDSaFX1RPALcAW4BXgCVXdIyL3FiJsAG4RkT0iMkLeT3+Ttxz4beC/FkIvR0TkN5M/DcMwDCOMkuGVtaa3t1ct6sYwDCMeIrJTVXuD1llmrGEYRotjQm8YhtHimNAbhmG0OCb0hmEYLY4JvWEYRotjQm8YhtHimNAbhmG0OCb0hmEYLY4JvWEYRotjQm8YhtHimNAbhmG0OCb0hmEYLY4JvWEYRotjQm8YhtHimNAbhmG0OCb0hmEYLY6T0IvIChHZKyL7RKQ/YP2fisjuQgepH4nIBb51dxT22ysiVyc5eMMwDKM0s0ptICIp4GHg0+Qbhe8QkU2q+rJvs79X1b8pbL8K+EtgRUHwrwcWAWcC3xeRj6nqRMLnYRiG0ZAMDmcZ2LKXQ6PjnNnVwdqrz6OvJ1PTMbhY9JcA+1R1v6oeBx4HrvVvoKq/9r2cA3j9Ca8FHlfVY6r6OrCvcDzDMIyWZ3A4yx3f3U12dBwFsqPj3PHd3QwOZ2s6jpIWPZAB3vK9PghcWryRiHyFfGPwdmC5b9/tRftOu5WJyM3AzQDd3d0u4zYMo4ZEWaWNYLE2KgNb9jKem+rAGM9NMLBlb00/Ixehl4Bl0zqKq+rDwMMi8kVgHXBTjH0fBR6FfHNwhzEZhlEjPKvUEyzPKvUIW2diD4dGx2MtrxYurpuDwFm+1/OBQxHbPw70lbmvYRgNRpRVGrXOgDO7OmItrxYuQr8DOFdEFopIO/nJ1U3+DUTkXN/LlcDPC39vAq4XkdkishA4F/jXyodtGEatiLJKG8VibVTWXn0eHenUlGUd6RRrrz6vpuMo6bpR1RMicguwBUgB31TVPSJyLzCkqpuAW0TkSiAHHCXvtqGw3RPAy8AJ4CsWcWMYzcWZXR1kA4Tbs0qj1s10PPdVvecwRLWxXOK9vb06NDRU72EYhlGg2EcPeav0/s8uBghdZz762iIiO1W1N2idy2SsYRgzmL6eDENvHuE7P3mLCVVSIlx3cWaKkNfbYjWiMaE3jCrRKmGHg8NZnt6ZZaLw9D+hytM7s/SefQZ9PZnJf0bjYkJvGFUgKiQxrihW64bhetxGiQX3aJUbaC0xoTdmFLUSiaTEMckbRrnHrWVkTanrU63Po9Wx6pVG0zE4nGXZhhdZ2L+ZZRtedE4nD0pHv23jCAtiHseFpMSxWnHqcY5bq1hwl3IBFrdfHib0RlNRSe2QIJHwYs6SrkGSlDhWy5oO2z87Oj7tM6hVLLiLiFvcfnmY0BtNRSUWXSkxSNIyrFQcvaeWsODnSq3pqP2Lb3h9PRnu/+xiMl0dCJDp6qhK+KSLiDdKpmmzYT56o6moxKILS/yJexwXKkmUCYpb95OENb326vNC32M8N8E9z+yZMtZaRNaUSsyC4HHXI9O02TChN5oKFzEII0rc4hzHlXLFMeipxSOT0ASyt/+tG0cC1x8dyzE4nK1phJCLiDdKpmmzYUJvNBWVWHR+kciOjiNMLaVaa8swTBTDnioE2Na/PHBdOfT1ZCY/iyDWb9pT0wghVxG3uP34mNAbTUWlFp1fJOoZjx0lipU8tcRl7dXnhVr1o+O5SV99kvH2UZ+7iXh1sFo3hlEHlm14MVDMPddMLevHfPy/Psd47mTgurmdad7PnXQay8L+zYGTxwK8vmElEF03xwS+MqJq3VjUjWHUgahJ5VpFuXicVhQd5OfoWC7ReHuLg68P5roxjDpQyj1TSxfG6Fgu9j5BNyqX+ROLg68PZtEbRh1olIYUAKd3pAOXC9AVsi7Iend5ErE4+PpgFr3RMMykYlVBk8pXnD+PgS17uW3jSM3Of3A4y3vHTwSuu3FpN71nnxEryqnUk4jFwdcHE3qjIZiJxaqKI4Dqcf4DW/aSm5g+hTq3M819fYunbJfEDdji4OuDU9SNiKwA/op8K8H/qaobitbfDvx78u0CDwN/rKpvFtY9QL6PbBvwAvCfNOJNLepmZhIVhZJk7HijEnb+czvTdLbPiqzmGCSark9HC/o3B47HHyljNAcVdZgSkRTwMPBp4CCwQ0Q2qerLvs2GgV5VHRORPwMeAFaLyCeBZcCFhe1+BFwO/LDckzFak0acpKu2K8l//DDL5+hYjqOFydJiKz/sKWDozSM8vTNb8ulgcDg7LWnMw3zmrYWL6+YSYJ+q7gcQkceBa8k3/AZAVbf6tt8OrPFWAacB7eSNhDTwy8qHbbQatUwScqHarpR1g7v59vYDoQIfhj8BKSxU0Wv5F7afx8CWvaHvf+S9Y7GSpYzGxiXqJgO85Xt9sLAsjC8DzwGo6o+BrcAvCv+2qOorxTuIyM0iMiQiQ4cPH3Ydu9FCNFIUClQe7x1VM39wOFuWyHt4TzlhTzvFIl+8X9hrP+O5k6x9ahdrn9xVVkloo7FwEXoJWBb4TRKRNUAvMFB4/dvAx4H55G8Oy0XksmkHU31UVXtVtXfevHmuYzdaiFonCZWiEldSqZr5UZa0C95TTtjTTkqCfrLTty/1tJSbUHIng58MjObCRegPAmf5Xs8HDhVvJCJXAncCq1T1WGHxHwLbVfVdVX2XvKW/tLIhG41CuZ2ewujrybCtfzmvb1jJtv7ldXURVBLvXeppoJJ5B/9TTthT0A2XnuX0dLT26vMCrbhSBDUnMRobF6HfAZwrIgtFpB24Htjk30BEeoBHyIv8275VB4DLRWSWiKTJT8ROc90YzUecTk9J3xBqQSWupFJPA+XMOwQ95YQ9Bd3Xt9jp6aivJ8ONS7vLEntz4TQXruGVvw88RD688puq+nURuRcYUtVNIvJ9YDF5PzzAAVVdVYjY+R/AZeTdPf+oqrdHvZeFVzYHruGQSRWxqkcyVbnvWeqzGRzOctvGEWf3TbVDTAeHs9zzzJ7J6B5XZkroa7NQUXglgKo+CzxbtOwu399Xhuw3AfyJ+1CNZsHVh+1aujaKeiUTlVtvplT2Z19PhqE3j0ybkE2nBJQpfvFaTEh757nknucZHXcXe9f5CovaqT9W68YoC1cfdhLx8c1W8dBlYvm+vsU8uHrJlG0GPncRA5+/qG4T0u/EEHko7YKqpJG7kSxWAsEoC9eaJUnExzdiMlUpgp4Ggqxbz/VRvO7B1UtqbvmGXauujjTHTkytSS/AFedHR8gl8TRnJINZ9EZZuIZDJhEf3woVD6Os20axfMOu1fpVi7ju4syUSVsFnt6ZjRxjM96gWxWz6I2ycfFhJ1HEqhUqHpZyPzWC5Rt1rYJi/0uNsdGynWcyJvRG1am0iUYrVDwsx7pN0vJ1nRQNu1bljL8VbtCtggm90RQ0e9PoUtZtNS3fJKKWyrXOZ89qm3zfuZ1p7r5mUVNfx2bFfPSGUQOi5iqqXecniailuGP0bi7+cM33QxqQG9XHLHrDqAEu7qdquabiuF3CXDxR4w/axyJuGgunzNhaYpmxhpEs1cxiDtunWOQ9rKFJ9YjKjDXXjWG0OK5ul3JcPGH7uFbQNGqDuW4Mo8rUuwyAa9RS3MiaweFs4JMC5GviF1v2FnFTP0zojYan3kJZCY3S9NwlaiksskbJu3/8n/vgcJa1T+0KPVbG56uv9nVr5u9HrTChNxqaRhHKchgczvLVJ3Y5tfVrBILi3j2KP/d7ntlDbiJ4fs+z3GsREtvM349aYj56o6FptoJmHp4Aubb1awT8ZS2C8H/uUSWNa1mIrVm/H7XGLHqjoSm3Xkq9H+eDBMhPo05Kelb4wv7NgfXyXW5QtfycrZ6OGyb0Rs1xFeHB4SxtIoFW8ZldHaHHqdbjfJybR5TQNMOkZKlM2K6OdGDt+q6OdNXHVjweq6dTGifXjYisEJG9IrJPRPoD1t8uIi+LyEsi8gMROdu3rltEnheRVwrbLEhu+Eaz4VqpMcr10ZFOccX580KPU43H+XWDu7lt48iU97t14wgL+jezoH8zPfc+P+Ucohp317PpuSulQjLXr1pEum1qCGW6TVi/alHNxgjJVEedCZQU+kI7wIeBzwAXADeIyAVFmw0Dvap6IfAU8IBv3beAAVX9OHAJ8DbGjMVVhMNcH55Qbn31cOhxkn6cHxzO8lhRN6hijo7lWPvUrkmxDxOgb3zhooYXeShdhrqvJzOtScrA52t/bq7lsmc6Lq6bS4B9qrofQEQeB64FXvY2UNWtvu23A2sK214AzFLVFwrbvZvQuI0GpZR7w1WEw7Y7qUpfT4bbNo6EHifpx/n1m/Y4bZebUO55Zk/JkgHNQqmomUYpNNco42hkXIQ+A7zle30QuDRi+y8DzxX+/hgwKiLfBRYC3wf6C71kJxGRm4GbAbq7u91GbjQcLr5xVxEutV3YeiT/vgJTLPBKHufj9FE9OpZjcDg7KT4mQEYj4OKjD8plDnyKFZE1QC8wUFg0C/gU8DXgE8BHgS9NO5jqo6raq6q98+ZFtyczGhcXt0yYS+OK8+exbMOLLCz4u4+8d2za8f1ivfbq8/LNtIvwXPrKqS9urR/nGyG0b3A4O/l5LtvwovVpneG4WPQHgbN8r+cDh4o3EpErgTuBy1X1mG/fYZ/bZxBYCvyvSgZtNCYubpkgl8YV58/j6Z3ZyZtEUIx2m0y9afT1ZFi/aU+kta1ML9zlit8F1SZwMkbtv3qH9lkSkVGMi9DvAM4VkYVAFrge+KJ/AxHpAR4BVqjq20X7zhWReap6GFgOWGnKFiXK3RLlu1+24cXImHM4JbTZ0XHWPrmLe56JFnmPsFosxfjH19WZ5t33T5ArvGncAq9BJQNqiZUINoop6bpR1RPALcAW4BXgCVXdIyL3isiqwmYDwAeAJ0VkREQ2FfadIO+2+YGI7Cb/NP23VTgPowGIcstEhVTGtYBzJzUyM9NPWBVFP8Uhn0fHcpMiX3wscTxmqQbf1XStWBKRUYxTHL2qPquqH1PVc1T164Vld6mqJ+hXqupvqeqSwr9Vvn1fUNULVXWxqn5JVY9X51SMehMW6hYVCgnVTW4JK0Hgp1QWq/9YD65ewklHEz8sdt81l6Bcwj5PSyKauVhmrJEoQZEmUaGQEF1Mq1LC6rYEjcOFO767m67OtPMTRdCxw1wrX30iXw2yUvdKpU25610+wkgeE3qj6riEVJ6WbquK0C/40Kn3CBOw0FDNAMZzE8ye5V4LMMiKDruxTKgmMmlaSQy/TeS2JtZK0Kg6US3qgEhrPp2S0HK4LqREeO3+369oDMUI0D6rjWMnoptdh7XhC2vt5x/zSdVpAl0LS9u17aDReES1EjSL3qg6URZmVMTN3M40x3ITFQm956OPikTxBMwfdRPlmmkTKSnyczvT3H3NokAhvuL8eXw7oqSCN2a/NQ3UxNK2idzWxITeqAlhWaJRAjI6lousL7NmaTff+clbkROuXoRM2PtkR8cnQyG9jkil3DguE7zv54JvBOsGd0eKfDH+Cd1q+vU9XDOXzY/fXFjjEaOudHWGl7UtJYYb/zVa5AFuuDSf6xcVceLF5q99apezr74UQRE3g8PZWCLvcWh0vKRfP6mIHZdqkNWOGjKSxyx6o2zWDe6etKhTItxw6Vnc17d4cn2x1XfF+fPY+uphX1JSjhDD14mgWHcP/3jWDe7mF+9EC3jUscqlWJwHtuyNFHmR4OQs7yYVdhPyLPvbNo5UbF27TORaQlbzYUJvlMW6wd08tv3A5OsJ1cnX9/UtDoze8G/vGp4YRLpNIoX5jQ0rQ8dZrfcMovgpIspN1dWR5r3jJ6bNR6TbZNKajpowDvLrVyL2UfuaH7/5MNeNURbf+clbkctdk5BcaJO8EPrrnrsSNs44dHWk+cBp8Wwiz93hz4BtC8moFfLWfNCkc/usNga27OW2jSPMntVGW+mk3Kr3TLWErObDhN4oizDfuLc8SevupMKxEyd5cPWSkiF+cws+f09gXSZOSzE6not8Asl0dbBmafe0jGBgii87bCyfPOcMRkOO/97xicn9R8dzpEQCq3YWU03r2ro6NR/mumkCqhXhUMlxUyG9XL0olzhJSC6M5ya4deMIA1v2Mnb8ROh2Ky/8CD33Pl+RaygOs2e18X/feZ/Hth8gJcKNS7sn5ylcirUBbHvtSOjnWUzupNLVkWbO7FmFyprB+3V1plm24cWqRMW0QlOVmYYJfYNTzUbXlRz3hkvPCvR9e1EuCz6UrNB7lDqmv9xxLfDH03vzFD/Z/yvGjp+Mdf5xnjzeGc8xcvdVQHAyWjolvPv+icmbXTVi7q2pSnNhrpsGpxqNrpM47n19i1mztHvSgk+JsKZgzQ4OZ/mX145UNL5ySInUVOTD+Pnb75V9k/MqZGa6OujqCA499fvCgwrJzWmfNW3iuNp+e6OxMYu+walWhEPUcV1dOvf1LZ4STulRKowwiI50G6elUxwdy01rA+i2f6qkyPtdHo1V+OMUE6qTUUNhZRs8X3jxdXpw9RL6ejIs7N8ceGyLipm5mNA3OEk1ui4WhbA0/67OdMWuonIE5f3cScZzJ8sS+fz+E3Sm2xgLCczvSKdYv+pUSYJS9WbqyY1/+2O+/R9+L9IXPjicZe2TuyYtdy/pC5L5zljma2thrpsGJ4kIh6BMxnffPzEteqMjnUI1ONU+zmN/OWF2WvR/OfuP5U4Ghh92daSnFRcL+lwbhW2vHZnMMu3rybD26vM4s6uDQ6PjDGzZy+BwlvWb9kxzz+ROKus37an4O2OZr62Hk9CLyAoR2Ssi+0SkP2D97SLysoi8JCI/EJGzi9Z/UESyIvLXSQ18phDWzCOOdRXkj8+dVOa0z5p23HdC2vPFsX7DRHROeyqw03ySqDLlnB5avYSRu6+a8nmtG9zNV5/Y1RD+/DC8G2uY6Ia1URwdzzl9Z6I6XFVrXsioHyVdNyKSAh4GPk2+2fcOEdmkqi/7NhsGelV1TET+DHgAWO1b/xfAPyU37JlFpREOYa4Uf/SGR1hRLyEvDi7jKBV+V023iWfjev7qYpLKlK02Wd9cSZDoliLqO1Mq4soyX1sPF4v+EmCfqu4vtAF8HLjWv4GqblXVscLL7cB8b52IXAz8FvB8MkM24hInk3Ht1ecFWt0KsSy6vp4M2/qX8/qGlZNJTp4F+d6x8Dj4JIhyNSSRKVsrPEs+DnMjisR5lLLYLfO19XAR+gzg/3UcLCwL48vAcwAi0gZ8A1gb9QYicrOIDInI0OHDhx2GZMQhjs+2rycT6icv16JbN7ib2zaOTMnwrDZhroYkMmXLwaGf+DTGcxOhjci7OtLT5lhSbYIqJRuOl7LYLfO19XCJugkz8KZvKLIG6AUuLyz6c+BZVX1LIr7pqvoo8CjkO0w5jGnGkET0Q7Er5fSONCL5Xq4DW/ZOO2YmwaiNeka2BAlauVE9QWQKE6Qux1PN1+yJWyQz7MY0Op6b8sOc057i+ImTkzfRqGipUlE5lvnaergI/UHgLN/r+cCh4o1E5ErgTuByVT1WWPx7wKdE5M+BDwDtIvKuqk6b0DWmk2RWrOezdTlmEs2lq9XsOw5BN6bO9hTvHU9mXK6NSjySroTsP9zY8YlpN5yw0sEu19cyX1sLF9fNDuBcEVkoIu3A9cAm/wYi0gM8AqxS1be95ap6o6p2q+oC4GvAt0zk3alG9IPLMSuN9EmycmW5hN2YxkJEXsiXN35o9RKn43d1pCdDHxshTDOOu63S6xsVsWM0JiUtelU9ISK3AFuAFPBNVd0jIvcCQ6q6CRggb7E/WXDRHFDVVVUc94yglC+1HLeOa0RFJRZdNaIz5namUS3t3xeI/Cxc3Ba3bhyJfI90Sli/atHk9gD3PLMnkUJqrsXNXAlzt5V7fatVe8moLk6Zsar6LPBs0bK7fH9f6XCMvwP+Lt7wZjZRolTuDy7JrMns6PikMHUV/P6jY7nQioqV4CKima4OtvUvnxxfUMelUm6LweFsSbE9UeSD8bvFKpmX6EinuP+zi0veaMIonn+oxgSqdZdqTiwztgHxHo2zo+PTZsK9H2+5bp0wV8N7x044PYL7E3jg1GShV7M9qu56NfE3+ghKMFo3uJtlG16cbOAxtzM9zW3h7Vtq/KoEhm+W48rpSJ/6CZ6WbmPozSNlJZV1pFPcGFATPynx9X8ng7AY+8ZGtE7hZmH09vbq0NBQvYdRN4ImMj1LzbM0w6JiPB4KSRbyv0eQq8GzKKP2bcQaMRmf1e46vnSbTHaq8txfcZ9EUiJ84wsXTX5eg8NZvvrELqdjCPmGIz898E7gtQ5j2Tln8Mavxqc8TWWqHBXjMrnuPU0Z9UNEdqpqb9A6K2rWYARZ6kpeAPx9QaMEoZQLp68nw8CWvdOE3uURvNEst2KBcR1f7qTyX777Esqp0sZxn0QmVCc/a8DpaQDy1/LB1UtCr3UUb/xqvOaCWmpy3WLsGx8T+gYjTKiKBcAT/yBhqESww5Z7/ufGev6Do+8dY2H/5klffJzOVmGVLuPgd5e5RhrNKiQ6lfNklB0dn3K+tfCLR42z2k8TRjKYj77BiDMpGiW62dHxyNC3OGnuxX75RmIsd3LSF3/rxhGOvHfMqaeqKx3pFGuWdkf63Q+Njsd60slNKANb9oZmvZai1hUlw8aZEmFb/3LnYmlG/TChbzCCJvPKla0oMYiT5t4IcfGujOdOgrrVfHHh/s/mm6vc/9nFoYJ3ZldH7Dowh0bHK560Lp58L0dkXfYp1Qjefywrb9yYmNAnRFKWTFAyy40BFqWr+I/nJvjqE7smx+ON87aNI5yWbsuHRRIdpdFofvlS5E4qne2zeGj1koqSmeZ2piejce55Zk+g4Hk3x7jRNmd2dZAJuTl45ZXDWgn68edUxBVZ132ixunHyhs3LuajT4Ckk0iCkll6zz5jSnJU3MbTd3x3N08OHeBfXjsy6fI5OpajI52aVtK3OE6+0fzyLhwaHZ9Ss6Uct5NqIZLmyV1MBNQvmNuZ5u5rFk357FzeS8h/R+Z2pkm3yZQGIt6Nw/8diIok8p4kyolvd93HtSSGlTduXEzoE6CaSSRhfUHjhjmO5ybYFtCwu3icxTetelV7rJQkSuqOjue483u7A0UeoLM9//NZtuFFDo2O01XI3o3CP4F+dCxHOiV0daR5ZzwXOsEaJZSe2JYjsmHrwiZ8S2VhJ9X20kgeE/oEqJYlE/SksPbJXZHp9uVUZ/Q3BG/ECde4CExJnqpkfiGqAJr35OYd3yV7t/ja5CaUObNnTWsA4ydMQD3XUtQ2USIb9WTod+WAW8mESovhGdXDfPQJkHSjBs+PfuvGkcAWgFGCcnpHOnY0x+kd6YaNqikHr0lK0OeXJCmRRI7vdZMK44rz5wUuX3nhRyb/LqeGvMu8QhwfexJtL43qYBZ9AiRpyVRqhb4znuPB1UucjyHkm2I0S1SNC54PvNok6daKmtPZ+mpwMx7/8nJqyBfvk0TDGStv3JiY0McgrFpkOT+ysGNVGsqo4JyGD3Dj0m6+3QQ9VOPQjLMKUXM61aw46jLpaz725sdcN46UCkXzClqdWeg6NLBlb+jjeNCxbt04Qs+9zydiicaxNO/rW9wyP2R/gbBmJEzQa9XD1VoIti7N/cuoIaVihOPEMYdZ7UnUM4+DF6cd1hC8WfDmJI6faEZb/hRe+enifIxaCbD52FsXq17pyML+zYEuAQFe37AyMtyxuB5I2LHCSLLPaRClqmE2MtX+bGpFRzrFdRdn2Pivb02Jqw+qstkIPVyT6GVsJEtU9Uqz6B0p9fgcNWFVbN3HeeSuRcJSs4o8tIbIA1x3cYZ/2PWLKSIP+Sir9Zv21GlUwVipg+bDSehFZIWI7BWRfSIyreeriNwuIi+LyEsi8gMRObuwfImI/FhE9hTWrU76BGpFWA0aL/StlHj73TxxHrmbNWHJiMfml34R2iZxdDznLKy1KCpmpQ6aj5JCLyIp4GHgM8AFwA0ickHRZsNAr6peCDwFPFBYPgb8kaouAlYAD4lIV1KDryV9PRmuuzgzxZetwNM7s6F+1GI8q7+vJ5NY0S2jNSg1P+MirLWytK3UQfPhYtFfAuxT1f2qehx4HLjWv4GqblXVscLL7cD8wvKfqerPC38fAt4GgrM/moCtrx6e5irwh8V5E1lh+K3+u69ZRKqtmadAjaSJe/MvFtZaWdq1igIyksNF6DPAW77XBwvLwvgy8FzxQhG5BGgHXgtYd7OIDInI0OHDwckhjUApS6avJ8O2/uWhVRPHjk/tyxpWQ8WYeXR1pKdkuvrpDAkbVZjinqmVpW1hmM2HS+RoaiYAABSuSURBVMJUkNkZqFAisgboBS4vWv4R4H8DN6nqtLY+qvoo8Cjko24cxlQXXOuJeNEH6zftmeJ3PTqWm8yANH+m4Wf9qkWh34n2WakpLQ/9+OvR1KqoWDkJgkZ9cbHoDwJn+V7PBw4VbyQiVwJ3AqtU9Zhv+QeBzcA6Vd1e2XDrSxxLpq8nw5zZ0++j3qN0klaWuYCag1JXKew78c54jusuzoTWMPK+U7W0tL2n19c3rJzWZcpoPFws+h3AuSKyEMgC1wNf9G8gIj3AI8AKVX3bt7wd+B7wLVV9MrFR14m4lkxUGdiUSCIRNSkRvlGIs660UqNRXX7zN9r55f87Hrjunmf20BbynTi9I83TO7OR35fi+vtmaRt+Sgq9qp4QkVuALUAK+Kaq7hGRe4EhVd0EDAAfAJ6UvNVxQFVXAV8ALgM+JCJfKhzyS6o6kvyp1IY49UTCHqWFeGGTYTcFAb7xhYsmuyDNntVmQt/AhIk8REfdHD8xkW+RGIHnnqlVUTFLmGounIqaqeqzwLNFy+7y/X1lyH6PAY9VMsBmJqiqZTmZnGE3BYVJkTdrvnUZKyHytZ4ITbqjmlF9LDM2hCQST4JqhyQ50+yFcjZT824jWepRj8YSppoPK1McQCmLJc5ja/Gj9Dl3PJuIb95vxVmiSnNT6ilPBGa1CbmJqb1l61VwzBKmmg8T+gBKWSyVPLYmIfL+ptSDw9nQSTyj8Um1Scl8ClWYmFDmdqYZHQvvLVsrrDds82FCH0CUxRJ2E/jqE7uA0mKfRKXI9ws+23WDu/n29gMtU9hrJuKaNHeSvOC/vmFldQfkgPWGbT7MRx9AVIp32E1gQtWprohLTZxSjOcmuHXjCI+ZyM8owoqe1RqrW998mNAHEJV4EvV46jIh5VITxzDCaJRSwJYw1VyY0AcQZbFccf68yAxHlwkpf00cy2mducxpT8W+/lb33SiHGe+jj2r4XWylDA5neXpnNtJdEndCylwvM5N0Svj6Hy5m6M0jseZZopqIG0YYM1ro4yZ+uMSre41IvBuIv9yBv6WgN5FqzDwyRQZF79lnTBobLhFUFsZoxGVGC31UGGWQ0Lv8wLa+enjaDcT74Xo3krhWnNE6dHWk2da/fMoy/9OjS5azhTEacZnRQh838SMsfthPdnScO78X/kMdz03wmFnyM5bR8RzLNrwYmmzn/f3VJ3aF1jeyMEYjLjN6MjZupxzX0Mj3jls5AiMYgWmt/tYN7p5SbgPyxeqCehTfuLTb/PNGbEQbLKOyt7dXh4aGavJeQY/J/tTyweHslOYhczvzXYC2vnqY7Oh4WQXKjJmBUChb4EuICvu+FC/3voNgJYcNd0Rkp6r2Bq6byUIP4VE3g8NZ1j65a8oPFfLREgOfO1Ua+NaNTVtx2agy6ZQwp30W74znnNx+fjJdHdN8+X6sTLBRTJTQzxgffZwwSshbUsUiD5Cb0MnJ2r6ezLR2gYbhkZtQ5syexcjdVzE4nOW2jSPOT4BRE/9WJtiIy4wQ+nJ+GFE/tOzoOMs2vFhxzRqj9fG+RwNb9sZy80VF1sSNFjMMp8lYEVkhIntFZJ+I9Aesv11EXhaRl0TkByJytm/dTSLy88K/m5IcvCvl1M+O+qF5E2qGUYquzjQQL/a9VIEwKxNsxKWk0ItICngY+AxwAXCDiFxQtNkw0KuqFwJPAQ8U9j0DuBu4FLgEuFtE5iY3fDfK+WGsvfo80iFNtxtrVsNoZN59/wSDw9lYse+lCoTFjRYzDBeL/hJgn6ruV9XjwOPAtf4NVHWrqo4VXm4H5hf+vhp4QVWPqOpR4AVgRTJDd6ecH0ZfT4aBz19EV0d6ctncznTo9oYRRO5kfk7HNTTXb1qEdTmLKrpnGEG4+OgzwFu+1wfJW+hhfBl4LmLfaaaKiNwM3AzQ3d3tMKR4RNXPjopeCJqoTcI3P6c9ZbH2M4hDo+OT3yPvu3Z6RzpwEl8L20DpBjcWdWO44iL0Qf6LQO+FiKwBeoHL4+yrqo8Cj0I+vNJhTLEI+2FA/G5Ra68+r2T0RKnmIibyjU1XiAiXi/fkWGw4LOjfHLh9VIMbf8SXCbvhiovr5iBwlu/1fOBQ8UYiciVwJ7BKVY/F2bcWBNXPLmeStq8nw41Lw586vCcFc/M0Jx3pFH9w0UcSPV6YSyWsJ0FUgxubcDXKwUXodwDnishCEWkHrgc2+TcQkR7gEfIi/7Zv1RbgKhGZW5iEvaqwrCGI82MaHM6y5J7nWdC/ObJWzexZ+Y/07msWkU5ZtflmItPVwXUXZ3h6Z2X13lMiTp2XymlwYxOuRjmUdN2o6gkRuYW8QKeAb6rqHhG5FxhS1U3AAPAB4EkRATigqqtU9YiI/AX5mwXAvap6pCpnUgauTY7DsmSDGB3PTbp2vGbOFqXT+AiwrX85yza8WLIUdSkmVHlo9ZKSrpW+ngxDbx7hOz95iwlVUiJcd/Epl4z1ZTWSwilhSlWfBZ4tWnaX7+8rI/b9JvDNcgdYTVybHIdlyYbhbXl0zDJmmwXv5p5UfoRLpqrXyMarUjmhytM7s/SefYZNuBqJYrVuHGqGLOzfbFZ5C+MVFct0dfCLd8aJcU+PpFS9mrAIrlL7GUYQVusmApfohbgFqYzmwtP1pK9xqYlTm3A1asWMrkfvSlSWrNG8SJUvaamJU5twNWrFjBf6sOxDP2FZsmuWdjtlOxqNSZJey2JDwGXi1DJcjVoxo103capahrl4/I2dz+zq4Irz57H11cOTr9/+9Ti5k9U/F6O+DHz+oiklq09Ll7ahbMLVqBUzWujLKfcaNHlbqkHE7RtHMK1vXbzEp2MnTl3lo2M5p8gby3A1asGMdt3EnQzzngCKe34GuXs8+noy/OXqJUkM16gTczvTk2Je7Nb3XC3lZFkbRq2Y0UIfdzKs3B9zX0/GSiI0AXM704G+9ruvWcS2/uW8sWElD65eQqarY1rmq0XQGI3MjHbduCZMeVTyY777mkXWX7aB6epIM3zXVSXzKsJcLa5Z1oZRD2a00MedDKvkx+ylu397+wFLvmpA3ilMopbrM49rNBhGLZnRQg/xfthxf8xB1iFgYt+AVGp5e9+huJE3hlELZrzQxyHOE0BY6ObsWW0m8g2GQGKWdzmRN4ZRbUzoY+L6BBA2cVtpZcRWo6sjzcjdV7FucHfdnnSUZIS4nHBdw6gF9mxZJSzawg3PN35f32IerFMYalgDkLhY5I3RqJjQV4kwn++cdiuZ4EdhsvREX08mVHRThcI0xXHs3uuujjTlliNKym1jtWuMRsWEvkqE1TFJp+wjL8afeBb0uQn5Wu2Zrg5uXNo9JY79wdVLeGPDSkbuvoq//MKS2LWH5namE3OrWO0ao1FxUh0RWSEie0Vkn4j0B6y/TER+KiInRORzReseEJE9IvKKiPx3kWrXDGwM+noy3P/ZxdOSa95JsOl0K+H3ZXufG5yqFQ/5G8LTO/M3A3/vX4+gz7wzIvLFS4ZKirBrbv55o96UbDwiIingZ8CnyTf73gHcoKov+7ZZAHwQ+BqwSVWfKiz/JPk2g5cVNv0RcIeq/jDs/WrdeKTWhDWbcKFNSKwpRj3xi3fx8tc3rJx8nURjjsHhLGuf2kVuYuo7dnWkWb9qkYmw0TJU2njkEmCfqu4vHOxx4FpgUuhV9Y3CuuLaXQqcBrST/x2ngV/GHH9LERSL78oHT0szZ/aspm6C0tURfg7FvuywScw4528VIg3DTegzwFu+1weBS10Orqo/FpGtwC/IC/1fq+orxduJyM3AzQDd3d0uh25agoRndOw47x0vLfzvjOcYufuqUCu10Um3CetX5V0lQTe7seMnJidlIbqz14L+zWQcRdsqRBozHRehD/KpOymMiPw28HFgfmHRCyJymar+85SDqT4KPAp5143LsZuZYuFZ2L/ZaT/P4i2+WcT9wDyrupx9O9JtHJ9QJmL6kIJcJf4sUpieYHTF+fN4bPuB0GNG9Q8wDOMULpOxB4GzfK/nA4ccj/+HwHZVfVdV3wWeA5bGG2Lr4xp+54/e6OvJsK1/Oa9vWMmape5PQQL8wUUfmdw3FXNu/P7PXsg3Pn9R7NhzEaZNnM6ZPd3O8CZlB4ezPL0zvPxz8faGYYTjIvQ7gHNFZKGItAPXA5scj38AuFxEZolIGrgcmOa6mekEheXF4b6+xaxZ2j0tjnxuZ5pl55wx5ZFMgad3Zidr6N9w6VnEwXsa2da/PPBRL4yjY7lpdfujfPC3bRxxnscolZDk0i7SMFqZkkKvqieAW4At5EX6CVXdIyL3isgqABH5hIgcBD4PPCIiewq7PwW8BuwGdgG7VPWZKpxHU1McUhhGlOV6X99i9t+/kjc2nPo3fNdVvPGr6e4ZvxV8X99ilp1zxpT17alwCfcLZdxEoOLxR+0fxzEUdZxymsUYRqvhFEevqs+q6sdU9RxV/Xph2V2quqnw9w5Vna+qc1T1Q6q6qLB8QlX/RFU/rqoXqOrt1TuV5sazkh+KKANQTip9qbT8weEsPz3wzpR1SvgXo1RyU5yxVPokA6UTkqzzk2FYZmzD0deToasjuBtVOan0XWGdrSQfuXJrgIskN6HBU/AFxnMT3LpxhIEte/nd7tMn/fwpEc79zTmh+xWP3/VJxs+aoszYUglJVn/GMKx6ZUOyftWiRJpYDA5neff9E4HrSuTJOSVmZUfHp4Q/Tqiy7+33ArcNKwXs+fxdEsnmdqa5r29x6YH5sM5PhmEWfUOSVCr9wJa95GqcShv2bqVKAZdy45RbrsDqzxiGWfQNSxJJPo3knijlninODTi9I40IjI7lKspmtcxYwzChb2miMkurSXEtG1cLuloZrJYZa8x0zHXTwgS5LdIpqepF70inppUStgqOhlFfzKJvYcLcFjC1/MCc9hTHT5yc4s/vSKc4Ld3G0bHpZZVTIpxU5cyuDq44fx5bXz1sbhHDaGBKlimuNa1eprhRGRzOBt4QgqJ/zEI3jMaj0jLFxgwgyo9tE5mG0dyY0BuR2ESmYTQ/NhlrGIbR4pjQG4ZhtDgm9IZhGC2OCb1hGEaLY0JvGIbR4pjQG4ZhtDhOQi8iK0Rkr4jsE5H+gPWXichPReSEiHyuaF23iDwvIq+IyMsisiCZoRuGYRgulBR6EUkBDwOfAS4AbhCRC4o2OwB8Cfj7gEN8CxhQ1Y8DlwBvVzJgwzAMIx4uCVOXAPtUdT+AiDwOXAu87G2gqm8U1p3071i4IcxS1RcK272bzLANwzAMV1xcNxngLd/rg4VlLnwMGBWR74rIsIgMFJ4QpiAiN4vIkIgMHT582PHQhmEYhgsuQh/UPdS1Etos4FPA14BPAB8l7+KZejDVR1W1V1V7582b53howzAMwwUXoT8InOV7PR845Hj8g8Cwqu5X1RPAIPC78YZoGIZhVIKL0O8AzhWRhSLSDlwPbHI8/g5groh4ZvpyfL59wzAMo/qUFPqCJX4LsAV4BXhCVfeIyL0isgpARD4hIgeBzwOPiMiewr4T5N02PxCR3eTdQH9bnVMxDMMwgrDGI4ZhGC1AVOMRy4w1DMNocUzoDcMwWhwTesMwjBbHhN4wDKPFMaE3DMNocUzoDcMwWhwTesMwjBbHhN4wDKPFabiEKRE5DLxZ5u4fBv4tweHUmmYfPzT/OTT7+KH5z6HZxw/1OYezVTWwKmTDCX0liMhQWGZYM9Ds44fmP4dmHz80/zk0+/ih8c7BXDeGYRgtjgm9YRhGi9NqQv9ovQdQIc0+fmj+c2j28UPzn0Ozjx8a7BxaykdvGIZhTKfVLHrDMAyjCBN6wzCMFqcphF5EVojIXhHZJyL9AesvE5GfisgJEflc0bqbROTnhX831W7U08ZYyTlMiMhI4Z9rG8dEcRj/7SLysoi8JCI/EJGzfeua5RpEnUMzXIM/FZHdhTH+SEQu8K27o7DfXhG5urYjnzLGss5BRBaIyLjvGvxN7Uc/OcbIc/Bt9zkRURHp9S2rz3VQ1Yb+B6SA14CPAu3ALuCCom0WABcC3wI+51t+BrC/8P/cwt9zm+kcCuvebYJrcAXQWfj7z4CNTXgNAs+hia7BB31/rwL+sfD3BYXtZwMLC8dJNdk5LAD+Tz2vges5FLb7DeCfge1Ab72vQzNY9JcA+1R1v6oeBx4HrvVvoKpvqOpLwMmifa8GXlDVI6p6FHgBWFGLQRdRyTk0Ai7j36qqY4WX24H5hb+b6RqEnUMj4DL+X/tezgG8SItrgcdV9Ziqvg7sKxyv1lRyDo1CyXMo8BfAA8D7vmV1uw7NIPQZ4C3f64OFZdXeN0kqHcdpIjIkIttFpC/ZoTkRd/xfBp4rc99qUck5QJNcAxH5ioi8Rl5k/mOcfWtAJecAsFBEhkXkn0TkU9Udaiglz0FEeoCzVPUf4u5bLWbV4k0qRAKWud7lK9k3SSodR7eqHhKRjwIvishuVX0tobG54Dx+EVkD9AKXx923ylRyDtAk10BVHwYeFpEvAuuAm1z3rQGVnMMvyF+DX4nIxcCgiCwqegKoBZHnICJtwIPAl+LuW02awaI/CJzlez0fOFSDfZOkonGo6qHC//uBHwI9SQ7OAafxi8iVwJ3AKlU9FmffGlDJOTTNNfDxOOA9eTTVNfAxeQ4Fd8evCn/vJO/f/liVxhlFqXP4DeB3gB+KyBvAUmBTYUK2fteh3pMbDpMfs8hP4C3k1OTHopBt/47pk7Gvk58EnFv4+4wmO4e5wOzC3x8Gfk7A5E+9x09e+F4Dzi1a3jTXIOIcmuUanOv7+xpgqPD3IqZOAu6nPpOxlZzDPG/M5CdCs436PSra/oecmoyt23Wo6YdUwYf7+8DPCj/COwvL7iVvdQF8gvzd8j3gV8Ae375/TH7SYx/w75rtHIBPArsLX5DdwJcbdPzfB34JjBT+bWrCaxB4Dk10Df4K2FMY+1a/AJF/SnkN2At8poGvQeA5ANcVlu8Cfgpc06jnULTtDykIfT2vg5VAMAzDaHGawUdvGIZhVIAJvWEYRotjQm8YhtHimNAbhmG0OCb0hmEYLY4JvWEYRotjQm8YhtHi/H+NWeyh4WPvKAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "## Lets make a scatter plot of the true versus predicted values\n", "\n", "plt.scatter(molecules_y_test,molecules_y_pred)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([508., 484., 364., 285., 175., 108., 48., 19., 5., 4.]),\n", " array([6.57534459e-05, 1.15205127e-02, 2.29752720e-02, 3.44300312e-02,\n", " 4.58847905e-02, 5.73395498e-02, 6.87943090e-02, 8.02490683e-02,\n", " 9.17038276e-02, 1.03158587e-01, 1.14613346e-01]),\n", " )" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD4CAYAAAAKA1qZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAP6klEQVR4nO3de4yc1XnH8e9TlktKGsxlQcR2u0RxpEDEJdoQJKqqxU0COIqRAilNWxxqyX+USkFpGkwSqaHqHyaqSlOporJKGhOlxZQUYcUoKTHQmwphzS0xlLA4Lt4a4QUMCUEhMXn6xxyTYb32zO47szPL+X6k0Zz3vGfmfY5n1r99LzMbmYkkqU6/NOgCJEmDYwhIUsUMAUmqmCEgSRUzBCSpYiODLgDgpJNOyrGxsUGXIUmLyvbt25/LzNEmzzEUITA2NsbExMSgy5CkRSUi/rfpc3g4SJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKtbVJ4YjYhfwI+A1YH9mjkfECcBmYAzYBXwsM/dFRABfAi4GXgE+kZkP9r70lrH1W/v11B3t2rBqYNuWpF6Yy57Ab2Xm2Zk5XpbXA9sycwWwrSwDXASsKLd1wI29KlaS1FtNDgetBjaV9ibgkrb+m7PlPmBJRJzaYDuSpD7pNgQS+NeI2B4R60rfKZn5DEC5P7n0LwV2tz12qvS9QUSsi4iJiJiYnp6eX/WSpEa6/RbR8zNzT0ScDNwVEf9zmLExS99Bf80+MzcCGwHGx8f9a/eSNABd7Qlk5p5yvxe4HTgXePbAYZ5yv7cMnwKWtz18GbCnVwVLknqnYwhExLER8SsH2sAHge8BW4A1Zdga4I7S3gJcES3nAS8dOGwkSRou3RwOOgW4vXXlJyPAP2bmNyPiAeDWiFgLPA1cVsbfSevy0Elal4he2fOqJUk90TEEMnMncNYs/c8DK2fpT+CqnlQnSeqrofjzkovVoD6o5ofUJPWKXxshSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKmYISFLFDAFJqpghIEkVMwQkqWKGgCRVzBCQpIoZApJUMUNAkipmCEhSxQwBSaqYISBJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKmYISFLFug6BiDgiIh6KiG+U5dMi4v6IeDIiNkfEUaX/6LI8WdaP9ad0SVJTc9kT+CTweNvy9cANmbkC2AesLf1rgX2Z+U7ghjJOkjSEugqBiFgGrAL+viwHcAFwWxmyCbiktFeXZcr6lWW8JGnIdLsn8NfAZ4Cfl+UTgRczc39ZngKWlvZSYDdAWf9SGf8GEbEuIiYiYmJ6enqe5UuSmugYAhHxYWBvZm5v755laHax7hcdmRszczwzx0dHR7sqVpLUWyNdjDkf+EhEXAwcA7yN1p7BkogYKb/tLwP2lPFTwHJgKiJGgOOAF3peuSSpsY57Apl5bWYuy8wx4HLg7sz8PeAe4NIybA1wR2lvKcuU9Xdn5kF7ApKkwWvyOYFrgE9FxCStY/43lf6bgBNL/6eA9c1KlCT1SzeHg16XmfcC95b2TuDcWcb8BLisB7VJkvrMTwxLUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKjanD4tpOIyt3zqwbe/asGpg25bUe+4JSFLFDAFJqpghIEkVMwQkqWKGgCRVzBCQpIoZApJUMUNAkipmCEhSxQwBSaqYISBJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkinUMgYg4JiK+ExGPRMSOiLiu9J8WEfdHxJMRsTkijir9R5flybJ+rL9TkCTNVzd7Aq8CF2TmWcDZwIURcR5wPXBDZq4A9gFry/i1wL7MfCdwQxknSRpCHUMgW14ui0eWWwIXALeV/k3AJaW9uixT1q+MiOhZxZKknunqnEBEHBERDwN7gbuAp4AXM3N/GTIFLC3tpcBugLL+JeDEXhYtSeqNrkIgM1/LzLOBZcC5wLtnG1buZ/utP2d2RMS6iJiIiInp6elu65Uk9dCcrg7KzBeBe4HzgCURMVJWLQP2lPYUsBygrD8OeGGW59qYmeOZOT46Ojq/6iVJjXRzddBoRCwp7bcAvw08DtwDXFqGrQHuKO0tZZmy/u7MPGhPQJI0eCOdh3AqsCkijqAVGrdm5jci4jHgloj4C+Ah4KYy/ibgqxExSWsP4PI+1C1J6oGOIZCZjwLnzNK/k9b5gZn9PwEu60l1kqS+8hPDklSxbg4HSa8bW791INvdtWHVQLYrvdm5JyBJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKmYISFLFDAFJqpghIEkVMwQkqWKGgCRVzBCQpIoZApJUMUNAkipmCEhSxQwBSaqYISBJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIq1jEEImJ5RNwTEY9HxI6I+GTpPyEi7oqIJ8v98aU/IuJvImIyIh6NiPf2exKSpPnpZk9gP/Anmflu4Dzgqog4HVgPbMvMFcC2sgxwEbCi3NYBN/a8aklST3QMgcx8JjMfLO0fAY8DS4HVwKYybBNwSWmvBm7OlvuAJRFxas8rlyQ1NqdzAhExBpwD3A+ckpnPQCsogJPLsKXA7raHTZW+mc+1LiImImJienp67pVLkhrrOgQi4q3A14GrM/OHhxs6S18e1JG5MTPHM3N8dHS02zIkST3UVQhExJG0AuBrmfkvpfvZA4d5yv3e0j8FLG97+DJgT2/KlST1UjdXBwVwE/B4Zv5V26otwJrSXgPc0dZ/RblK6DzgpQOHjSRJw2WkizHnA38AfDciHi59nwU2ALdGxFrgaeCysu5O4GJgEngFuLKnFUuSeqZjCGTmfzL7cX6AlbOMT+CqhnVJkhaAnxiWpIoZApJUMUNAkipmCEhSxbq5OkgauLH1Wwey3V0bVg1ku9JCcU9AkipmCEhSxQwBSaqYISBJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKmYISFLFDAFJqpghIEkVMwQkqWKGgCRVzBCQpIoZApJUMUNAkipmCEhSxQwBSarYSKcBEfFl4MPA3sx8T+k7AdgMjAG7gI9l5r6ICOBLwMXAK8AnMvPB/pQu9d/Y+q0D2/auDasGtm3Vo5s9ga8AF87oWw9sy8wVwLayDHARsKLc1gE39qZMSVI/dAyBzPx34IUZ3auBTaW9Cbikrf/mbLkPWBIRp/aqWElSb833nMApmfkMQLk/ufQvBXa3jZsqfQeJiHURMRERE9PT0/MsQ5LURK9PDMcsfTnbwMzcmJnjmTk+Ojra4zIkSd2Ybwg8e+AwT7nfW/qngOVt45YBe+ZfniSpn+YbAluANaW9Brijrf+KaDkPeOnAYSNJ0vDp5hLRfwJ+EzgpIqaAPwM2ALdGxFrgaeCyMvxOWpeHTtK6RPTKPtQsSeqRjiGQmb97iFUrZxmbwFVNi5IkLQw/MSxJFTMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxQ0CSKmYISFLFOv49AUmDMbZ+60C2u2vDqoFsV4PhnoAkVcwQkKSKGQKSVDFDQJIqZghIUsUMAUmqmCEgSRUzBCSpYoaAJFXMEJCkihkCklQxvztI0hsM6juLwO8tGgT3BCSpYoaAJFXMEJCkihkCklSxvoRARFwYEU9ExGRErO/HNiRJzfX86qCIOAL4W+ADwBTwQERsyczHer0tSW8ug7wyaVAGfUVUP/YEzgUmM3NnZv4UuAVY3YftSJIa6sfnBJYCu9uWp4D3zxwUEeuAdWXx5Yh4Yp7bOwl4bp6PHVbOafF4M87LOS2guH7eDz0J+LWm2+9HCMQsfXlQR+ZGYGPjjUVMZOZ40+cZJs5p8Xgzzss5LQ5lTmNNn6cfh4OmgOVty8uAPX3YjiSpoX6EwAPAiog4LSKOAi4HtvRhO5Kkhnp+OCgz90fEHwPfAo4AvpyZO3q9nTaNDykNIee0eLwZ5+WcFoeezCkyDzpcL0mqhJ8YlqSKGQKSVLGhDoFOXz8REUdHxOay/v6IGGtbd23pfyIiPrSQdR/OfOcUER+IiO0R8d1yf8FC134oTV6nsv5XI+LliPj0QtXcScP33pkR8d8RsaO8XscsZO2H0uC9d2REbCpzeTwirl3o2g+lizn9RkQ8GBH7I+LSGevWRMST5bZm4ao+vPnOKSLObnvfPRoRv9PVBjNzKG+0Tio/BbwDOAp4BDh9xpg/Av6utC8HNpf26WX80cBp5XmOWORzOgd4e2m/B/i/Qc+n6Zza1n8d+Gfg04OeTw9epxHgUeCssnzim+C993HgltL+ZWAXMLZI5jQGnAncDFza1n8CsLPcH1/axy/yOb0LWFHabweeAZZ02uYw7wl08/UTq4FNpX0bsDIiovTfkpmvZuYPgMnyfIM27zll5kOZeeDzFjuAYyLi6AWp+vCavE5ExCW0fgD7eQXZXDWZ0weBRzPzEYDMfD4zX1ugug+nyZwSODYiRoC3AD8FfrgwZR9Wxzll5q7MfBT4+YzHfgi4KzNfyMx9wF3AhQtRdAfznlNmfj8znyztPcBeYLTTBoc5BGb7+omlhxqTmfuBl2j95tXNYwehyZzafRR4KDNf7VOdczHvOUXEscA1wHULUOdcNHmd3gVkRHyr7LJ/ZgHq7UaTOd0G/JjWb5ZPA3+ZmS/0u+AuNPk5X8z/R3QUEefS2pN4qtPYYf4bw918/cShxnT11RUD0GROrZURZwDX0/qNcxg0mdN1wA2Z+XLZMRgWTeY0Avw68D7gFWBbRGzPzG29LXHOmszpXOA1WocYjgf+IyK+nZk7e1vinDX5OV/M/0cc/gkiTgW+CqzJzJl7QAcZ5j2Bbr5+4vUxZVf1OOCFLh87CE3mREQsA24HrsjMjgm/QJrM6f3AFyNiF3A18NnyQcNBa/re+7fMfC4zXwHuBN7b94o7azKnjwPfzMyfZeZe4L+AYfgeniY/54v5/4hDioi3AVuBz2fmfV09aNAnQg5zgmSE1rHi0/jFCZIzZoy5ijeeyLq1tM/gjSeGdzIcJ+eazGlJGf/RQc+jV3OaMeYLDM+J4Sav0/HAg7ROoI4A3wZWLfI5XQP8A63fUo8FHgPOXAxzahv7FQ4+MfyD8nodX9onLPI5HQVsA66e0zYHPekO/yAXA9+ndVzrc6Xvz4GPlPYxtK4qmQS+A7yj7bGfK497Arho0HNpOifg87SOyz7cdjt50PNp+jq1PccXGJIQ6MF77/dpnej+HvDFQc+lB++9t5b+HbQC4E8HPZc5zOl9tH67/jHwPLCj7bF/WOY6CVw56Lk0nVN53/1sxv8RZ3fanl8bIUkVG+ZzApKkPjMEJKlihoAkVcwQkKSKGQKSVDFDQJIqZghIUsX+H22SkG/1SI3eAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "## and a histogram of the magnitude of the error\n", "plt.hist(abs(molecules_y_test- molecules_y_pred))" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Intercept: 0.25 \n", " Coefficients: [ 8.12849803e-04 -1.03028728e-03 -9.85332103e-05 1.19988936e-03\n", " -4.89037868e-04 -4.57045254e-04 -2.04281125e-03 -1.43264786e-03\n", " -7.22503668e-04 -4.02626465e-02 -1.11831216e-01 2.96729799e-02\n", " 1.21237161e-01] \n", "\n", "Mean squared error: 0.00115\n", "Coefficient of determination: 0.2940\n" ] } ], "source": [ "# The intercept and coefficients\n", "print('Intercept: {0:.2f} \\n Coefficients: {1} \\n'.format(regr.intercept_, regr.coef_))\n", "\n", "# The mean squared error\n", "print('Mean squared error: %.5f'\n", " % mean_squared_error(molecules_y_test, molecules_y_pred))\n", "\n", "# The coefficient of determination: 1 is perfect prediction\n", "print('Coefficient of determination: %.4f'\n", " % r2_score(molecules_y_test, molecules_y_pred))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "oU8GH-H0CyNF" }, "source": [ "## Gradient Descent \n", "\n", "Although we can compute the coefficients $\\mathbf{\\theta}$ for a given data set, this approach has some restrictions:\n", "\n", " * To compute $(X^\\intercal X)^{-1}$, the matrix $(X^\\intercal X)$ has to be non-singular. \n", " * Even if we can compute $(X^\\intercal X)^{-1}$, this is computationally cost for large data sets. If we have $10.000$ data points for instance, $(X^\\intercal X)^{-1}$ will have a dimension of $10.000 \\times 10.000$, requiring a large ammount of memory to store it, and a very large computational time for its inversion. \n", "\n", "The gradient descent is a computational technique that can be used to minimize functions interativelly. The idea is as follow:\n", "\n", " 1. Guess an initial value for $\\mathbf{\\theta}$.\n", " 1. Compute the gradient of the error function at $\\mathbf{\\theta}$.\n", " 1. Adjust the the value of $\\mathbf{\\theta}$ doing a \"small step\" in the oposite direction of the gradient.\n", " 1. Repeat 2 and 3 untill convergence.\n", "\n", "This process is illustrated in the picture bellow\n", "\n", "![](https://miro.medium.com/proxy/1*wsBakfF2Geh1zgY4HJbwFQ.gif)\n", "\n", "The size of the step in 3 is a parameter of the algorithm, and is known as **learning rate**. More formally, the update rule is:\n", "\n", "$$ \\mathbf{\\theta}_{n+1} = \\mathbf{\\theta}_{n} - \\alpha \\nabla_\\theta E(\\mathbf{\\theta})$$\n", "\n", "The learning rate $\\alpha$ shoulde be choose wisely:\n", " * A very small learning rate may delay de convergence, requiring more iterations to converge.\n", " * A very large learning rate could diverge instead of converging, and we the minimum will not be reach.\n", "\n", "The picture bellow illustrated the influence of the learning rate minimizing a quadratic function. The first two converge to the minimum, although the case in the middle converges more quickly. On the other hand, the third graph has a too high learning rate, and the gradient descent diverges from the minimum. \n", "\n", "![](https://miro.medium.com/proxy/1*Q-2Wh0Xcy6fsGkbPFJvMhQ.gif)\n", "\n", "In general, the setting of the learning rate requires some experimentation. There are some approaches that uses a variable learning rate, aiming to speed up learning while avoiding divergence and local minimas (you can find more details [here](https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB2_CIV-gradient_descent.html))\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": {}, "colab_type": "code", "id": "UTZ3i9C-ypKm" }, "outputs": [], "source": [ "# Create a gradient descent linear regression model\n", "# note that the learning rate is name eta0 in sklearn\n", "# You can also choose the type of the learning rate\n", "regr = linear_model.SGDRegressor(learning_rate='constant', eta0 = 0.001, max_iter=2000)\n", "\n", "# Train the model using the training sets\n", "regr.fit(diabetes_X_train, diabetes_y_train)\n", "\n", "# Make predictions using the testing set\n", "diabetes_y_pred = regr.predict(diabetes_X_test)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 104 }, "colab_type": "code", "id": "0X_pBSodzRu7", "outputId": "79105116-a571-4fc6-a7f5-165307c60785" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Intercept: 152.81 \n", " Coefficients: 660.64 \n", "\n", "Mean squared error: 3024.76\n", "Coefficient of determination: 0.37\n" ] } ], "source": [ "# The intercept and coefficients\n", "# They are a bit different from linear regression due to som numerical issues and regularization (discussed later)\n", "print('Intercept: {0:.2f} \\n Coefficients: {1:.2f} \\n'.format(regr.intercept_[0], regr.coef_[0]))\n", "\n", "# The mean squared error\n", "print('Mean squared error: %.2f'\n", " % mean_squared_error(diabetes_y_test, diabetes_y_pred))\n", "\n", "# The coefficient of determination: 1 is perfect prediction\n", "print('Coefficient of determination: %.2f'\n", " % r2_score(diabetes_y_test, diabetes_y_pred))" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 252 }, "colab_type": "code", "id": "F8aHgWT46Nte", "outputId": "8d6fb844-f25d-4025-9349-677656785e23" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADrCAYAAABXYUzjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAPL0lEQVR4nO3dXYhc5R3H8d+ZxGiOWoumJV645wgiLUWK7Cq+kor2qha13jlGhdKxvVLx7eKoNzK0FxoVWqpzIagzikUxQa9bakoaYSJSLCZV0plVqMFI1JCJMe48vRhnz87uJjlnds485zzn+4FFPMyT/NfVXx7/z8vxjDECAExfxXYBAFBWBDAAWEIAA4AlBDAAWEIAA4AlBDAAWLI+zYc3bdpkwjDMqBQAcNOePXsOGmN+sPx5qgAOw1DtdntyVQFACXie113tOS0IALCEAAYASwhgALCEAAYASwhgALCEAAbgtFarpTAMValUFIahWq2W7ZIWpdqGBgBF0mq1VKvV1Ov1JEndble1Wk2SVK1WbZYmiRkwAIdFUbQYvkO9Xk9RFFmqaBQBDMBZ8/PzqZ5PGwEMwFkzMzOpnk8bAQzAWfV6Xb7vjzzzfV/1et1SRaMIYADOqlarajQaCoJAnucpCAI1Go1cLMBJkpfmpZxzc3OGy3gAIB3P8/YYY+aWP2cGDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDACWEMAAYAkBDCCXWq2WwjBUpVJRGIZqtVq2S5q49bYLAIDlWq2WarWaer2eJKnb7apWq0mSqtWqzdImihkwgNyJomgxfId6vZ6iKLJUUTYIYAC5Mz8/n+p5URHAAHJnZmYm1fOiIoAB5E69Xpfv+yPPfN9XvV63VFE2CGAAuVOtVtVoNBQEgTzPUxAEajQaTi3ASZJnjEn84bm5OdNutzMsBwDc43neHmPM3PLnzIABwBICGAAsIYCBHCrDKTBwEg7InbKcAgMzYCB3ynIKDAQwkDtlOQUGAhjInbKcAgMBDOROWU6BgQAGcqcsp8DASTgAjltYkHbskMJQuvRSyfOmXwMn4QCUxoED0l13DcJ2/Xrp1lul2Vmp0bBd2Sj2AQNwwl//Kv3ud9J//nPiz+zfP716kmAGDKCQvv5aqtcHs1zPk66//uThe/rp0iOPTK++JAhgAIXx0UfSL34xCNyNG5MF6hNPSMeODQL77LOzrzENWhAAcssY6bHHpCeflI4eTTbmpz+V/vQn6eqrs61tEghgALly4IC0ZYu0b1/yMb/9rfT449KmTdnVlQUCGIB1O3ZIN9+c/PO+L/35z9Ltt0uVAjdSC1w6gKI6fly64454AS1J+N5wg/Tvfw/aEkeODMYXOXwlZsAApmTv3kF/9ptvko+55hrprbekc87Jri6bCv7nB4A8++Mf41nuj3+cLHyfeWYwyzVG2rnT3fCVmAEDmKAvv5RuvFH6xz+Sj9mwQXrvvUFAlw0zYABr8vbb8Sz3+99PFr5btw725hoz+GsZw1cigAGk1O9LDzwQh+6WLcnGvfFG3Fp48cXBzLfsaEEAOKVPPpGuukr6+OPkYy6+WPr736XNm7Orq+iYAQNY1auvxrPcCy5IFr6PPjqYIRszOEhB+J4cM2AAkgZ3JWzdKr32Wrpx//yndMUV2dTkOmbAQIm9+WY8y924MVn4/vzn0uHDcT+X8B0fM2CgRIwZHG7YtSvduEZD+s1vsqmpzAhgwHHd7uB1PGmcc47UbksXXZRJSfgOLQjAQU88EbcWkoZvEAzuaDBG+uILwncaCOBlWq2WwjBUpVJRGIZqtVq2SwJO6fhx6bTT4tB98MFk437/+7iX2+kM3p+G6eEf9xKtVku1Wk29Xk+S1O12VavVJIlXgiN3du+Wrrwy/bj9+6ULL5x8PUiPGfASURQthu9Qr9dTFEWWKgJGDd/063nJw/fyy+O9ucYQvnlCAC8xPz+f6jnclKc21KFDceB6nvTCC8nGvfZaHLjvvDMYi/whgJeYmZlJ9RzuGbahut2ujDGLbahphvDSN/2ee27ycV98EYfurbdmVx8mhwBeol6vy/f9kWe+76ter1uqCNNmow1lzOgsN+mr02u1OHCNcfveXFcRwEtUq1U1Gg0FQSDP8xQEgRqNBgtwJTKtNlS7HQdumtfqtNtx4D733ERLggUE8DLValWdTkf9fl+dTofwLZks21C//GUcupddlnzccG+uMdLs7JrLWCFPPe+yIYCBJSbZhjpyZLS18NZbycbdffdoayHLvbl56HmXGQEMLLHWNtTLL8eBe9ZZyX/fffviwH322TGLHwNbL+3yjDGJPzw3N2fa7XaG5QDFc+aZ0rIMSyTFf3qZqVQqWi0DPM9Tv9+3UJGbPM/bY4yZW/6cGTCQUrc72lpIGr6NxmhrIQ/YemkXAQwk8NBD6S+3kQYHKYaBm8frHNl6aRd3QQCrWFgYb/HrmmuknTsnX09Whr3tKIo0Pz+vmZkZ1et1dv9MCT1g4Dvbt0u33JJ+3NtvS9deO/l64I4T9YCZAaPUxr0j4dtvpXXrJlsLyoceMErls89GF9CSevjh0QU0wheTQADDeVEUB+4Pf5h83N69ceD+4Q/Z1YfyogUB5xiT7n6F5WOBaWEGDCfs3Dne5TbbtuVvby7KgxkwCut735MOH04/7quvpLPPnnw9QFrMgFEYR4+OLqAlDd/Nm0dnuYQv8oIARq5t2xYH7rIDWye1fXscuP/7X3b1AWtBAFvEPayrWzrLvf/+5OMWFuLQvemm7OoDJoUAtoR7WGN79463N/e660ZbC+PufABs4SiyJWEYqtvtrngeBIE6nc70C5qyiy+WPvww/bgPP5Quumjy9QBZ4ihyzkzr3WN5wd5cYCX+p82SMtzD+sor4+3Nvf9+9uaiHJgBW1Kv11Wr1UZeB+PCPazjXm5z+HC6V/gALmAGbMla3z2WF4cOjbeAJo3OcglflBGLcEjt17+Wnn8+/bg335RuvHHy9QB5xyIc1mTc1kK/P/5YwHW0ILCq998fr7Vw1VWjrQXCFzgxZsBYNDsrvftu+nH//W+6F1UCGCCAS2zcF09KbA8DJoEWRMm8/HLcVkgTvo0Ge3OBSWMGXALj9mGPHZM2bJhsLQBizIAdNO6LJy+8cHSWS/gC2SKAHXH33eO9eLLdjgN3//7VP8O1mUA2aEEU2LithTQ93OG1mcMj08NrMyUV7tQekDfMgAtk9+7xWgv33DP+AloURSP3VUhSr9dTFEXpfiEAKzADzrnzz5c+/TT9uIMHpfPOW/vvX7ZrM4FpYgacM8eOjc5y04Tv0lnuJMJXKse1mYAtBHAOvPpqHLhnnJF83F/+kv3e3Hq9Ln/Z2zBduDYTyANaEJZceqn03nvpxy0sTPfdZ8OFtiiKND8/r5mZGdXrdRbggAngOsop+fxzadOm9OOuvFLatWvy9QCYnhNdR0kLIkPPPBO3FtKE7wcfxG0FwhdwFwE8QcPrF4df996bbNy55472cn/0o5Wf4TAE4B4CeI2W3pubpjf7xhtx4H7++ck/OzwM0e12ZYxZPAxBCAPFRg94DLffLo2TfUePptvlMBSGobrd7ornQRCo0+mk/wUBTBWvJFqDr7+WNm5MP+6228YL6uU4DAG4iRbECezYEbcW0oTvv/4VtxYm1SHgMATgJgJ4ia1b49C9+ebk4/r9OHQvuWTydXEYAnBTqQP4yJHRXQvNZrJxTz013RdPVqtVNRoNBUEgz/MUBIEajQaHIYCCK10Av/NOHLhnnZV83MGDceAm3V42SdVqVZ1OR/1+X51Oh/A9BbbtoQicD2BjpMcei0P3iiuSjbvllmwut0H22LaHonByG9qBA9KWLdK+fenGvfvu4I4GFBvb9pA3zh9FXrprYfPmZOF7332Dy22Gs1zC1w1s20NRFDaAjx+X7rgj/a6Fv/0tDtxt26Z7sximg217KIpCxc/evdLppw8Cd8MG6aWXTj3m6qulQ4fi0P3ZzzIv0zlFW9Bi2x4KwxiT+Gt2dtZM265dS5fCkn09/fTUy3RWs9k0vu8bSYtfvu+bZrNpu7STajabJggC43meCYIg9/XCbZLaZpVMzfUi3IsvSnfeeerPrV8/uNz8Jz/JvqayYUELWLtCLsKdrMVw222D96cZM+gHE77ZYEELyE6uA3j79tG/f/310XsWNmywU1eZsKAFZCfXAXzmmaPd3V/9ynZF5cOCFpCdXAcw7OMeCiA7uV6EAwAXFHIRDisVbU8ugBPjjRgFMrxkptfrSdLiJTOSaAkABcQMOAeSzmqjKFoM36Fer6coiqZRJoAJYwZsWZpZLXtyAbcwA7YszayWPbmAWwhgy9LMatmTC7iFALYszayWPbmAWwhgy9LOank3HOAOAtgyZrVAeXESDgAyxkk4AMgZAhgALCGAAcASAhgALCGAAcASAhgALCl1AHO3LgCbSnsbGnfrArCttDNg7tYFYFtpA5i7dQHYVtoA5m7d4qJ3D1eUNoBduVu3bGE07N13u10ZYxZ7965/33CUMSbx1+zsrHFJs9k0QRAYz/NMEASm2WzaLimVZrNpfN83kha/fN8/6fdR9O85CIKR73f4FQSB7dKAE5LUNqtkKrehFVgYhup2uyueB0GgTqez4vnynR/SYNZfpOsvK5WKVvt31vM89ft9CxUBp8ZtaA5Ku5Dows4PevdwCQFcYGnDyIWdH6707gGJAC60tGHkwuyRN4jAJQRwgaUNI1dmj7wXD64oRACXbatVGmnCiNkjkC+53wXhwso9gHIr7C4IF1buAWA1uQ9gF1buAWA1uQ9gF1buAWA1uQ9gV1buAWC5XAdwq9Va7AGvW7dOkli5Lwl2vqAMcvtGjOW7HxYWFhZnvoSv23hbCcoit9vQ0l40A3fws4drCrcNjd0P5cXPHmWR2wBm90N58bNHWeQ2gNn9UF787FEWuQ1g7i0oL372KIvcLsIBgCsKtwgHAK4jgAHAEgIYACwhgAHAEgIYACxJtQvC87zPJK08IwoAOJnAGPOD5Q9TBTAAYHJoQQCAJQQwAFhCAAOAJQQwAFhCAAOAJQQwAFhCAAOAJQQwAFhCAAOAJf8HalCvD7PAM9YAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot outputs\n", "# The graph is similar to linear regression using ordinal least squares\n", "plt.scatter(diabetes_X_test, diabetes_y_test, color='black')\n", "plt.plot(diabetes_X_test, diabetes_y_pred, color='blue', linewidth=3)\n", "\n", "plt.xticks(())\n", "plt.yticks(())\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "oTMndk2HDXJD" }, "source": [ "## Polinomial Regression\n", "\n", "Our model function need not be linear (a straight line) if that does not fit the data well.\n", "\n", "We can change the behavior or curve of our model function by making it a quadratic, cubic or square root function (or other polinomial forms).\n", "\n", "Recall that a linear model is $pred_\\theta(x) = \\theta_0 + \\theta_1 x$. We can create additional features based on $x$ to add higher order polimonial terms. For instance, if we want to include a quadratic term, then we can add an extra feature $x_i^2$ an run the least squares algorithm in the augmented data set to compute the quadratic model function $pred_\\theta(x_i) = \\theta_0 + \\theta_1 x_i + \\theta_2 x_i^2$. Similarly, if we want to also add a cubic function, we can add another extra feature $x_i^3$ so that the model function is now $pred_\\theta(x) = \\theta_0 + \\theta_1 x_i + \\theta_2 x_i^2 + \\theta_3 x_i^3$.\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "colab_type": "code", "id": "SXaXlEA2bNPI", "outputId": "b03779d3-1cac-4a3a-98c0-209f2d4937bf" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3wc9Z34/9dn+2pVrV4t9y7LxhWDIYAhwRAgBNLujpRfSL0k3KVw31wuJL/4m3LpkOQuBY5cHCAhAQwh9GaMq1ywcS+SrWL1tr3M5/vHrmXJlm1Z2tWu5PfTj33sTtmZt0bye2Y+85n3KK01QgghxhdTsgMQQggRf5LchRBiHJLkLoQQ45AkdyGEGIckuQshxDhkSXYAAHl5ebqysjLZYQghxJhSU1PTprXOH2xaSiT3yspKtm3bluwwhBBiTFFK1Z1rmjTLCCHEOCTJXQghxiFJ7kIIMQ5JchdCiHFIkrsQQoxDktyFEGIckuQuhBDjkCR3IYRIAv+Bgxy5cTVtv/pVQpYvyV0IIZLA8+Z6gkePEjx+IiHLH3JyV0qZlVI7lFLPxIYnKaU2K6UOKaUeU0rZYuPtseHDsemVCYlcCCHGMPf6NwFIv/KKhCz/Yo7cvwjs6zf8feAnWutpQCfwidj4TwCdWuupwE9i8wkhhIgxPB68NTVgMpG2fHlC1jGk5K6UKgNWA7+NDSvgGuDx2CwPA7fGPt8SGyY2/drY/EIIIQDP5i0QCuGcNw9LTk5C1jHUI/efAl8FjNhwLtCltQ7HhuuB0tjnUuAEQGx6d2x+IYQQRNvbAVwrr0zYOi6Y3JVSNwEtWuua/qMHmVUPYVr/5d6tlNqmlNrW2to6pGCFEGKs01rjfiOa3NOvTGJyB1YA71VK1QKPEm2O+SmQrZQ6VTK4DGiMfa4HygFi07OAjjMXqrX+tdZ6kdZ6UX7+oOWIhRBi3AnW1hKqr8ecnY1jzpyEreeCyV1r/W9a6zKtdSXwQeAVrfVHgFeB98dmuwt4KvZ5XWyY2PRXtNZnHbkLIcSlyBPrJeNasQJlNidsPSPp5/414F+UUoeJtqn/Ljb+d0BubPy/APeOLEQhhBg/3Kfa2xPUBfKUi3oSk9b6NeC12OejwJJB5vEDd8QhNiGEGFcMvx/vlq0ApF+R2OQud6gKIcQo8W6rQfv9OGbPxpKXl9B1SXIXQohR4n7jdQBcCT5qB0nuQggxKrTWuF95FYD0q69O+PokuQshxCgIHj4c7QI5YQLO+VUJX58kdyGEGAW9/Y7aE9kF8hRJ7kIIMQrcr8aS+7uuHpX1SXIXQogEC7e14du1C2WzkX755aOyTknuQgiRYO7XXwetSVu+DJPLNSrrlOQuhBAJdqq9PeNd14zaOiW5CyFEAhl+P54NG4DRa28HSe5CCJFQno0bo3elzpmDtbBw1NYryV0IIRLI/eprAKRf865RXa8kdyGESBAdidD7yisAZFwzeu3tIMldCCESxltTQ6StDWtFBfaZM0d13ZLchRAiQXqffwGAzBuuR6nBnkCaOJLchRAiAbRh0PtCNLlnXH/DqK9fkrsQQiSAb+dOwq2tWEtKcMxN3LNSz0WSuxBCJEDPc88BkHHDDaPeJAOS3IUQIu6iTTIvAtH29mSQ5C6EEHHmf/ttwidPYikuxjF/flJikOQuhBBx1nOql8z1q5LSJAOS3IUQIq601vQ+/zwQbW9PFknuQggRR74dOwk1NmIpKMBZXZ20OCS5CyFEHHU/vQ6AzJtuQpmSl2IluQshRJzoUIjev0e7QGbdfFNSY5HkLoQQceJ+800iXV3Ypk4Z9VoyZ5LkLoQQcdLz9DMAZN383qT1kjlFkrsQQsRBxO3pK++bddPqJEcjyV0IIeKi96UX0X4/zssuw1pamuxwJLkLIUQ8nG6SuTnJkURJchdCiBEKt7bi2bgRrFYy3528G5f6k+QuhBAj1P3UU2AYpK9ciTk7O9nhAJLchRBiRLTWdD3+FwCyb789ydGcdsHkrpRyKKW2KKV2KaXeUUp9KzZ+klJqs1LqkFLqMaWULTbeHhs+HJtemdgfQQghkse3fTvB2los+fmkr7wy2eH0GcqRewC4Rms9H6gG3q2UWgZ8H/iJ1noa0Al8Ijb/J4BOrfVU4Cex+YQQYlzq+vPjAGTdeivKYklyNKddMLnrKHds0Bp7aeAa4PHY+IeBW2Ofb4kNE5t+rUp2b34hhEiAiNtNT6wCZPbt70tyNAMNqc1dKWVWSu0EWoAXgSNAl9Y6HJulHjjVsbMUOAEQm94N5A6yzLuVUtuUUttaW1tH9lMIIUQS9PztWbTPR9rixdgqK5MdzgBDSu5a64jWuhooA5YAswabLfY+2FG6PmuE1r/WWi/SWi/Kz88farxCCJEyuh6PNl5kvz91LqSeclG9ZbTWXcBrwDIgWyl1qoGpDGiMfa4HygFi07OAjngEK4QQqcJ/4CD+3bsxZWSQcX1ynpN6PkPpLZOvlMqOfXYC1wH7gFeB98dmuwt4KvZ5XWyY2PRXtNZnHbkLIcRY1vXYYwBk3rQak9OZ5GjONpRLu8XAw0opM9GdwZ+01s8opfYCjyqlvgPsAH4Xm/93wP8qpQ4TPWL/YALiFkKIpIm43XQ/+SQAOR/8UJKjGdwFk7vW+m1gwSDjjxJtfz9zvB+4Iy7RCSFECup+4kkMr5e0xYtxzJie7HAGJXeoCiHERdCGQefatQDk/MM/JDmac5PkLoQQF8Hz1sboHalFRWRce02ywzknSe5CCHER+o7aP/jBlLoj9UyS3IUQYoiCJ07gfu01lNVK9h3vv/AXkkiSuxBCDFHnHx8Brcm88UYsuWfdeJ9SJLkLIcQQRHp66PrTn4DUvpB6iiR3IYQYgs5HH8PweEhbtgznvLnJDueCJLkLIcQFGIEAHb//PQC5n/jEBeZODZLchRDiArqffIpIWxv2WbNwXbEi2eEMiSR3IYQ4Dx2J0PHgg0D0qH2sPJ5CkrsQQpxH70svE6yrw1paSua7b0h2OEMmyV0IIc5Ba037b34DwISPfSylb1o6kyR3IYQ4B/drr+Hfswdzbm7KPUbvQiS5CyHEILTWtN5/PwC5n/z/UrJm+/lIchdCiEH0vvgigb37sOTnk/PBsfdYCknuQghxBm0YtN3/AAC5n/oUJocjyRFdPEnuQghxht7nniNw6BCW4mKy7xybzx6S5C6EEP3ocJjWB34BQN6nP43JZktyRMMz5pN750kP8vxtIUS8dP3lrwSPHsVaWkr2bbcmO5xhG9PJ/cmf7OCP922mrd6d7FCEEONAxO2m9ec/B6DgX/8FNUaP2mGMJ/ecojQADm5pTnIkQojxoP2/f02kvR1ndTUZ73lPssMZkTGd3KcvKQLg0NZmDEOaZoQQwxesb6Dj4YcBKLz3a2Omhsy5jOnkXjQ5k4xcB56uAE2HupIdjhBiDGv98Y/QwSCZq1fjrK5OdjgjNqaTu1KK6YsLATi45WSSoxFCjFXemhp6nv07ym6n4F//JdnhxMWYTu6G309J4BAAh7e3Eg5GkhyREGKs0cEgJ++7D4AJH/8Y1pKS5AYUJ2M2uetwmLqP/AOeb95Dbo4m6AtzdGdrssMSQowx7Q8+RODQYawTK8j71KeSHU7cjJ36lWdQFguZN9+E/513yN/5JO0Tb2Pvhqa+i6wiObTWuDsDdLf66GmLvrw9QfzuEH5PiFAgQiSs0YbGMDQWqwmL1YTZasLhsuLMtOFMt5Ke4yArz0lmvoOMCQ5M5jF7HCJSWLCujrZf/QqA4m9+c0yWGTiXMZvcASbcdRfebdvIf20DBytupuFAJz1tPjLzxlb1trHM0x2g6XA3TUe6aDvhpr3BTcAbjus6zBYTE0pc5JWnk1+eQV5ZOnkVGVht5riuR1xatNac/Na30YEAme+9Gdfllyc7pLga08ldKUXJmjUEbn8/+c3baS5awt4NjSy7ZUqyQxu3IiGDhoOd1O5u58S+DrqavWfN40i3klOYRkaeg8xcJ65sOw6XFUe6FbvTgsmsMJkVTz/9ND//2f20t3ZQVlrBx++6m8XVS/H2hujt8NMTO/p3dwZoPd5L6/Fe9tEEgMmsKKzMpGR6NqXTcyiakiXJXlyUnnXr8Lz1FuasLArvvTfZ4cSdSoVb9xctWqS3bds27O/7du9h56f/ne3zvoDDpvnoj96F2Sqn8fESDkU4vqeDg1tPUvdOB+HA6QvXFruZ4smZFE/NJr8ig/zyDNKybBfsI7x27VruvvtuvN7TO4e0tDR+/etf85GPfGTAvAFfmPb6XlpPuGk70UvrcTftjW7o96drMiuKp2YzqSqPyqpcsvLT4vPDi3Ep1NjI0VtuxejtpXjNmjH3II5TlFI1WutFg04bD8kdoH3tH3n67xE86aWsvMbFvDuXxim6S5PWmubaHva+2ciR7a0EfaebWvLK05k4N5eJc/MoqMzAPIz28MrKSurq6s4aP3HiRGpray/4/YA3ROPhbhoOdtJ4sIvWE70Dkn1OsYtJVblMWVhAfkXGmL8hRcSPNgyOf+zjeDdvJv2aayj7xQNj9u9j3Cb33+7+Lbtad3FD5Q1cVXoVb3/jj+zsmUaWp447v7sKW1lZAqId30LBCAc2neSd9Q20nThdsyevPJ3pi4uYuqiAjAkjv+hkMpkGLfimlMIwjItent8Tom5PO7W72zi+p52g//TZRVa+k2mLC5m6qIDckvQRxS3GvvYHH6LlBz/AnJvL5HVPYcnNTXZIwzai5K6UKgd+DxQBBvBrrfXPlFITgMeASqAWuFNr3amiu8CfATcCXuCjWuvt51vHcJP7bU/dxuGuwwDYTDauyl/J9CeuJ6LsLD+5lvm//wnmzMyLXu6lyOcOsvu1Bna/Wo/fEwLA4bIyc3kRs1aUMKHYFdf1jfTI/XwiYYPGw10c29nG4e0t+HqCfdMmlLiYsayIGUuLcGXZR7QeMfb4Dxyg9v13oEMhyn71SzLe9a5khzQiI03uxUCx1nq7UioDqAFuBT4KdGitv6eUuhfI0Vp/TSl1I/DPRJP7UuBnWuvztpEMN7m3+dp4se5Fnq99nu3N29ForjjyXua2XEtBSw1TzK8z+bcPkp1VcNHLvlT0tPvY+dIJ9m1oJByMHjEXVGYy/5oyJi/Ix2JNzEXKC7W5a60xPB7CLa2EW1oIt7UR6enG6HVjuHuJuN0YHg+Ew+hwBG1EIBwBrVF2e+xlQ1nttKt8TgSLqO9KJxiONiEpBRVzcph9RRkT5+UOq2lJjC0Rt4faD3yA4JEjZN95J8Xf/layQxqxuDbLKKWeAh6Iva7WWjfFdgCvaa1nKKX+O/b5kdj8B07Nd65lDrvN/cBz4GmBrDKabU5e6trHq4e2MP+F2zFpxfLN93GouJNXPr+UlZOv5eryqylNL7349YxDnq4AW5+tZd+bjX1F1yrm5LLwhgpKpmWPShvk2j/8gfu/8Q1srW1UFxZw+4oVTHI6CR0/Qai5Ge09uyfOSBjKTHvuHJqKltOeOwetojsum+Gj3NrI1NIgebNLsU+Zgm3SZMzp8T1bEcmjtabhS/fQ+/zz2KZOYdJjj2Fyjf3fb9ySu1KqEngDmAsc11pn95vWqbXOUUo9A3xPa/1mbPzLwNe01tvOWNbdwN0AFRUVlw12in5Ba++AQy8MHOfM4enuf+Z4xwIKm99gzr7H2DZV8aP3mYiYFdOzp3FV+dWsKF1BVX4VVpP14tc7hvndIba/UMfuV+sJhwyUgmmLC1lw/UTyyhLXHm14PPgPHiRw4AD+AwcI7D9A4ODB6NH3OSinE0tBPpb86MuclYU5IwOTKx1TRjomlwtlsaIsZjCZUGYzKIUOBtHBIEYggA4EMbxeIl1dRDo7o+9dXXjaPdSriTQVLMXjKu5bZ3bXIUoa11PQugtbYR6OmTNxzJmDY+4cHHPmYC2I31ng2rVr+frXv87x48epqKhgzZo1Z/UUEvHR/rsHafnP/8SUnk7ln/6EffKkZIcUF3FJ7kqpdOB1YI3W+q9Kqa5zJPe/Ad89I7l/VWtdc65lD/fI/Z0nf4jt5HYygy2kB5px+powGSHaQxU82v4zzARZtu0/sLt7OTYlwv+91UJ3v77QLkwssRewImsGlxctobxwHmSVQ1pu9Lw9xV1McggHI+x8+QQ7nq/ru9g4eUE+S2+ezISS+B7BaK0J1dXh3bET344d+HbuJHDoEAzyt2bJz8c2dQq2ionYKiqwTazAWl6BtbQkmrwT+HvQkQihlhaadtWzf3sHtfVmwka0ecYa7KX45EZKG9/E6W8/HW9hIc6FC0hbtIi0xYuxT52KMl18k87FdAUVI+PZtInjH/8EGAZlv3iAjGuvTXZIcTPi5K6UsgLPAM9rrX8cG9fX3JKsZpmPPrSF1w6criejMMillxLVxlJ3IdmhfOxqF8vf/A2mkMZSEKFlVQ9vZtl5y+nkqG3gUXt5KMQSX4CFwTBVKouCtCLIKMaUWYI1pxRzVglkFENmcfTdmrw7YYeaHLTWHNneylt/OUxvhx+A8lk5LL1lCoWV8bnYrCMR/Pv24928Ce+2Gnw7dxLp7Bw4k9WKfcoUHDNmYJ8xA8fM6Hsq9VQI+sIc3NrMntcbaG843VOo0NlNeVcNmbtfBHfPgO+Ys7JwLlpE2uJFuJYuxT5z5pB2SPG6oCxH/+cXOHaMug99mEhXF7mf/hQFX/pSskOKq5FeUFXAw0Qvnn6p3/j/BNr7XVCdoLX+qlJqNfB5Tl9Q/bnWesn51jHc5P7Y1uMcOOnGHQjhDoTp9YdxB8K4/WHMvWFuaoII8Dx1fH3jfzEh0MvRrGJ+cfn7SHf4ybHWY7iO0elq4XiaB5954LYoDYW5zO9nkT/AIn+AsnCY/v9t3SqdbksevbZ8PPYCAs4CgmmFGOnFkFGEKbMEW1YhGU4H6Q4L6XYLGQ4LdotpxEekQ0kObfW9rH/sEI2xWve5pS5W3DGN8pkTRrRurTXBo0fxbNqEd9MmPFu2YnR3D5jHnJdH2oJqnNXVOBcswDFnDib72OidorWm+VgPe95o4PC2FiLh6IVmV7aNGbOdVHAM3t6Md+tWws0DnwJmyc/HtfJK0q9ciWvF5ZgzMgZdRzy6gsrR//mFW1up/dCHCdXX47pqJeW//GW06W4cGWlyvwJYD+wm2hUS4P8Am4E/ARXAceAOrXVHbGfwAPBuol0hP3Zme/uZ4nET02Ce/+0eDm9rYerSQuYuc9Lzhc/AieOcVGbuMVwctmdisqVhsqeh7A5yJtuZd3UZnaE99KjDRFRgwPIywyam+6Eq4GdpoJv5QT+uC2y/sDbRSjbNOptWnU2bzqJdZdNrmUB72EVtZ4h6N/RoF1NnzmXOjKlkxHYE6XYL6Q4LGbH3UzuHdLuV0sJcjICPAXfuEE0OPneATU8e5Z31DWgd7dK49JbJzF5RPOwCXBG3G89bb+F+/XU8698k3NIyYLq1pIS05ctwLV2Kc8ECrGVlY/bGkP787hD7NzWx540Gult8ACiTYlJVHrOvLKEo04tv6za8W7fi2bBh4HYxm3EuqCZ95VVkXHct9smT+ybF48g9kd1Jx7qI28Pxf/on/Hv34pg3j4kP/w+mtPF31/K4vYnpQrpavDxy32YMrfnA1xeTnRbixKc/g//tt/EZBt9qPsm6ntOn2f2PmiJGhIOdB9nWvI2a5hpqmmvoCgx82pNCUWwrotKUz6RIGlMCJib5PGT423EFWsgMtZFpDP0JUV5tp1Vn0Uo2rTqLNp1Fq86mlX6fdRZtZBHAhtYGOujDCHgxgl50wMd8axbvdk3BGtJoBZEp6Tirc8jItA++s7BbcdnNWM5I+lprgseO4X7tddxvvIG3pgZCob7p5rw8XEuXkrZsKa7ly8f9DWNaaxoOdLLnjUaO7Wzt62GUmedg1ooSZl1eTFqmjcCBA7jfWI/njTfw7tgBkdM3U9mmTCFj1XVkrFrFX3fs4O5PfWpER93xvhFsvDCCQeo/81k8GzZgraig8pE/plTzXzxdsskdYP2fDvL2K/WUzczhvV+sRgeD/HzOXG6wRGumPdrVyfdaWghqfd4jHkMbHOs+xu623exp28Putt0c7DhIWA+sgGhRFiqzKpmeMz36ypzMdFsWBaEQytMKnhbCPc08+rv7SVc+CtNNFKVpCtMM0ixD/1306DRadRYdZNCpM2gJl9Ptvg5C0a6evZYedqS1ctxkp4MMApz/Ke5Oq5kci8HCjqNUN+5l9vE95HSfvp6hlYmeyTPpXbCE8GXLsE6fQbrT2u9sIrrDsFvG12nvYDzdAfZtaOKdNxtwd0TP7pRJMXFuLnOuKKFizgRMZhORnh48b23E/eqr9L722oCmK2tJCfXl5Xz/jTd4vq52WO3lcuR+NiMQoP4LX8Dz+huYJ0yg8pE/Yps4MdlhJcwlndz97hB/+I+NBLxhbvjkXKZeVsDatWv52z338NXsHOwmE0cCAf7/rk6+8MADF/WfKxAJsL9jf1+y39O2h+M9x9GcvU2z7FlMz5nO5KzJTMqaxGc+8Bn8TX5C7aG+lpUMGxRlmDhYsx7czeBujb03g7sl2qff3RIdNqI7lYi2sN1zGzXu9xPBhl31cnnGw8xyvoJSp+MImJz0mrLoNmXRRSbtOoNWI4PuHgv2E70UnuygvLUFW+T0zqrblsa2gplsLZpFTcEM3LYLn9bazKa+s4LBzhT6hu0W0h1n7xxOzeu0mlO+WccwNCf2dbD3zUZqd7X1Hc27su3MuryYWZcX95Wf1qEQ3q1b6XnxRXpfeolIa1vfcqwlJWSuvpHM1auxz5gx5J87FdrcU+mCruH3U/+5z+PZsAFzdjYVDz2IY9aspMQyWi7p5A6w540GXv/jAdKybHzkvmXYnBbWrl3LQ//+73wJxRS7Ha0UuR/7GPlf+OcRFez3hrwc6TrCwc6Dfa8DnQfoDfYOOr8RMgg2BwmcDBA8GSQjksH//uJ/KXGVUOwqxmoepB++YYC/i4Y99bz2ZDtdHdHf4cxJbVw+422c4SbwtoGnPfbeBka0SUVHwNtmw93kwN1oJ9gzcPmOnCDpJQFcxQHMhU5C9kx8lky85kzcKoMelU436XQaLtoNF63hNFrCTk4GnTQEHHQYaYTjUEnapIglfmtf4j975zDwOsTA4dh0mwWTKfE7CW9PkP0bm9j7ZiPdrdG2eRSUzchh+pJCJi8owO6MbhdtGPh27qL3hRfo+fvfB1yUtU2dQtbq1WTeeOOQjjiTmVxTYedyiuH1cuJzn8O7cRPmCROoeOghHDOmj2oMyXDJJ3dtaP7ynzU0H+th7lWlXPWhGX3TjECAtvvvp/3Bh8AwsBQXU/ClL5J5883D6r886Pq1ptnbzMHOgxzrPkZtTy2bD27mWNcxLNnnToQKRX5aPqXppZSkl1DiKqHIVUSOkUf36w6ad0a7NmYXpnH1h2dQOiNn0OWETp7E/coLeF5/Hc/W7Rhef980k92Ca1o26ZU2XCVBrOYe8HWCv3vQZQ2FYUvHsGcTsmUTsGYSsGTgM7nwqjTcykWvTqNbO+mOOOkwnLSHHLSGHbSE7Jz02+gOavyh+LUZpw+6MxjszMJ69plGbH6X3YJ1CBejtdY0HuzinTcbObqjta+njdlqonJeHtOXFDJxbi5mS3RZ2jDwbttGz9+epff554l0nb5G45g3L3pE/54bsRamXgmNVGkWCjW3UP+Zz+Dfuxdzfh4TH3oI+9Spo7b+ZLrkkztEuwX++bvbMCKa936hmvLZA7sD+nbtoumb9xHYvx8Ax+zZ5N9zD64rViSseWDt2rV8/dtfpyXYQsncEq674zpyJ+fS4G6g0d1Is7cZQ/dLchpmtC5led0tOMIuwirE9rIXODF5J3npueQ585jgmMAEaxZldR6KdjWSveMo1qMNA9ZrmzqF9JVXkb7yStIWLkTZBmmPj4QhEEv0F/vSI0zMFifakYlhyyBszSBkzSBoScdvSsd7aieBE7dhp8ew0x2x0xWx0Rm20RG00hay0Raw0ho00xuI307CYTWRbreetzfTgKYlbSJU56Z7fxfddaf7zdtdFqYsKKCyegLF07IwWaJ/X0YoiG/jJtzPPofn5VdPl19QCvviy3Dd+G7SrrsWc1b0/gSzyYxZRV/JaMJKhQu6/r17OfGZzxJubsZaUUH5f/8X9knj4+7ToZDkHrPt77Vsfuoormw7H/z3JTjSBzZJ6EiE7nVP0/rTn/adKttnzCD34x8j88YbUdbRLVUQMkI0e5ppdDdSd/wkLc8paIreTdqRd4LNU57iuOkwaIOyVphbp5lbp5l9XOPq14vTb4XdlYqdkxU7pij8eRm4rC4i3gjNJ5rxdfuwm+wsmLOAqhlVuKwuXFYXadY07GY7VpMVu9mO3WzHZrZhM9v6PtvNdmwmG1azFYuyYAJMIS8mfw9mfzcmXzfmQA+moBv8PdEdhr+73+ceCAwc1jqCJtrvVp96KTBQaCCkIIIirBRhBWHOeFeKMIqIgpDFQcjqJGi2EzLbCZhs+E02/CYrfqz4lBkfFnzaFHuBTyv8xql3jd8AvzYIEb1vIqLAUGCoWJQqAiqCUrHPREAZqNh4lEF6IJ1pnfOY3raAXF9J3+8mYPZxPHsvx3Lf5kT2PkLm6C/OFtJcdlizYq9mwRGNNdbpJmyCHVMUb85R1ExVBK3RpG5WZiwmSzTZm8xYlKUv+VtMFmxmGw6zA4fFgd1sx2Fx4DCf8dlix2F2kGZNI8OWQbo1nQxbxoDPLqsLiyl6tjmSI3etNZ6uAL3tfvye6PN1fe4QAU+IUMBAa43W0RvFbU4LdqcFe5qFzHwn2QVppGfb6X3hBRr/7d/QPh/ORZdRdv/9WHIGP3sdNVpHr4eFvBDy9Xv3DTIu9p47FWa8Z1irk+QeY0QMnvjRdk4e7aFiTi43fa4KNUh7rOHz0fG/f6Djf3/fd+HLnJ9H1uqbyLrlvUO+CzEewsEINc/Vsf35OoyIxplhZcX7plAxoRf/9u24N23Cs3kzumtgM4q7OIv6uQUcnJnO/goT7ZEeuvxddAe7B54NjKJTR5jmWMEurTUGBmgwiP2HHuRi9HgzwVPM5FSB3jMAACAASURBVI5qJnVUkes9negNFaE5vZb6rAPUZx2g1VWPVpo0v2bJQYMVew3mHjcwxTaRzwZbpik2zFHsrlREzKPzN+m0OMmwZYAXavfVEuwMEu4NE+mNYPKb+NQ/fYrbbriNCY4JZJJNoAO6mr2nXy3R91NVSIfDTIT0rmPkdB2idGYWs7/2YWwOczSxRkLRa0yR0BnD4ejwgGlBCAdiL3/0PXLGcN/7YOP8sWX0G77Y/19zb4f3Pzis7SDJvZ+edh9/+r9bCXjCLL5pEktuOvcpnBEM0vP007Q/+BDBI0f6xvc1a1yxAudllyXszssT+zp4/Y8H+i7QTcpqZ1rzSxi7a84quGUpLMS1bClpS5aStnQptrLBq18a2qA32Ev1kmqa2pswOUyYnWZMDhMmh4m80jy+/G9fxhP24A15CUaCBI0gwUiQQCRAIBKIjosN93+P6AiGNvpep4YjOjJoLOejUCilMGECBSaid/WalAmLskSPUk3Ro9JTwxaTBavJ2ne0ajFZsKCwEH1YsEWDRevYy8BsRLBoA6sRwRIJY9EGFiOMxYhEX5Fw7BWMjg8HsURCWCIBLOEQ1kgwtlwdXTYac+yzldPj+tYb+2zVGjNgArrDRRwNLOGYfxknQ9OJTomyKzfFtr0UWg9SZD1AgfUwKhCk97iT7jon/o7TzWkmewRXhZ+0iT5M+SEMFTtz0SZ8yoRfmfEqMz6TKfquTNGzFpPCh4mQSREwQdCk8JkUXpPCYwKPSeE1Ef2soi99xj7EZJjJ9OeR7c8n21dAlr+ALF8+2f5C0kKD36ELYDb34rK2k2l24zK5cZjcOMxurMqHIoJJRzC0iaDhIGA48UfS6fQV0hksIWjNGrAsm/Iw1bGBmc5XKbLuT25pKGUGa1q0PInV2e/zIONsLiieD1V3Dm9VktwHqnunnWce2AUaVn18NtOXFJ13fq01/l276F73ND3PPjvgopey23HMmhWtHDh7NrZJk7CWlmLJzxvyBVkdiRBuayfcfJLQyZN0H21m+14z9cFoXC5PIzMOPkp29+kdjKWkmLTqatKWLMW1bCnWiRMv6mxitNtLz0z6ACZl6kvip/6Z1MhLM4waIxI9Ugv5o6fYYf/pU/DwqfdA9CgxEogd4QUhEmTn9i38/emnUEYIuwVsZkWaI4sZVbcTMs3leHsxPf6BtX8UBhMcJ8m3N5BjayQj1IClvoHwoS7C/U7cLK4ImRV+siZ6cGSHiYewttEdLqQrUkRbpIgOo5iuSBHucDHBcAHRXdUg3zMF6XK00O1spcvRQpezhW5HK13OFoIWX998ueEIZeEw5eEwZaH+7yHyIgZGSNG2J4OOg67o3iXHhnlpOS3W2dQHZtMerOhbVrathQUFG5gxYXf0WcomK5itYLLE3q1gtpweb7GDxXH2u9kWGz5zmn2Q7/Sb3zzy3mJDJcl9EDtfOs6Gxw9jsihu+uz8sy6wnosOBvFu347nzTdxv7mh7wLsmZTVijkvL1qiNiMDk9N5utJkJILh8WB4PUR63YTb2iAcJmKycrz8Wuoqrscw2zFFglTW/Z1J3p04p07GOW8ezvlVOKqqRlx6NlV6OlyqhrL9e9p8NB3ppvloNyeP9dBe7+7rS9+fxWYi3aWw+TuxNB3F1tWILdiDJezDkZdN5oK5ZC6txjWjMtpkEIk93MSIYITC+L0h3N4QHncEjzeMuzeEt9fA544Q8GjCPo3hP/cOV6Pxm0O4TUE8pgAeUwCvOYDbHMSrIoTMIcJWD2GLh8ipd4sbw+rGsPRi2LpBDX5AobRmxTsW/vHVMDnuCIZS7Fy8gH3vvYu8rIlkOG1k2C3YvBECh3ro3t9FyB3doaXnOlh+6xSmLSoYOwcMF0mS+yC01rz550O8/Uo9ZquJ1Z+tonzWxRfUCnd2Eti3D9877xDYt4/giXpCDQ1EOjqGHgvQNnklh0pvxG+OnsaW5/lYcnUueQunY5kwskJfg0mlPsqXouGcOYWCEVqP99LR6KGjydP33v8xgoliMikych1k5TvJyneSme8kqyCNrLzosNka/XkCYWNAAb/eQAj3qeEzivudGu4NBOgOttEbbsarWwjQija1sfx4PR/Y3EZFW3R7HCqG377bzLGiaKLWhhUjUEAkUIzhL4m+fMXMDKax3G8hN1a++YTNYFuugozhd4VNt1vOKtGRCiS5n4M2NK/98QB732zEbDFx7UdnMW1RYVyWbXg8hDu7oo+E6+lB+0/3LUeZMLlcmFwumtsUW9d30VwbvckptyydK++cRun0xF/1T6W7Cy818TxzCnhDuDsDuLsCeDoDuDv9eHqCBD0h/M3t+Fs7CfT4iBgK0KBBmUyY0pxYXE4cuZnYM51YHRZsDjOOdCuuLDuuLDtpWTbSsmykZ9uHXXTuYkTcbnqeeYaOtX8keOgQAEZeHnW3XsfWhUWc8NZx0ldHa+AE3kjnWd/XGkyhPMyhCqaeXMTS1hk4tYUgmpfSQrxjjcAwD+KH3BXWYT3HjXbxL9Ehyf08tKFZ/9hBdr8e7Qu+aHUli1dPSvhdjU1Hutm87igNB6J/oM4MK0vfO5lZK0pG5Y5KkVyjfeakDQP/nj30vvgSvS+/TPDo0QHTbZMn47r8ctIuWxit6ll0/utQcY0tGMS7bRvdzz5Lz9+eRfuibfGWoiLyPnU3WbffjmmQezEqZ1bSarTimOjAWeHEMdGBo9SBspz+/+MIpXPlsfczpX0BAMZUL/nXFOEyF+MNRs4+k+j3+fTZRbSk+CAtYsNyZomOq2bk87V3zxzWsiS5X4DWmp0vneCtvx4GDSXTsnnXP84kuyC+JUINQ1P7dhs7XjjOyaPRK2A2p4UFqyqouqYMm2P0LsSI5EvmmVOwrg73hg14NryFd9Oms3tflRSTVr0AR9U87NOmYZ86DUtBflzarnUkQuDIEXy7duHduBH3G+sx3Kdv8kpbsoTsO+8k4/pVgyb1UwZr2lIWhaPMwR9e+AP72vext30ve9v3MqV5IVccuwOrYeNk+jE2zHuUGaVTqc6vZnHRYublzRu81MepmLXGF4rEmpoGNitFdwShQXcOA4ZjO4pQZGDMt1aX8NMPLhjWtpTkPkQn9nfw4oN78fUEMVtMVF9XTvWqChyukd281Nvh58Cmk+zb2ERPrFujPc3CvKvLmH9t+YiXL8RI6FAI365deDZvxrdzF76dOzF6z66FZMrMxD51KraJE7EUFWItLMKSn4cpPQNzRvqAeunaMDB6o02Ska4uQg2NBI8fJ1hXR2D//rN2JvZp00i/5hqybrllyM83HWrTViASYF/7Prbv3UvPuiysXiedjmb+NvtXuO2xM2eLk4UFC1lSvISlRUuZOWEmZlNiKpwGwpEBOweX3cKkvOE96lKS+0Xw9QZ56y+H2b/pJAA2h5kZy4qZubyI/IqMIR25aK3paPJwYm8HtbvbaDjYdbryY66D6uvKmbm8WI7URUrShkHg8GF823fg37+PwOHDBA4dPutpWyNhLSnBWT0fZ/UC0q++CltFxYW/dIbhNG15ugM8/fOdtDd4sGRq2q/fzhb3Bo7062YMkGHNYFHRIpYWL2VFyQoqsyovOr7RIMl9GJqOdLPl6aPU7z990caVZaN4ajZ55elkTHBgT7NiMivCwQie7mDfnXdtx3vxdJ/uwWC2mJhcnceM5cWUz8wZlQtTQsST1ppwayvBw4cJNjQQbjpJqPkkkbZ2Ih43Rq8bw3e6GqZCYcrIwJyVhTkrE0tRcd8D0G1Tpoy4K+8pw2naCnhDPH3/LpqP9ZCZ5+C2f12Iz97L1pNb2XJyC5ubNlPvrh/wnYqMCq4su5IrS69kUdEi7ObUeGSkJPcRaKvvZe/6Ro7ubB2QsC8kLdNG+awJlM/KobIqD3uaNL0IkSoCvjDrfrqDlrpecktd3Pbly/pKMgM0uBvY0rSFjU0beavxLboDp89anBYnS4uWsrJ8Je8qfxd5zrxk/AiAJPe40FrT0eihubaHjiYP7o4AIX+YSERjtZlwuKxkF6WRXZBGTrGLnKK0cXvjhBDjgd8T4i8/qKGr2UvZzBxu/uf5g55VR4wIu9t280b9G7zZ8Cb7Ovb1TVMoFhYu5LqK67i24lqK04tH80eQ5C6EEIPpafPx+A9q8PUEmX9tOVfcMe2C32nxtrC+fj2vnHiFjY0bCRmnny08J3cO1028jhsqb6A8ozyRoQOS3IUQ4pyaDnfx5I93YBiaVZ+YzfTFQ+/j7w66Wd+wnpfqXmJ9w3p84dP1cubnz+fGSTdyQ+UN5DoT84BuSe5CCHEeu1+r541HD2K1m/nAvy8mK//i73Hxh/1sbNzIc7XP8eqJV/sSvVmZWVayjNWTVnNNxTW4rMPr9jgYSe5CCHEeWmue/807HNneQkFlJu/7ykLMI+jV5g15ee3Eazx77Fk2NGwgrKPFzJwWJzdU3sCtU29lYcHCEV+Xk+QuhBAXEPCGePQ7W3B3BFhy8yQWr47P4/o6/Z28WPcizxx9hh0tO/rGT8ycyK1Tb+XmyTdT6BpeTStJ7kIIMQT1Bzp56ic7MJkVd/6fxeSWpsd1+bXdtTx15CnWHV5Hi68FgKq8KtauXjus5Z0vucvdNEIIEVM2I4c5K0sxIppXfr9v0Pr5I1GZVckXF36R59//PL+89pesmriK26ffHtd1nCL3vwshRD+Xv28KdbvbaKnrZe/6BuZeVRb3dVhMlugdr2VXxn3Zp8iRuxBC9GNzWFjx/mh/901PHcXXm/iHoSSCJHchhDjDlIX5lM+eQMAbZtO6oxf+QgqS5C6EEGdQSnHlndNQJsW+NxvpaPRc+EspRpK7EEIMIqfIxZwrStAa3nricLLDuWiS3IUQ4hwW3zQJq8NM3e52Gg+d/czWVHbB3jJKqQeBm4AWrfXc2LgJwGNAJVAL3Km17lTR261+BtwIeIGPaq23JyZ0MR6EQiHq6+vx93+AuBgyh8NBWVkZVquUlE6EtEwb868tZ9vfatny9DFu/ZfEP7g+XobSFfJ/gAeA3/cbdy/wstb6e0qpe2PDXwPeA0yLvZYCv4q9CzGo+vp6MjIyqKyslBLJF0lrTXt7O/X19UyaFJ+7KcXZqq8tZ/er9TQc7KJ+fwdlMyckO6QhuWCzjNb6DaDjjNG3AA/HPj8M3Npv/O911CYgWyk1ugWOxZji9/vJzc2VxD4MSilyc3PlrCfB7GlWqq+LPgZwyzPHkhzN0A23zb1Qa90EEHs/9cysUuBEv/nqY+POopS6Wym1TSm1rbW1dZhhiPFAEvvwybYbHVXvKsOeZqHpcDdNR+L3LNlEivcF1cH+0ga9f1dr/Wut9SKt9aL8/Pw4hyGEEPFjc1qYe1X0OHX783VJjmZohpvcm081t8TeW2Lj64H+jx8pAxqHH54Qo+u+++7jhz/8YbLDAODPf/4zc+bMwWQyIYX1kq/qXeWYrSZq326jvcGd7HAuaLjJfR1wV+zzXcBT/cb/k4paBnSfar4R4lISiURGvIy5c+fy17/+lZUrV8YhIjFSaZk2Zl0evYT49isnLjB38g2lK+QjwNVAnlKqHvgm8D3gT0qpTwDHgTtisz9LtBvkYaJdIT+WgJjFOFV5798Sstza760+7/Q1a9bw+9//nvLycvLz87nssssAOHLkCJ/73OdobW0lLS2N3/zmN8ycOZMjR47wkY98hEgkwnve8x5+/OMf43a7ee211/jWt75FcXExO3fuZO/evfzhD3/g5z//OcFgkKVLl/LLX/4Ss9nMCy+8wDe/+U0CgQBTpkzhoYceIj19YHnZWbNmJWR7iOGbf005e15v4MCWZpbdNgVnui3ZIZ3TUHrLfEhrXay1tmqty7TWv9Nat2utr9VaT4u9d8Tm1Vrrz2mtp2it52mt5VxSpLSamhoeffRRduzYwV//+le2bt3aN+3uu+/m/vvvp6amhh/+8Id89rOfBeCLX/wiX/ziF9m6dSslJSUDlrdlyxbWrFnD3r172bdvH4899hgbNmxg586dmM1m1q5dS1tbG9/5znd46aWX2L59O4sWLeLHP/7xqP7cYniyC9OYODeXSMjgnfWp3eIsJX9FyrjQEXYirF+/nttuu420tOgzM9/73vcC4Ha7eeutt7jjjjv65g0EAgBs3LiRJ598EoAPf/jDfPnLX+6bZ8mSJX19zl9++WVqampYvHgxAD6fj4KCAjZt2sTevXtZsWIFAMFgkOXLlyf4JxXxMv+acur2tLPntXoWXl+BaQSP40skSe7ikjdYd0LDMMjOzmbnzp0XtSyX6/TDj7XW3HXXXXz3u98dMM/TTz/NqlWreOSRR4YXsEiqslk5ZBem0dXspXZ3O5OrU7O3X2rucoQYJStXruSJJ57A5/PR29vL008/DUBmZiaTJk3iz3/+MxBN1Lt27QJg2bJl/OUvfwHg0UcfPeeyr732Wh5//HFaWqKdyTo6Oqirq2PZsmVs2LCBw4ejxai8Xi8HDx5M2M8o4kspxZwro81x76xvSHI05ybJXVzSFi5cyAc+8AGqq6u5/fbbufLK00/GWbt2Lb/73e+YP38+c+bM4amnop3CfvrTn/LjH/+YJUuW0NTURFZW1qDLnj17Nt/5zne4/vrrqaqqYtWqVTQ1NZGfn8///M//8KEPfYiqqiqWLVvG/v37z/r+E088QVlZGRs3bmT16tXccMMNidkI4qLNXF6M2WLi+N4Oetp8yQ5nUPKAbJFU+/btG3O9QrxeL06nE6UUjz76KI888khf4k+GsbgNx4MXH3qHg5ubuezdE1l265SkxCAPyBYijmpqaqiurqaqqopf/vKX/OhHP0p2SCIJ5lwRbZo5sPlk3B+kHQ9yQVWIi3TllVf2tb+LS1fxlGwy8xz0tPlp2N9J+ezUqhYpR+5CCDEMyqSYuTx6x+q+jal3I74kdyGEGKYZS4sAOLqzlYAvnORoBpLkLoQQw5SZ56R0ejaRkEHtrtQqXS7JXQghRmDqokIADtW0XGDO0SXJXYh+Uqnk71e+8hVmzpxJVVUVt912G11dXckOSQxicnU+SsGJvR34PaFkh9NHkrsQCRCPkr+rVq1iz549vP3220yfPv2sMgYiNaRl2iidkYMR0Rzb1ZbscPpIV0iROu4b/E7PkS/3/I9FS9WSv9dff33f52XLlvH444/Hf9uIuJh6WQH1+zs5XNPcV/M92eTIXVzSxkrJ3wcffJD3vOc98d8AIi4mL8hHmRT1+zrxu1OjaUaO3EXquMARdiKMhZK/a9aswWKx8JGPfCReP7aIM2e6jbKZOZzY28HRXa3MXlFy4S8lmCR3cclL5ZK/Dz/8MM888wwvv/zyoHGK1DH1sgJO7O3g8LbmlEju0iwjLmmpXPL3ueee4/vf/z7r1q3rO7MQqWvy/HxMJkX9gS58vcFkhyPJXVzaUrnk7+c//3l6e3tZtWoV1dXVfPrTn07MRhBx4Ui3UjYrB22kRq8ZKfkrkmoslquVkr/iXPa80cDrfzzApPl53PiZqoSv73wlf6XNXYiLVFNTw+c//3m01mRnZ/Pggw8mOySRIibOzQXgxP5OIiEDszV5jSOS3IW4SFLyV5xLxgQHuaXptDe4aTjUScXs3KTFIm3uQggRRxPnRRN63e72pMYhyV0IIeKocl4eALW720jmNU1J7kIIEUeFkzJxuKz0tPnpavYmLQ5J7kIIEUcmk6JiTvSRe7VJbJqR5C5EP6lU8vcb3/gGVVVVVFdXc/3119PY2JjskMQQ9bW770lef3dJ7kIkQDxK/n7lK1/h7bffZufOndx00018+9vfjkNkYjRUzM5FKWg61J20x+9JV0iRMuY9PC8hy9191+7zTk/Vkr+ZmZl9nz0ej9SWGUMcLitFU7JoOtzNib0dTL2sYNRjkCN3cUlL9ZK/X//61ykvL2ft2rVy5D7GnOo1U/dOctrd5chdpIwLHWEnQqqX/F2zZg1r1qzhu9/9Lg888ADf+ta34vnjiwQqnzWBjU8coX5/B1rrUT/zkuQuLnmpXPL3lA9/+MOsXr1akvsYkleWjj3NgrsjQE+bn6x856iuPyHNMkqpdyulDiilDiul7k3EOoSIh1Qu+Xvo0KG+z+vWrWPmzJlx+InFaFEmRemMHAAaDnSO+vrjntyVUmbgF8B7gNnAh5RSs+O9HiHiIZVL/t57773MnTuXqqoqXnjhBX72s58lZiOIhCmLJff6JCT3uJf8VUotB+7TWt8QG/43AK31OR/dLiV/L11jsVytlPwVQ9XR5OGRb23GmWnjY99fEfd299Eu+VsKnOg3XA8sTcB6hEgKKfkrhiqnKI20TBveniCdTV4mlLgu/KU4SURyH2zXdNbpgVLqbuBugIqKigSEIURiSMlfMVRKRdvdD21tpv5A56gm90RcUK0HyvsNlwFn3Tettf611nqR1npRfn5+AsIQQojkK5uZnIuqiUjuW4FpSqlJSikb8EFgXQLWI4QQKe/URdWGg50YxuiVAI57ctdah4HPA88D+4A/aa3fifd6hBBiLMjMc5KZ5yDgDdNe7x619SbkJiat9bPAs4lYthBCjDWlM3LoaWuifn8n+RUZo7JOqS0jRD+pVPL3lB/+8IcopWhrS175WDEyp/u7d4zaOiW5C5EA8Sj5C3DixAlefPFF6VE2xpVOjyb3k0e6R63dXWrLiJSxb2ZibsSZtX/feaenaslfgHvuuYcf/OAH3HLLLQnZNmJ0uLLtZOQ66G3309nkIbf07N91vMmRu7ikpXLJ33Xr1lFaWsr8+fMTuxHEqCiaHC1T0XSke1TWJ0fuImVc6Ag7EVK15K/X62XNmjW88MILifixRRIUT8ni0NZmTh7pZu7K0oSvT5K7uOSlYsnfI0eOcOzYsb6j9vr6ehYuXMiWLVsoKiq6qJhEaiiacurIvWtU1ifNMuKSlqolf+fNm0dLSwu1tbXU1tZSVlbG9u3bJbGPYbklLqx2Mz1tfjzdgYSvT5K7uKSlcslfMb6YzCYKJ0Wfi9t8tCfh64t7yd/hkJK/l66xWK5WSv6K4dr89FG2/a2W6uvKWfH+aSNe3miX/BViXJOSv2K4imM9Zk4eTXyPGUnuQlwkKfkrhqtwchYoaDneSzgUwWI1J2xd0uYuhBCjxO60kFviwghrWut6E7ouSe5CCDGK+m5mSnDTjCR3IYQYRaf6u59M8J2qktyFEGIUFU85fVE1kb0VJbkL0U8qlfy97777KC0tpbq6murqap59Vh6RMB5k5jlxZljx9YboafMnbD3SW0aIBIhEIpjNI+8Jcc899wyoXSPGPqUUBZWZ1O1up6Wuh6x8Z0LWI8ldpIxffPqVhCz3c/91zXmnp3LJXzE+FUyMJffaHqYtKkzIOqRZRlzSUrnkL8ADDzxAVVUVH//4x+ns7EzchhCjqmBi9FF7zbWJK0MgR+4iZVzoCDsRUrXkL8BnPvMZvvGNb6CU4hvf+Ab/+q//KnfDjhOFldEaM63HezEiBiZz/I+zJbmLS14qlvwFKCw8fbr+yU9+kptuuumiYhGpy5lhO/1kppPehDyZSZplxCUtVUv+AjQ1NfV9fuKJJ5g7d24cfmKRKk4dvSeqaUaSu7ikpXLJ369+9avMmzePqqoqXn31VX7yk58kZiOIpCiYGE3uLQkqQyAlf0VSjcVytVLyV8RDw8FOnnlgF9MXF/Kufxze709K/goRR1LyV8RD8dRsPvmTlQm5mAqS3IW4aFLyV8SDyaSAsy/mx235CVuyEEOUCk2DY5VsO3EuktxFUjkcDtrb2yVJDYPWmvb2dhwOR7JDESlImmVEUpWVlVFfX09ra2uyQxmTHA4HZWVlyQ5DpCBJ7iKprFZr3x2dQoj4kWYZIYQYhyS5CyHEOCTJXQghxqGUuENVKdUK1CU7jnPIA9qSHcQFpHqMEt/IpHp8kPoxjtf4Jmqt8webkBLJPZUppbad6/beVJHqMUp8I5Pq8UHqx3gpxifNMkIIMQ5JchdCiHFIkvuF/TrZAQxBqsco8Y1MqscHqR/jJReftLkLIcQ4JEfuQggxDklyF0KIcUiSO6CUKldKvaqU2qeUekcp9cVB5rlaKdWtlNoZe/1HEuKsVUrtjq3/rEdXqaifK6UOK6XeVkotHMXYZvTbNjuVUj1KqS+dMc+obkOl1INKqRal1J5+4yYopV5USh2Kveec47t3xeY5pJS6axTj+0+l1P7Y7+8JpVT2Ob573r+FBMd4n1Kqod/v8cZzfPfdSqkDsb/He0cxvsf6xVarlBr0KeejsQ3PlVtG5e9Qa33Jv4BiYGHscwZwEJh9xjxXA88kOc5aIO88028E/k70CQDLgM1JitMMnCR6g0XStiGwElgI7Ok37gfAvbHP9wLfH+R7E4Cjsfec2OecUYrvesAS+/z9weIbyt9CgmO8D/jyEP4GjgCTARuw68z/U4mK74zpPwL+I1nb8Fy5ZTT+DuXIHdBaN2mtt8c+9wL7gNLkRjUstwC/11GbgGylVHES4rgWOKK1Tupdx1rrN4COM0bfAjwc+/wwcOsgX70BeFFr3aG17gReBN49GvFprV/QWodjg5uApNbzPcc2HIolwGGt9VGtdRB4lOi2j6vzxaeUUsCdwCPxXu9QnSe3JPzvUJL7GZRSlcACYPMgk5crpXYppf6ulJozqoFFaeAFpVSNUuruQaaXAif6DdeTnJ3UBzn3f6hkb8NCrXUTRP/jAQWDzJMq2/HjRM/EBnOhv4VE+3ys6ejBczQppMI2vBJo1lofOsf0Ud2GZ+SWhP8dSnLvRymVDvwF+JLWuueMydv/X3tnz1pFFIThZ8BYKCJ+FCpaqFgrEkRUrCSYIAGtFEFRm4D2FvkPNiIKKgghhQh+pAhY2AuCaFQUjF1ISMBCERs/xuLMDcuye3ObPTcu7wPL7p4zyw4vw9xz5hz2ksoM+4GbwNPc/gFHbFuTGQAAAgVJREFU3f0gMAxcNbPjpf6qP2TMutfVzNYCo8Cjiu7VoGEvrAYdx4HfwGSNyUqx0CS3gb3AAWCBVPoo03cNgXN0H7Vn03CF3FL7WEVbzxoquQdmNkASf9LdH5f73f27u/+I62lgwMy25vTR3efjvAQ8IU19i8wBuwr3O4H5PN4tMwy8dvfFcsdq0BBY7JSq4rxUYdNXHWPh7BRw3qP4WqaHWGgMd1909z/u/he4W/Pufmu4BjgDPKyzyaVhTW5pPA6V3Fmuzd0HPrr7jRqbbWGHmR0iafc1o4/rzWxD55q08Pa+ZDYFXIhdM4eBb52pX0ZqR0v91jCYAjq7Di4CzypsngNDZrYpSg5D0dY4ZnYSuA6MuvvPGpteYqFJH4vrOKdr3v0K2Gdmu2M2d5akfS5OAJ/cfa6qM5eGXXJL83HY5Erx/3IAx0jTnRngTRwjwBgwFjbXgA+kVf+XwJHMPu6Jd78NP8ajveijAbdIuxTeAYOZfVxHStYbC21905D0I7MA/CKNgq4AW4AXwOc4bw7bQeBe4dnLwGwclzL6N0uqs3bi8E7Y7gCmu8VCRh8nIr5mSElqe9nHuB8h7Q750pSPVf5F+4NO3BVss2vYJbc0Hof6/IAQQrQQlWWEEKKFKLkLIUQLUXIXQogWouQuhBAtRMldCCFaiJK7EEK0ECV3IYRoIf8APcjesQuVbOgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.pipeline import make_pipeline\n", "\n", "\n", "# The, we load the linnedrud dataset\n", "# This is a sample data set provided by sklearn\n", "# You can use our own, as long as X is a matrix and y a vector with the same number of rows as X\n", "linnerud_X, linnerud_y = datasets.load_linnerud(return_X_y=True)\n", "\n", "# Use only one feature (this is not necessary, just for plotting)\n", "linnerud_X = linnerud_X[:, np.newaxis, 0]\n", "# Use only one feature (this is not necessary, just for plotting)\n", "linnerud_y = linnerud_y[:, np.newaxis, 0]\n", "\n", "\n", "# Split the data into training/testing sets (we will discuss this latter on)\n", "linnerud_X_train = linnerud_X[-10:]\n", "linnerud_X_test = linnerud_X[:-10]\n", "\n", "# Split the targets into training/testing sets (we will discuss this latter on)\n", "linnerud_y_train = linnerud_y[-10:]\n", "linnerud_y_test = linnerud_y[:-10]\n", "\n", "\n", "regr = linear_model.LinearRegression()\n", "\n", "x_plot = np.array([np.arange(1,20,0.1)]).transpose()\n", "\n", "plt.scatter(linnerud_X_test, linnerud_y_test, color='black')\n", "\n", "\n", "for count, degree in enumerate([1, 2, 3, 4, 5]):\n", " model = make_pipeline(PolynomialFeatures(degree), regr)\n", " model.fit(linnerud_X_train, linnerud_y_train)\n", " y_plot = model.predict(x_plot)\n", " plt.plot(x_plot, y_plot, linewidth=2,\n", " label=\"degree %d\" % degree)\n", " \n", "\n", "plt.legend(loc='lower center')\n", "\n", "plt.show()\n" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error: 0.00110\n", "Coefficient of determination: 0.33\n" ] } ], "source": [ "regr = linear_model.LinearRegression()\n", "\n", "model = make_pipeline(PolynomialFeatures(2), regr)\n", "model.fit(molecules_X_train, molecules_y_train)\n", "\n", "molecules_y_pred = model.predict(molecules_X_test)\n", "\n", "# The mean squared error\n", "print('Mean squared error: %.5f'\n", " % mean_squared_error(molecules_y_test, molecules_y_pred))\n", "\n", "# The coefficient of determination: 1 is perfect prediction\n", "print('Coefficient of determination: %.2f'\n", " % r2_score(molecules_y_test, molecules_y_pred))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df4xd5X3n8fd3rq/x2EmYQVjqMuDYRI4prou9TCmq1bSggOmyGCvAQn5I6TYVSjcoG0itNQoKhtCNFaul+QMpsSK0u2oSG5JoNAm0TrqYrkoE8bhj6h2KG9v88jjaTmOGKniwZ8bf/ePeMz5z55xzn/tr7q/PS7I899x7zjx37sz3nPM83+f7mLsjIiKdq6fZDRARkcZSoBcR6XAK9CIiHU6BXkSkwynQi4h0uCXNbkCpSy+91FevXt3sZoiItJVDhw79q7uvTHqu5QL96tWrGRkZaXYzRETaipm9kfZcUNeNmd1iZkfN7JiZ7ch43Z1m5mY2WHy82symzOxw8d83Km++iIjUouwVvZnlgCeAm4CTwEEzG3b3V0pe937g88BLJYc47u4b69ReERGpUMgV/XXAMXc/4e7ngL3A7Qmv+wrwNeC9OrZPRERqFBLoB4C3Yo9PFrfNMbNNwBXu/qOE/deY2aiZ/Z2Z/W7SNzCze81sxMxGJiYmQtsuIiIBQgK9JWybK5BjZj3A48AXE173C2CVu28CHgC+Y2YfWHAw9z3uPujugytXJg4ai4hIlUIC/Ungitjjy4FTscfvB34DeN7MXgeuB4bNbNDdz7r7LwHc/RBwHPhwPRouIiJhQgL9QWCtma0xs6XAPcBw9KS7v+Pul7r7andfDbwIbHX3ETNbWRzMxcyuBNYCJ+r+LkREJFXZrBt3nzGz+4D9QA540t3HzOxRYMTdhzN2/wjwqJnNALPAZ939dD0aLiIiYazV6tEPDg66JkyJiFTGzA65+2DSc6p1IyLS4RToRUQ6nAK9iEiHU6AXEelwCvQiIh2u5coUi0hrGRodZ/f+o5yanOKyvl62b1nHtk0D5XeUlqFALyKphkbHefAHR5iangVgfHKKB39wBEDBvo2o60ZEUu3ef3QuyEempmfZvf9ok1ok1VCgF5FUpyanKtourUmBXkRSXdbXW9F2aU0K9CKSavuWdfTmc/O29eZzbN+yrkktkmpoMFZEUkUDrsq6aW8K9CKSadumAQX2NqeuGxGRDqdALyLS4RToRUQ6nAK9iEiHU6AXEelwCvQiIh1OgV5EpMMp0IuIdLigQG9mt5jZUTM7ZmY7Ml53p5m5mQ3Gtj1Y3O+omW2pR6NFpD0MjY6zeddzrNnxDJt3PcfQ6Hizm9SVys6MNbMc8ARwE3ASOGhmw+7+Ssnr3g98Hngptu1q4B5gPXAZ8Ldm9mF3n1/3VEQ6jmrZt46QK/rrgGPufsLdzwF7gdsTXvcV4GvAe7FttwN73f2su78GHCseT0Q6nGrZt46QQD8AvBV7fLK4bY6ZbQKucPcfVbqviHQm1bJvHSGB3hK2+dyTZj3A48AXK903dox7zWzEzEYmJiYCmiQirU617FtHSKA/CVwRe3w5cCr2+P3AbwDPm9nrwPXAcHFAtty+ALj7HncfdPfBlStXVvYORKQlqZZ96wgpU3wQWGtma4BxCoOrn4iedPd3gEujx2b2PPCn7j5iZlPAd8zsLygMxq4Ffla/5otIq1It+9ZRNtC7+4yZ3QfsB3LAk+4+ZmaPAiPuPpyx75iZPQW8AswAn1PGjUj3UC371mDuC7rMm2pwcNBHRkaa3QwRkbZiZofcfTDpOc2MFRHpcAr0IiIdToFeRKTDKdCLiHQ4BXoRkQ6nQC8i0uEU6EVEOpwCvYhIh1OgFxHpcAr0IiIdToFeRKTDKdCLiHQ4BXoRkQ6nQC8i0uEU6EVEOpwCvYhIh1OgFxHpcAr0IiIdToFeRKTDKdCLiHQ4BXoRkQ6nQC8i0uGCAr2Z3WJmR83smJntSHj+s2Z2xMwOm9nfm9nVxe2rzWyquP2wmX2j3m9ARESyLSn3AjPLAU8ANwEngYNmNuzur8Re9h13/0bx9VuBvwBuKT533N031rfZ0s2GRsfZvf8opyanuKyvl+1b1rFt00CzmyXSskKu6K8Djrn7CXc/B+wFbo+/wN3/LfZwBeD1a6LIBUOj4zz4gyOMT07hwPjkFA/+4AhDo+PNbppIywoJ9APAW7HHJ4vb5jGzz5nZceBrwOdjT60xs1Ez+zsz+92kb2Bm95rZiJmNTExMVNB86Ta79x9lanp23rap6Vl27z/apBZ1nqHRcTbveo41O55h867ndBLtACGB3hK2Lbhid/cn3P1DwH8DHipu/gWwyt03AQ8A3zGzDyTsu8fdB919cOXKleGtl65zanKqou1SGd0xdaaQQH8SuCL2+HLgVMbr9wLbANz9rLv/svj1IeA48OHqmioCl/X1VrRdKqM7ps4UEugPAmvNbI2ZLQXuAYbjLzCztbGHtwI/L25fWRzMxcyuBNYCJ+rRcOlO27esozefm7etN59j+5Z1TWpRZ9EdU2cqm3Xj7jNmdh+wH8gBT7r7mJk9Coy4+zBwn5l9FJgG3gY+Xdz9I8CjZjYDzAKfdffTjXgj0h2i7Bpl3TTGZX29jCcEdd0xtTdzb60EmcHBQR8ZGWl2M0S6UtRHH+++6c3n+OrHNuhk2uLM7JC7DyY9V/aKXkS6h+6YOpMCvYjMs23TgAJ7h1GgFxFAM447mQK9iCzom4/y5wEF+w6g6pUiovz5DqdALyLKn+9w6rqRrqO+6IWUP9/ZdEUvXUW1XJJpxnFn0xW9dJW0vuidw2NdfZWv/PnOpkAvXSWtz3lyaprJqWmgezNOqs2fr7QrTF1ni09dN9JVQvuclXESptKuMHWdNYcCvXSVpL7oNMo4Ka/StMx6pHFqYZTKqetGukpSX/SZczO8fWZ6wWuVcVJepWmZtaZxamJXdRTopeuU9kWnVWxUxkl5laZl1prGmXVHoECfToFeup4yTqofIN2+Zd2Ck6RRuNLe+MiPMYPJM9Nzx0x6fSUnVU3sqo7q0YtkqGeGSKtmm2TVoIfyJ8DofY1PTmEkLChdxTHTbN71XOIdwUBfLy/suDHoGJ0qqx69Ar1IinouwtHKC3qkBc/+5Xnemz4f3Oa048TVGpBb+efYbFmBXlk3IinqWegr9FjNyChJ6/Z4+8x0Re8/pPuk1i6WbZsG+OrHNjDQ14tROHEoyJenPnqRFPXsDw45VrMySvqW5xOzjtKkvZe0gdbS19RKC6NUTlf0IimyMkcacaxmlQqutPe2xyzxTqPcHAVlMjWPAr1IinoW+go5VrMySt6ZCr+aB5h1T5zNWtqt0tebp395Xl0sLSCo68bMbgG+DuSAb7n7rpLnPwt8DpgFfgXc6+6vFJ97EPhM8bnPu/v++jVfpHHqmXYZcqxmlQoO6XIplZa7rm6V1lQ268bMcsA/AzcBJ4GDwMejQF58zQfc/d+KX28F/ou732JmVwPfBa4DLgP+Fviwu8+SQlk30q2GRsfZ/r2XmZ698DeZzxm777ymocEzKZMl1Ou7bm1Ai6QatWbdXAccc/cT7n4O2AvcHn9BFOSLVnAhlfZ2YK+7n3X314BjxeOJdJ2gjJrS665FyH6OulxyZhXvqzoz7SEk0A8Ab8Uenyxum8fMPmdmx4GvAZ+vcN97zWzEzEYmJiZC2y4dppOLVZWr2jg0Os4Xn3qZ6fPzI/v0eW/4YGw04WnWndJQX64AnCp8toeQPvqk0/yC6wx3fwJ4wsw+ATwEfLqCffcAe6DQdRPQJukwrVisqtaZrPH9e8yYLekmjWfUPPiDIwuejzRyMLb05+4wN7t1oPieo1mvi902qZ+QK/qTwBWxx5cDpzJevxfYVuW+0sIadcUdXc02I7Uwq0211E0v3T8riCelVcY1cjA26XuXtnT7lnWJV2ygCp/tIiTQHwTWmtkaM1sK3AMMx19gZmtjD28Ffl78ehi4x8wuMrM1wFrgZ7U3WxZboxaMiI7bjKvZLLXmtJcL3pHL+noz32O5dM5aT75Z3zt+V/XJ61cldusoL749lA307j4D3AfsB/4JeMrdx8zs0WKGDcB9ZjZmZoeBByh02+DuY8BTwCvA3wCfy8q4kdbVqMk8zbyazVJrTnvI66JAmfYec2aZuef1OPmW+/lGn/Fj2zbw+N0bVXqgTQXl0bv7s8CzJdu+HPv6v2bs+2fAn1XbQGkNjZrMU8vVbCNd3JufW0M2LvTEk1ZWoMcKM1FL+/yTCnXdce0Au/cf5f59hxPHCOpRm/2Gq1byVy++mfma6DNSjnz7Uq0bCdKoyTxpxy13NdtIQ6PjvHtuZsH2fI8Fn3jSpqd8YFmeww/fPG9b0mSqG65ayfcPjWcOTtfj5Pujl39R9jWX9fW2bIllCaMSCBKknuUAyh0X4AO9zbsG2b3/6LxJS5H3LVsSHNzSygqkbd+2aYAXdtzIa7tu5YUdN3Lg1YmyXWX1qMWTdNcS15vPccNVKxd0EW3/3stsfOTHHZkK24kU6CVIo8rDRsft683P2/72mem6DPZWI+2KeLKCCo+1BuGQdMZaT77lfrbRZ5x00pmedSanpus6MC+No64bCdaoPtptmwp90aVXl/VeCzS0+6Ee3VQhS+altWdodDx1paaocmT8s6i2SyVrIL1/eX5ugZD79x0ueyyt29raFOilJTS6cmMlE7JqXdc0fsy0IDw0Os72py/MhB2fnGL70y/P7ZM2azCqHBl9j7STb8hJLauQ2cO3rZ87RugMxlOTU+rLb1EK9NJ0Q6PjiTNHIWwgMCS4VJKhUq+qlVl3QDuHxxLLHTz4g39kavp85nHLXT2HntRyKT/zqORNpYXOLu7Nt9zsZilQoJemypowFR8ITAseoUGt0juGRqcSpg2Clgvykaw7ndCTWtokNffs+Q39y/P86r2ZeSeq3nwOM2pO95TG0GCsJKp2xmWl+2UFlKnpWb770luJwWPn8Fjq/kkTueq5WlQryGp36EltIOUYAxmzdQ0Y/fLN7L7rmnkD6MvyPanLEaoeTvPpil4WCL1KLu0yCcn9LlUuCKRddU5OTTM0Oh4c1OrR717P/uf+CtdpjYvudDbvei6xLaGDyVk/k7RCZvFjnJ25cPfx9pnp1AHkdj2ZdhIFelkg5NY/6WTw7RffXPCHXu7WvZrVjSI7h8eCg1qt/e5J7/f+fYcZeeM0gx+8ZC4wRv3eUeXHtO/58G3rFywyEqKvN89/vObfZZ5QQ09q5X4mWcdIK4ZWGuxVD6c1lF1harFphanmW7PjmcQrMwNeK64otHnXc8EB2oDH796YmkpY7epG8eOXBpdKc/yjq/WkYL1t0wAbH/lxar96PmeJATvfY2DMey7etoeGjvDdl95i1p2cGddf2c8/vPlO6s+iN9/DJSsuSv25D/T1zqVE1uPuI+sYq3c8k7pf1PWjrJvFlbXClK7oZYGQq+RK+l2dQi52FO6SunSyap6HHL+0hnpIcIkH9/jJIuouito58sbpzBmkaVflpVk1MH/8YN/P3pr7XrPu/Oy1t7n7uitSa89MTZ/P/BnFP5N6DCZnHSMtYydnNneykdahwVhZIGTGZVq/a1rd8qQunS8+9TJrdjzD7v1H2b5lXergYIgoyL+w48bgIB9N609qX7yd33kpu+hXpcYnp/jCvsOJ6ZXfLlNgLMti9YUPjY6njp2kbZfmUqCXBULKHWzfsm7BL08PhbrloQF71n1uCv0X9h1m8sy5xF/IzR+6hHyu/HqmldxlhNaLB0i4MG+Yar/VYvWFRyfINLnizF1pLeq6kUTlbv1H3jhNacZ39PiFHTem9vNnefdccuB94fhp+nrzmBXqzWRNrgpVbTdRK0rqrqr3DNV4N1eW0pm70hoU6KUq333prdTtj23bUFM2TZKoj7zH4MqVy/n5v7y74DU3XLUy+Hhpfcy1atRxk6QNOidlCG3/3svsHB7jnanp1MCfVXunkgFzTZJqPQr0UpVyfbRJKX71cN5JDPIAf/Xim/MGMvt68+zcuh5YWOu9UcG40UE+KSOoVFK3VFRtEpIHw7PmTlTSzRXRJKnWokAv84Te8mdlXUAhgIy8cTo1gyRtck09TU5N88BTh8mZzSseVm5FpVY26z7XH1/tJDRYeNWdNXeimqCtSVKtRYOxMqeSNUg//ttXJB4j2j40Os73D6UPyjksqEHfCOc9Oc0xTb4N/iLKrdUbGmTjATxrhnGlQVuTpFpPG/xay2IJrRszNDrOgVcn5m3LmfGp61fx2LYNqceKG+jr5fDDN/Op61fN3QW0gsCaYk2XdZWdtmpXqXgAz6oFFHo80KLhrUpdNzInpG5M0sBc0qBgyKLf0VW/cq8rl3WVXVraoC+l2mT8qjurbELp8dKynuIzc6W1BAV6M7sF+DqQA77l7rtKnn8A+GNgBpgA/sjd3yg+NwtEibdvuvvWOrVd6ixkRmxoCdysrBvDuX/f4dSAIdniJ8q08ZTS9Nik1wLzCqPdce0AB16dKHu80kVToLKF02XxlQ30ZpYDngBuAk4CB81s2N1fib1sFBh09zNm9ifA14C7i89NufvGOre7ozVrlZ4brlqZOFAZT1uspVpk5Eyxf6TRQT7Kva+2SmSr+verLgYIqjAaSQr8pft//9B4eLdLaW9b6/S+SYKQPvrrgGPufsLdzwF7gdvjL3D3A+5+pvjwReDy+jaze1QyIFpvpf3uSdsrqeu+rAkjmyuW5nh916385d0bWXHRkrYN8llx86fHT7NzeCzxzuoL+w6zOmAtgNDxmLR9S+v7TM960L7SHCF/iQNAfHbMyeK2NJ8B/jr2eJmZjZjZi2a2LWkHM7u3+JqRiYnkYNMtavkDzBKyIEjI1XrSwJxROCFFx41OVs0IsmfOzS6oY9Nu+pfnM1NPnfQVqiLlLhBqWaO30ev7Sv2F9NEnXVwk/h6a2aeAQeD3YptXufspM7sSeM7Mjrj78XkHc98D7IFCmeKglneoRvwRZU2GgfKDbPGr9dJqk/F8+Oi4y/I9dZ8oFeqyvt6qJvi0knqdILNmqIbW8U9Sy77SHCFX9CeBeNL05cCp0heZ2UeBLwFb3f1stN3dTxX/PwE8D2yqob0drxFL3qXdJewcHpvXTZS2bmvSghUv7LiRgb7exKqUzewuGZ+catsr+UZIu0C44aqVC67gQvPfQ6qbSmsJCfQHgbVmtsbMlgL3AMPxF5jZJuCbFIL8v8S295vZRcWvLwU2A/FBXCnRiD+itD/2yanpxCvfnFlq1cqQ40rrSLpAiNJa4ydpA+64NqyGfUh1U2ktZbtu3H3GzO4D9lNIr3zS3cfM7FFgxN2Hgd3A+4CnrTD5JUqj/HXgm2Z2nsJJZVdJto6UqHXJuySVFhg77z63klQ9jyvV6V+eZ/nSJVX9rJMKvaUtAxgfdC+X+VWPhU1k8WgpwS5QafXBKLBEk23cSax6+NDQkbauG9MOosloAF/Yd7ji/ZMmMZVbKjJ0Upy0Fi0l2KFKr7puuGrl3ISXi2P120snw0B6QbF8zvjVezNz/ezx/vZogZBHfjjGw7etT03HlPq549oLC3VXI6l7rdxgauikOGkfCvQN0uhJT0mZNPGr63j6XelkmKyFndPWP417+8x0Q0oQy0IHXp3gwKsTZX/WZpB0c57UR59V7gCamz7ZrMmCnU6BvgGy0hmr/aUt/QM4c26mokA7NT3LIz8cq9ukFgX5xRHaL9/Xm+e96fOpwTuu3DhQ2hV/jxlrdjzTsABc7d+NTg7lqY++jsott5ZW9KncL2qlfezSfQx4/O6NdQl4Ib9vjeiz37zrucS/naxiaRpPuEB99Isg5I9jfHJqwRVRyFVMu08Aksa7rK+3bpkwIdUqG9FnX02XkcYTwijQ18kjP1xYeyRJvH4NZE9miv7QWuueS1pNIyYrxU8aa1LGdOrdZ1/NjFuVYwijhUfqYGh0vOLZoOWWaZucmp6bsSpSqn95PnOyUkhto1CNmK2dpJrJgovVtnanQF8H1Q5wVrNMm3SuSlbaem/6PI/fvZEXdtyYGOTrWQF1sUoeVDPjVuUYwqjrpg6ybhP/sjhAlnZLmlW3fbFE4UV3D80RnxRV+ruQ6zFmE9a8zeqHrqXfOisxYDEyWyodZ1jMtrUzBfo6SOtb7OvNz/3ClVumrZpZj/XiFLoC2rV2ezvLmXHHtQNzgeri3jzL8j1zE922b1nH/fsOJ56E07K7qu23LpcY0KrBs5Xb1irUdVMHabePO7euB9JvSQE2PvLjpgb5iIJ8cyzJGft+9tZcN8vk1PSCbpm07j2DxO6YavutG7UWgjSfAn0dZAXyaEBs9/6jbN+yjtd23TqXE7z96ZfLLiAhne3szPl5a6/ChZWiokHU7VvWpS4KkRSEq+23VgZL51LXTQUqXYw56zZ49/6jC/7AReKi35mvfmxD6vhJUhCutt9aC4p0Ls2MDVTpDLxys/zSKgiKlOrrzfPO1HTq78tAnQYgNcu0vWlmbB1UmsmQdRs8NDqeumyfSKnQ9WEj1WagKIOlMVqhFo8CfaBK+y9TM3GW59n+9MtBQT6nk4EEimZTn505X3FRsDhlsNRXIwocVkODsYEqyWQYGh3nzLmZBdt78znem54N6pvP9yjISyGzZsXSXNnXQfLSkMqaaa5WyWRSoA8UmskQncFL0xX7evN89WMbmJo+H/T9rlvTz4AGwbqeA++emyXXEz5rtpSyZpqnVTKZFOgDhU7PTqs0OTk1XdFZ/KfHT3PDVSsXnFykO82ed1YszWEULhryufmBvzefo395PnFfZc00T6vU4lGgr8C2TQO8sONGHr97IwD3x3KdI1ln6koWd44Wa/7qxzZQQQkU6WBnzs3y+N0bOfzwzey+85oFFx0P37ZedV9aTKvU4lF6ZYWSUtAM+OT1q3hs24bUtMpqLc/3cCawu0c6X9YiHNAaGR4y32J9JlnplUGB3sxuAb4O5IBvufuukucfAP4YmAEmgD9y9zeKz30aeKj40sfc/X9mfa9WCfRJHw7AF59KzpiJVviBhXVtROqpXnnz0llqCvRmlgP+GbgJOAkcBD7u7q/EXnMD8JK7nzGzPwF+393vNrNLgBFgkEJvxCHgWnd/O+37tUKgT7pqz+cMnMyMmehqa2h0vCXq10jn0kQmKZUV6EP66K8Djrn7CXc/B+wFbo+/wN0PuPuZ4sMXgcuLX28BfuLup4vB/SfALdW8icWUNKA6Petl0yKjyVBKZ5N8g0e/lDYplQj5dRwA3oo9PlncluYzwF9Xsq+Z3WtmI2Y2MjExEdCkxqo69ckKhcrq2Ucv7akewyorluYyFyNR2qSECpkZm1Y4b+ELzT5FoZvm9yrZ1933AHug0HUT0Ka6SeqLT5vVWo47TLfY4La0p3zOODdzPnPSnNImJVTIFf1J4IrY48uBU6UvMrOPAl8Ctrr72Ur2bZa0JdduuGol+RomqIikyZml5sJHDFjSY5ldhUqblEqEXNEfBNaa2RpgHLgH+ET8BWa2CfgmcIu7/0vsqf3Afzez/uLjm4EHa251naRNTz7w6gTvW7ZEi3FIXRnw5//pmrkB1KHRcR546jCl8dwhcwa1sm6kUmUDvbvPmNl9FIJ2DnjS3cfM7FFgxN2Hgd3A+4CnrdCn+Ka7b3X302b2FQonC4BH3f10Q95JFdK6Z8YnpxL7nESqFc21KA3OlS5JUC6PXiRJUPVKd38WeLZk25djX380Y98ngSerbWAjpVWHzJnxaxcv06Cq1EX/8jwP37Y+sVxG1j7vTZ9PXGdYpFJdXaY4baBr1j1zUWaRLNEFRLkulqysGfdCN2LosUSydHWtm7TqkAN9vWzbNMAnr1+1oAsnn7OggVp1/XSf3nyOT12/il+7eFnQ55+VNRMtNjLrPnclryAv1erqQF+u4NBj2zbw+N0b5xWP2n3nNey+60JBqf7l+QWBvzef45PXr5p7zdKU7ArpLJf3L+P7h8YXZHHFi97FpS36XUqTo6RWXVPULK2wUGjBoaz9H/nh2FyGTl9vnp1b1887tvr6u1vWAOrqHc8EHcOA13bdWsdWSafp+jVjyy3nVe6WOG3/p0fe5KfHT8/rxz87c35un+1Pvxy0mpS0jlyP8ed3XVPXE3RWX/xA4OQ8TY6SWnRF102ty3ml7f9CSZCPH3fn8JiCfJvpX57nz+8q5LnfcNXKxNdU8weTFaSTug9LKdtGatUVV/S1LudVaU0RddW0p4dvW8/u/Ue5f99helJqzFy8PI/7hcHSUsb8Gh/lgnR0NxnvFrzhqpUceHVCNeWlbroi0KfVrgm9Ha609k3pH7vUV19vnnempquuSZTEbP46Ammpt5Nnpnlt1608NHSEb7/45oKgfse1A3NB+uLePGaFlch27z+aGrBDug9FatEVXTe1LueVtH9WtoQDKpXTOGaFgckXdtyYuYD65g9dEpzm2rukJ2ixmOjiICkj66sf28Bj2zbMLTd5duY8b5+ZDsrAEWmkrgj0oQt7V7L/J69flVqUCiqf2i7h3j4zPRcws1IUX//lFL/zoUvKHm95vieztkyk9OIgaw3hWseFROqpa9Ir09SynuPGR36c2ldbmMI+GxRApHLxlMW0FEUjrNvNgIt784mfZc6M8+5z3TCTZ6YXpNeWrkbWm8+l3h0oTVIapevTK9OUS7tM2yc6MWSdIn/13oyybhooPkCelqLYtzwfNJB+cW+ed8/NLNie7zF233UNQOrvSdqVe1odJaVJSjN0RddNmkpvr0vr16cxS15bVt329RMPmNu3rEvsRvvVezP0Lc9nHqc3nyt8XrMLP6/3LVvCtk0Dqb8nj/xwLPVuISpdUPq9lCYpzdDVgb7StMukP/hS+ZyR1hvWzdf35Qane/M9C4J1PmeJv6D5nC3oK1+xdOHN6fR5x53UgfRorGYyZd2BaHva70PWegXRsasdFxKpp67uuqk07TIklW/F0iWsuGiJculLXNybPGYR1Wl/bNuGBeMl756dSew3X7F0yYKA+U7KWMk7U9M8fvfGzHGYtFmw0e9BpWmc8SJkCuzSCro60GOC6L8AAAnPSURBVG/fsi5xIC3t9jqt3zXunalpdm5drxLHJdKufh048GphQfjSwLgmZZA1KahnnbTLBdxyvwdJz2fRlbu0mq7uuqk07bJckIcLgUVBPlxa10janVXS9lrmSpT7PUh6vq83ue8/KnEt0kq6+ooeKpuVWK4AVTywhBar6kQDfb1ls5Li0gJ6JXdcSaUEKkmVLfd7UPp8WlqlBlulFXX1FX2lsmbIll4FhhSrajX9y/P0l8lSKScaKA1NI8wKjts2DXDHtQPkinVncmbccW16QI4mMEWzZht5ZV3rJDyRxdT1V/SVqOSqMdoWr1UfyeeMFUuXpE62KtW/vDCZp5a5bZs/dMmCksqR3nxuLkil9YuHtjO+NmraOEU0CancVffQ6DjfPzQ+12U26873D40z+MFLWiKgarBV2kXXz4xdDCGzb9O6AuIBuNpPqq83z+GHb563EEraWqSbdz1XVZfT6wmzPdMKf4Ve+aa1JWshD5FuVfPMWDO7Bfg6kAO+5e67Sp7/CPCXwG8C97j792LPzQJHig/fdPetlb+F9hZy5Rc9v3N4bO5Kf1n+Qs9aWlZJdHXck5IRZMDOreuD21Fphgmkr7372LYNDH7wkqr7zWstLy0iBWUDvZnlgCeAm4CTwEEzG3b3V2IvexP4Q+BPEw4x5e4b69DWrhCtUAWFlMRoqn3awGR0dZx0RxDlqFfSvRDvngq5sg+pt15t90at5aVFpCDkiv464Ji7nwAws73A7cBcoHf314vPqYJXDbJKMkRdFWlXx7VmnSR1L8XvLuKiAehq1titRKXzHEQkWUigHwDeij0+Cfx2Bd9jmZmNADPALncfqmDfrjE0Op56BR11VVSaAljJ904q2nXHtQPs+9lb8+r2RIW+sr5PaLG4+MkgrTokVH/yEpGCkECfVKWkknHBVe5+ysyuBJ4zsyPufnzeNzC7F7gXYNWqVRUcujNEgTFNo7sq0u4kDrw6we7iQtmVBNqsO5No39KTQfzOodLF20UkW0igPwlcEXt8OXAq9Bu4+6ni/yfM7HlgE3C85DV7gD1QyLoJPXanyCqWthhdFVmDntUE2pBB1HIF4kpPDCJSvZAJUweBtWa2xsyWAvcAwyEHN7N+M7uo+PWlwGZifftSkJVFEpqKODQ6zuZdz7FmxzNzqxyFqqTUQL2OF5I5o+wakfooG+jdfQa4D9gP/BPwlLuPmdmjZrYVwMx+y8xOAncB3zSzseLuvw6MmNnLwAEKffQK9CXSAmNo3ZTSOvmVrk9a65q61Rzv4pRaMXHKrhGpj6A8end/Fni2ZNuXY18fpNClU7rfT4ENNbax49WaXRLSJ56l3oOeIcezsvXplV0jUi8qgdACag209ZhYVO9Bz3LHS1vsA1gwW1dEaqNA3yK6bWJRWptV3kCk/lS9sgPUu499MbRjm0Xala7oO0A7TixqxzaLtCtVrxQR6QA1V68UKacetW1EpDEU6KVmobVtRKQ5NBgrNcvK4xeR5lOgl5ppgRCR1qZALzWrd60cEakvBXqpmXLiRVqbBmOlZsqJF2ltCvRSF1ogRKR1qetGRKTDKdCLiHQ4BXoRkQ6nQC8i0uEU6EVEOpwCvYhIh1OgFxHpcAr0IiIdToFeRKTDBQV6M7vFzI6a2TEz25Hw/EfM7B/MbMbM7ix57tNm9vPiv0/Xq+EiIhKmbKA3sxzwBPAHwNXAx83s6pKXvQn8IfCdkn0vAR4Gfhu4DnjYzPprb7aIiIQKuaK/Djjm7ifc/RywF7g9/gJ3f93d/xE4X7LvFuAn7n7a3d8GfgLcUod2i4hIoJBAPwC8FXt8srgtRNC+ZnavmY2Y2cjExETgoUVEJERI9UpL2OaBxw/a1933AHsAzGzCzN4IPH6pS4F/rXLfVtDu7Yf2fw/t3n5o//fQ7u2H5ryHD6Y9ERLoTwJXxB5fDpwK/MYngd8v2ff5rB3cfWXgsRcwsxF3H6x2/2Zr9/ZD+7+Hdm8/tP97aPf2Q+u9h5Cum4PAWjNbY2ZLgXuA4cDj7wduNrP+4iDszcVtIiKySMoGenefAe6jEKD/CXjK3cfM7FEz2wpgZr9lZieBu4BvmtlYcd/TwFconCwOAo8Wt4mIyCIJWmHK3Z8Fni3Z9uXY1wcpdMsk7fsk8GQNbazEnkX6Po3S7u2H9n8P7d5+aP/30O7thxZ7D+YeOq4qIiLtSCUQREQ6nAK9iEiHa4tA3wm1dmp8D7Nmdrj4LzTjqa4C2v+Amb1iZv9oZv/bzD4Ye65dPoOs99AOn8FnzexIsY1/Hy9VYmYPFvc7amZbFrfl89pY1Xsws9VmNhX7DL6x+K2fa2Pme4i97k4zczMbjG1rzufg7i39D8gBx4ErgaXAy8DVJa9ZDfwm8L+AO2PbLwFOFP/vL37d307vofjcr9rgM7gBWF78+k+AfW34GSS+hzb6DD4Q+3or8DfFr68uvv4iYE3xOLk2ew+rgf/bzM8g9D0UX/d+4P8ALwKDzf4c2uGKvhNq7dTyHlpBSPsPuPuZ4sMXuZCF1U6fQdp7aAUh7f+32MMVXJiFfjuw193PuvtrwLHi8RZbLe+hVZR9D0VfAb4GvBfb1rTPoR0CfcNr7SyCWtuxrFgL6EUz21bfpgWptP2fAf66yn0bpZb3AG3yGZjZ58zsOIUg8/lK9l0EtbwHgDVmNmpmf2dmv9vYpqYq+x7MbBNwhbv/qNJ9GyUoj77JGl5rZxHU2o5V7n7KzK4EnjOzI+5+vE5tCxHcfjP7FDAI/F6l+zZYLe8B2uQzcPcngCfM7BPAQ8CnQ/ddBLW8h19Q+Ax+aWbXAkNmtr7kDmAxZL4HM+sBHqdQtr2ifRupHa7oa621U+2+9VRTO9z9VPH/ExRqBW2qZ+MCBLXfzD4KfAnY6u5nK9l3EdTyHtrmM4jZC0R3Hm31GcTMvYdid8cvi18fotC//eEGtTNLuffwfuA3gOfN7HXgemC4OCDbvM+h2YMbAYMfSygM4K3hwuDH+pTX/g8WDsa+RmEQsL/49SVt9h76gYuKX18K/JyEwZ9mt59C4DsOrC3Z3jafQcZ7aJfPYG3s69uAkeLX65k/CHiC5gzG1vIeVkZtpjAQOt6qv0clr3+eC4OxTfscFvWHVMMP9z8A/1z8I/xScdujFK66AH6LwtnyXeCXwFhs3z+iMOhxDPjP7fYegN8BjhR/QY4An2nR9v8t8P+Aw8V/w234GSS+hzb6DL4OjBXbfiAegCjcpRwHjgJ/0MKfQeJ7AO4obn8Z+AfgtlZ9DyWvfZ5ioG/m56ASCCIiHa4d+uhFRKQGCvQiIh1OgV5EpMMp0IuIdDgFehGRDqdALyLS4RToRUQ63P8HbeO0El93sKgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "## Lets make a scatter plot of the true versus predicted values\n", "\n", "plt.scatter(molecules_y_test,molecules_y_pred)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([926., 613., 288., 132., 32., 5., 3., 0., 0., 1.]),\n", " array([3.04527283e-06, 1.93773367e-02, 3.87516281e-02, 5.81259195e-02,\n", " 7.75002109e-02, 9.68745023e-02, 1.16248794e-01, 1.35623085e-01,\n", " 1.54997377e-01, 1.74371668e-01, 1.93745959e-01]),\n", " )" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAQZ0lEQVR4nO3de4xc5XnH8e8Djk0hAhtYIrCdrGmcthAqoBsgiRIVnCgB2hgJaEhJsKgltwltk1KpmJAqUqVKRq0KQa1AVlwwbS5QmgirJK0ol0qVYpo190uJF0NhwYUlXJpAQ3B5+se8C8N67Z2dO7zfjzSac97znnOePXP4zfF7ZobITCRJddln0AVIkvrP8JekChn+klQhw1+SKmT4S1KFFgy6AIBDDz00R0dHB12GJL2lbNu27dnMHGln3aEI/9HRUcbHxwddhiS9pUTEf7W7rsM+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUoaH4hm8nRtffNLB9P7bh9IHtW5I64ZW/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalChr8kVcjwl6QKGf6SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQi2Ff0T8UUQ8EBH3R8S3ImK/iFgREXdExPaIuC4iFpa+i8r8RFk+2ss/QJI0f3OGf0QsBf4QGMvM9wP7AucAlwKXZeZK4HlgbVllLfB8Zr4XuKz0kyQNkVaHfRYAvxARC4D9gZ3AKcANZflm4IwyvbrMU5aviojoTrmSpG6YM/wz80ngL4HHaYT+i8A24IXM3FW6TQJLy/RS4Imy7q7S/5CZ242IdRExHhHjU1NTnf4dkqR5aGXYZwmNq/kVwBHAAcCps3TN6VX2suyNhsyNmTmWmWMjIyOtVyxJ6lgrwz4fAx7NzKnMfBX4DvAhYHEZBgJYBjxVpieB5QBl+UHAc12tWpLUkVbC/3HgpIjYv4zdrwIeBG4Dzip91gA3luktZZ6y/NbM3O3KX5I0OK2M+d9B48btncB9ZZ2NwEXAhRExQWNMf1NZZRNwSGm/EFjfg7olSR1YMHcXyMyvAl+d0bwDOGGWvj8Dzu68NElSr/gNX0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalChr8kVcjwl6QKGf6SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUIthX9ELI6IGyLiPyPioYj4YEQcHBE3R8T28ryk9I2IuCIiJiLi3og4vrd/giRpvha02O9rwD9n5lkRsRDYH/gycEtmboiI9cB64CLgVGBleZwIXFme33ZG1980kP0+tuH0gexX0tvHnFf+EXEg8FFgE0Bm/jwzXwBWA5tLt83AGWV6NXBtNmwFFkfE4V2vXJLUtlaGfY4EpoCrI+KuiPh6RBwAvCszdwKU58NK/6XAE03rT5Y2SdKQaCX8FwDHA1dm5nHASzSGePYkZmnL3TpFrIuI8YgYn5qaaqlYSVJ3tBL+k8BkZt5R5m+g8Wbw9PRwTnl+pqn/8qb1lwFPzdxoZm7MzLHMHBsZGWm3fklSG+YM/8z8b+CJiPil0rQKeBDYAqwpbWuAG8v0FuC88qmfk4AXp4eHJEnDodVP+/wB8I3ySZ8dwPk03jiuj4i1wOPA2aXv94DTgAng5dJXkjREWgr/zLwbGJtl0apZ+iZwQYd1SZJ6yG/4SlKFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalChr8kVcjwl6QKGf6SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalCLYd/ROwbEXdFxD+V+RURcUdEbI+I6yJiYWlfVOYnyvLR3pQuSWrXfK78vwg81DR/KXBZZq4EngfWlva1wPOZ+V7gstJPkjREWgr/iFgGnA58vcwHcApwQ+myGTijTK8u85Tlq0p/SdKQaPXK/3LgT4DXyvwhwAuZuavMTwJLy/RS4AmAsvzF0v9NImJdRIxHxPjU1FSb5UuS2jFn+EfEbwDPZOa25uZZumYLy95oyNyYmWOZOTYyMtJSsZKk7ljQQp8PA5+KiNOA/YADafxLYHFELChX98uAp0r/SWA5MBkRC4CDgOe6XrkkqW1zXvln5sWZuSwzR4FzgFsz81zgNuCs0m0NcGOZ3lLmKctvzczdrvwlSYPTyef8LwIujIgJGmP6m0r7JuCQ0n4hsL6zEiVJ3dbKsM/rMvN24PYyvQM4YZY+PwPO7kJtkqQe8Ru+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalChr8kVcjwl6QKGf6SVCHDX5IqZPhLUoXm9f/w1XAYXX/TwPb92IbTB7ZvSd3jlb8kVcjwl6QKGf6SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUJzhn9ELI+I2yLioYh4ICK+WNoPjoibI2J7eV5S2iMiroiIiYi4NyKO7/UfIUman1au/HcBf5yZvwKcBFwQEUcB64FbMnMlcEuZBzgVWFke64Aru161JKkjc4Z/Zu7MzDvL9E+Ah4ClwGpgc+m2GTijTK8Grs2GrcDiiDi865VLkto2rzH/iBgFjgPuAN6VmTuh8QYBHFa6LQWeaFptsrTN3Na6iBiPiPGpqan5Vy5JalvL4R8R7wT+EfhSZv7P3rrO0pa7NWRuzMyxzBwbGRlptQxJUhe0FP4R8Q4awf+NzPxOaX56ejinPD9T2ieB5U2rLwOe6k65kqRuaOXTPgFsAh7KzL9qWrQFWFOm1wA3NrWfVz71cxLw4vTwkCRpOCxooc+Hgc8B90XE3aXty8AG4PqIWAs8Dpxdln0POA2YAF4Gzu9qxZKkjs0Z/pn578w+jg+wapb+CVzQYV2SpB7yG76SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klShVn7eQXrd6PqbBrLfxzacPpD9Sm9XXvlLUoUMf0mqkOEvSRUy/CWpQoa/JFXI8JekChn+klQhw1+SKmT4S1KFDH9JqpDhL0kVMvwlqUKGvyRVyPCXpAoZ/pJUIcNfkipk+EtShQx/SaqQ4S9JFTL8JalChr8kVcjwl6QKLRh0AVIrRtffNJD9Prbh9IHsV+o1r/wlqUKGvyRVyPCXpAr1JPwj4pMR8XBETETE+l7sQ5LUvq6Hf0TsC/wNcCpwFPCZiDiq2/uRJLWvF5/2OQGYyMwdABHxbWA18GAP9iX11KA+ZTRIfsKpDr0I/6XAE03zk8CJMztFxDpgXZn9aUQ83Ob+DgWebXPdfhjm+qytPcNcG3RYX1zaxUp2N8zH7q1Y23va3WAvwj9macvdGjI3Ahs73lnEeGaOdbqdXhnm+qytPcNcGwx3fdbWnl7U1osbvpPA8qb5ZcBTPdiPJKlNvQj/HwIrI2JFRCwEzgG29GA/kqQ2dX3YJzN3RcTvA/8C7Av8bWY+0O39NOl46KjHhrk+a2vPMNcGw12ftbWn67VF5m7D8ZKktzm/4StJFTL8JalCQxf+c/00REQsiojryvI7ImK0adnFpf3hiPhEq9vsdW0R8fGI2BYR95XnU5rWub1s8+7yOKzPtY1GxP827f+qpnV+rdQ8ERFXRMRsH+PtZW3nNtV1d0S8FhHHlmVdOW4t1vfRiLgzInZFxFkzlq2JiO3lsaapvV/HbtbaIuLYiPhBRDwQEfdGxKebll0TEY82Hbtj+1lbWfZ/Tfvf0tS+opwD28s5sbCd2jqpLyJOnnHe/SwizijL+nXsLoyIB8trd0tEvKdpWXfOucwcmgeNG8SPAEcCC4F7gKNm9PkCcFWZPge4rkwfVfovAlaU7ezbyjb7UNtxwBFl+v3Ak03r3A6MDfC4jQL372G7/wF8kMZ3N74PnNrP2mb0OQbY0c3jNo/6RoFfBa4FzmpqPxjYUZ6XlOklfT52e6rtfcDKMn0EsBNYXOavae7b7+NWlv10D9u9HjinTF8FfH4Q9c14jZ8D9u/zsTu5aZ+f543/Xrt2zg3blf/rPw2RmT8Hpn8aotlqYHOZvgFYVd7hVgPfzsxXMvNRYKJsr5Vt9rS2zLwrM6e/6/AAsF9ELGqjhq7XtqcNRsThwIGZ+YNsnFnXAmcMsLbPAN9qY/8d15eZj2XmvcBrM9b9BHBzZj6Xmc8DNwOf7Oex21NtmfmjzNxepp8CngFG2qih67XtSXnNT6FxDkDjnGjnuHWzvrOA72fmy23W0W5ttzXtcyuN70tBF8+5YQv/2X4aYume+mTmLuBF4JC9rNvKNntdW7Mzgbsy85WmtqvLPyH/tM3hgU5rWxERd0XEv0XER5r6T86xzX7UNu3T7B7+nR63Vuub77r9PHZziogTaFxhPtLU/OdlSOGyNi9EOq1tv4gYj4it00MqNF7zF8o50M42u1nftHPY/bzr97FbS+NKfm/rzvucG7bwb+WnIfbUZ77t89VJbY2FEUcDlwK/27T83Mw8BvhIeXyuz7XtBN6dmccBFwLfjIgDW9xmr2trLIw4EXg5M+9vWt6N49ZqffNdt5/Hbu8baFwR/h1wfmZOX+FeDPwy8AEawwcXDaC2d2fj5wp+G7g8In6xC9ts1q1jdwyN7yxN6+uxi4jPAmPAX8yx7rz/3mEL/1Z+GuL1PhGxADiIxpjcntbt1s9NdFIbEbEM+C5wXma+fgWWmU+W558A36TxT8K+1VaGyX5cathG4+rwfaX/sqb1B3Lcit2uvrp03Fqtb77r9vPY7VF5E78J+Epmbp1uz8yd2fAKcDW9O+f2aHoYNBu//ns7jftizwKLyzkw7212s77it4DvZuar0w39PHYR8THgEuBTTSMF3TvnOrlx0e0HjW8c76Bxw3b6RsjRM/pcwJtvDl5fpo/mzTd8d9C4sTLnNvtQ2+LS/8xZtnlomX4HjbHO3+tzbSPAvmX6SOBJ4OAy/0PgJN64gXRaP2sr8/vQOLGP7PZxa7W+pr7XsPsN30dp3HhbUqb7euz2UttC4BbgS7P0Pbw8B3A5sKHPtS0BFpXpQ4HtlBuewD/w5hu+X+j369rUvhU4eRDHjsab4SOUm/a9OOfmfVB7/QBOA35U/vBLStuf0Xj3A9ivnCATNO5uN4fCJWW9h2m60z3bNvtZG/AV4CXg7qbHYcABwDbgXho3gr9GCeI+1nZm2fc9wJ3AbzZtcwy4v2zzrynfCO/za/rrwNYZ2+vacWuxvg/QeAN6Cfgx8EDTur9T6p6gMbTS72M3a23AZ4FXZ5xzx5ZltwL3lfr+Hnhnn2v7UNn/PeV5bdM2jyznwEQ5JxYN6HUdpXEhtM+Mbfbr2P0r8HTTa7el2+ecP+8gSRUatjF/SVIfGP6SVCHDX5IqZPhLUoUMf0mqkOEvSRUy/CWpQv8PCLP/PonQwgkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "## and a histogram of the magnitude of the error\n", "plt.hist(abs(molecules_y_test- molecules_y_pred))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "rc3nkPWlDQSf" }, "source": [ "## Regularization\n", "\n", "The more features we have, the more freedom the algorithm has to fit the data. For instance, when doing polinomial regression, the higher the degree of polinomious we add, the more closely the algorithm can build a curve the pass through the data. \n", "\n", "Unfortonatelly, this flexibility cames with a price: the model can learn very well the training set, but may fail in predicting correctly out of the sample data points. This process is known as **overfitting**. \n", "\n", "However, reducing the number of features (e.g., reducing the degree of the polonima in the polinomial regression), we may have very simple models, which are not able to fit the data. This process is known as **underfitting**.\n", "\n", "\n", "There is a trade-off between making the model more specific (capturing more information from the training set) or more geneal (aiming to generalize to out-of-the sample data). \n", "\n", "\n", "![](https://cdn.analyticsvidhya.com/wp-content/uploads/2018/04/Screen-Shot-2018-04-03-at-7.52.01-PM-e1522832332857.png)\n", "\n", "Regularization is a way to try to control this trade-off, adding a penalty to the parameters of the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Tb79pVNGZR__" }, "source": [ "## Ridge Regression\n", "\n", "In Ridge-Regression, the regularization penalty is taken to be the L2-norm of the coefficients. The idea is not to remove features beforehand, but adding a penalty to the magnitude of coefficients of the regression. This make the coefficients \"compete\" to each other: if a coefficient increase, other have to decrease so that their sum does not increase to much. \n", "\n", "$$ \\mathbf{\\theta} = (X^\\intercal X - \\alpha I)^{-1} \\cdot X^\\intercal y$$\n", "\n", "Where $\\alpha$ is a parameter of the algorithm that controls the strenght of the regularization.\n", "\n", "The example below experiments with different values of $\\alpha$ for Rigid Regression\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for alpha 0.001 : 0.000924 \n", "Mean squared error for alpha 0.010 : 0.000940 \n", "Mean squared error for alpha 0.100 : 0.000945 \n", "Mean squared error for alpha 1.000 : 0.000952 \n", "Mean squared error for alpha 10.000 : 0.000969 \n", "Mean squared error for alpha 100.000 : 0.000986 \n" ] } ], "source": [ "from sklearn.linear_model import Ridge\n", "\n", "for alpha in [0.001, 0.01, 0.1, 1, 10, 100]:\n", " \n", " regr = Ridge(alpha)\n", " model = make_pipeline(PolynomialFeatures(2), regr)\n", " \n", " model.fit(molecules_X_train, molecules_y_train)\n", "\n", " molecules_y_pred = model.predict(molecules_X_test)\n", "\n", " print('Mean squared error for alpha %.3f : %.6f '\n", " % (alpha, mean_squared_error(molecules_y_test, molecules_y_pred)))\n" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "5B_KPSPMZgF0" }, "source": [ "## Kernels\n", "\n", "The models we have seen so far are linear or ponomial. This restriction limits their usage for data sets which do not fit in this family of functions. \n", "\n", "In machine learning, a \"kernel\" usually refers to the kernel trick, a method of using a linear classifier to solve a non-linear problem. It entails transforming linearly inseparable data like to (hopefully) linearly separable ones. \n", "\n", "\n", "\n", "\n", "The RBF kernel is one exemple of a kernel used in machine learning. The ideia is to associate a Gaussian centered in each point of the training set, and compute the inverse of this Gaussian function w.r.t the other points:\n", "\n", " \n", "\n", "Examples which are closer to the reference example are mapped to a higher value, whereas examples which are far are mapped to a low value. \n", "\n", "\n", "\n", "The parameter $\\sigma$ control the neighborhood ratio around the reference example.\n", "\n", "Larger $\\sigma$ takes into account distant examples\n", "\n", "\n", "\n", "While small $\\sigma$ takes into account close examples\n", "\n", "\n", "\n", "\n", "The example next shows the use of Kernel Ridge Regression " ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "colab": {}, "colab_type": "code", "id": "RyNK9VqMlGwM" }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUxfrA8e/sbnrvJJBkA6F3CEXFhnLBBhasoGABAVH02kUUQSwIiIqoXBQRY/ldUcCC13LhIioldAidFNLLpiebze7O74+zhAAJCWkQMp/n2WeTPZNTQnhnzjtzZoSUEkVRFOXipzvfJ6AoiqI0DxXwFUVRWgkV8BVFUVoJFfAVRVFaCRXwFUVRWgnD+T6BswkMDJRGo/F8n4aiKEqLsW3bthwpZVB12y7ogG80GomLizvfp6EoitJiCCGSatqmUjqKoiithAr4iqIorYQK+IqiKK2ECviKoiithAr4iqIorUSDA74QIlwIsU4IsV8IsU8IMa2aMkII8a4Q4ogQYrcQol9Dj6soiqKcm8Zo4VuBJ6WUXYHBwCNCiG6nlbkO6Oh4TQQ+aITjKkqDxMbGYjQa0el0GI1GYmNjz/cpKUqTavA4fCllOpDu+LpICLEfaAvEVyk2CvhManMxbxJC+AohQh0/qyjNLjY2lokTJ1JaWgpAUlISEydOBGDMmDHn89QUpck0ag5fCGEE+gKbT9vUFjhe5fsUx2fV7WOiECJOCBGXnZ3dmKenKJWmT59eGexPKC0tZfr06efpjBSl6TVawBdCeAIrgcellIWnb67mR6pdeUVKuURKGSOljAkKqvbpYEVpsOTk5HP6XFEuBo0S8IUQTmjBPlZK+W01RVKA8CrftwPSGuPYilIfERER5/S5olwMGmOUjgA+BvZLKRfUUGwNcJ9jtM5goEDl75Xzac6cObi7u5/ymbu7O3PmzDlPZ6QoTa8xJk+7DLgX2COE2On47AUgAkBK+SHwE3A9cAQoBe5vhOMqSr2d6JidPn06ycnJREREMGfOHNVhq1zUxIW8iHlMTIxUs2UqiqLUnRBim5Qyprpt6klbRVGUVkIFfEVRlFZCBXxFUZRWQgV8RVGUVkIFfEVRlFZCBXxFUZRWQgV8RVGUVkIFfEVRlFZCBXxFUZRWQgV8RVGUVkIFfEVRlFZCBXxFUZRWQgV8RVGUVkIFfEVRlFZCBXxFUZRWQgV8RVGUVkIFfEVRlFaisRYx/0QIkSWE2FvD9quEEAVCiJ2O10uNcVxFURSl7hpjTVuAT4FFwGdnKfOHlPLGRjqeoiiKco4apYUvpdwAmBpjX4qiKErTaM4c/iVCiF1CiLVCiO41FRJCTBRCxAkh4rKzs5vx9BRFUS5uzRXwtwORUsrewHvAqpoKSimXSCljpJQxQUFBzXR6iqIoF79mCfhSykIpZbHj658AJyFEYHMcW1EURdE0S8AXQrQRQgjH1wMdx81tjmMriqIomkYZpSOE+BK4CggUQqQALwNOAFLKD4HRwGQhhBUoA+6SUsrGOLaiKIpSN40S8KWUd9eyfRHasE1FURTlPFFP2iqKorQSKuAriqK0EirgK4qitBIq4CuKorQSKuAriqK0EirgK4qitBIq4CuKorQSKuAriqK0EirgK4qitBIq4CuKorQSKuAriqK0EirgK4qitBIq4CuKorQSKuAriqK0EirgK4qitBIq4CuKorQSKuAriqK0Eo0S8IUQnwghsoQQe2vYLoQQ7wohjgghdgsh+jXGcRVFUZS6a6wW/qfAiLNsvw7o6HhNBD5opOMqTSg2Nhaj0YhOp8NoNBIbG1vtZ6eXDQwMJDAw8IwyiqKcZ1LKRnkBRmBvDds+Au6u8v1BILS2ffbv318qzePzzz+XkZGRUgghIyMj5eTJk6W7u7sEKl9OTk7S2dn5lM/c3d2rLXt6mc8///x8X6KitApAnKwhpgpte8MJIYzAD1LKHtVs+wF4Q0q50fH978CzUsq4s+0zJiZGxsWdtYhSD7GxsUybNo3c3FwAPDw8qKiowGKxVJYRQlDXvw29Xo/NZjtrmcjISBITE+t9zoqi1I0QYpuUMqa6bYbmOodqPqs2mgghJqKlfYiIiGjKc2p1Tg/0J5SUlJxR9lwaArUFe4Dk5OQ6709RlKbRXKN0UoDwKt+3A9KqKyilXCKljJFSxgQFBTXLybUGsbGxTJw48Yxg3xj0en2tZRpSedfUb1Af1rw8SjZtInfZp6Q+8wzHbhpJyqOPIetQaSlKS9dcLfw1wFQhxFfAIKBASpneVAczlVgwlZQTFeiJXlfdzUXrM336dEpLS8/pZ05P6zg5OSGEOCX14+7uzrhx41i+fHmN+3d3d2fOnDn1Ou8TFdWJfSclJTFx4kQAxowZU+f95H6yDNOKFVjTT/7ZGUJCcA4Pp+jXX8l5fzFBjz1ar3NUlBajpuT+ubyAL4F0oAKtNf8gMAmY5NgugPeBo8AeIKYu+61vp+3nmxJl5LM/yM4v/iRHLdooX/h2t4zdlCR3JOfJMou1Xvu8UJ3e2VpT56gQosZO1epeJzpjT993Tcer+nlAQIAMCAio9ZzqIjIystrzi4yMlFJKOXnyZKnX6yUg9Xq9nDx58hn7KNm6VcZ37iITx94rc5YulUUbN8qK3NzK7anPvyDju3SVRRs31vs8FeVCQXN02jaF+nbapuaX8ffRXOLTCtmXVkB8eiFFZisAOgEdgjzpHuZNtzBvuof50C3UGz8P58Y+/SZ3eusXtNb0kiVLzmj9Go1GkpKSatyXXq/H19cXk8lEREQEc+bMOacWdFPR6XTV9icIIZg0aRIffHDmCN/JkyezePFiAOxlZRy7+Waw2Wm/ZjU6d/czytvLyki84w6suSaivvsOp5Dgxr8QRWkmZ+u0vSgD/umklKTklbEvrZB4RwWwL62Q9AJzZZkwH1e6hXnTzVEBdA/zpp2fG0JcuCmhmoJ4dSNiqqscTggICOCdd965IAL86c52jSkpKdV2GOv1eqxWrYLPfONNTJ9+SsSnn+IxeFCNxyk/coSE2+/ArUcPIpZ9gjA0V7ZTURpXqwr4sbGxTJ8+neTk5FpbqqYSC/FphcSnFzgqg0KOZhdjd/xKvF0NWiUQ6lN5RxAd7ImT/sKYkeJsrV+73X7G5+fyu6mOlBJpNmM3m9H7+jZLZXi2u5ixY8ee9VxLt+8gacwYfO+6k9CXX671WPmrVpH+3PMETplM0GOPNcr5K0pzazUB/1xSHDUps9g4kFFYeRcQn1bIgYxCzBVaAHXW6+jUxpPuoT6OlJA3XUK98XRpxBZhRRkUpUOpCcz5YC4EcwGUF57y9a8/fIuLLMPHReDlInDRg7MeXJx0eLu7gs0CsmrgdwRonR50TqB3BoMLOLmBs4f27uQOLt5IVx/K0m2Y/jxOWWIu9nIr9tJycFQkbv3702bGi7h26dJ4112Dmioqg8FQYwvfUlxMwi23IsvL2Tp2DM/PmlWnii7thekUfPcd4f/6F55DLmvqS1OURtdqAv65pDjOhdVmJzG3pLIC2OfoG8grrQBACDAGeNAt1NuRFtIqgmAv1zN3Zi6A3KNQlAFFaVCYfvLrogwoTNOCfE2EDlx9wMUbU5mdPYeTyCu1UVQuMdvALgxccfW1dO3eUwvqOseQycp/Zwl2G9grwFYBVjNUmKGiBCrKkKVFFOzOxbSjlPJc0DnZ8Qwzo3e2o3OS6JwkNjvkHfJEWnRY+nmRMyISk58fBc6uFBicKNIbKMGOTdrQCR06dOh02rsQAp3QoRd67esaPtOJk68T290N7gS7BxPsHkwb9zbMeX4OHy3+6Ixf0eTJk5nZvj25Sz/m8Jh7uPuNN+rcCFD5fKWlazUB/1xTHA0hpSSj0FxZAcSnFbIvvYDjpjIAnLDSzyOHq3yz6eOSSnt7Ev7FR3AqTj3t5HTgGQJeoeAdBl5tWL1uK6v/u4XMYhuFFsE/bhrNjDnzwMVba4lXSaU0NE1TVclff5E+ezYVCYnYjGFk3TCAg/2DyLBkkFWSTk5ZDlnl+eRZS3Avk9zxh53h2yXFrvDVlTp+7y1wQeJpl3gABr0LdoPjpXfGrtMjpcQmbUgpsWPHLu3VfnbiVfWz6hgqDBSnF2NOM2NJsXB1j6uZf/MkCu9/BN/bbuWSFSvOuRFQfvQoCaNvP2/5/Mb8N1Van1YT8JuqhX9WUkLBcciMh6x9WNL2Yk3fi2vBMXRS6ziskHqOyjAOynCOiQgq/KLxCzUSHhFNxw7tiQryRud4XmDKlCm1jjxpqHxzPomFiRwvOk5yUTKZmQl0+WIzfTfnkO4Hn12jY1u0ACEQCALcAghyCyLYPZhAt0CC3IMIcA3A18UX/7QivBZ9DTv24dwpirCJt+AWZIfcI5C+G9J3gVWrBHHxgbA+0LYfhPXT3r3bnlKB1fxrlpRUlJBVmkVmaSaZpZlklWaRVZpFWnEaR/KPkF6SjsEqeXOZDQ+LjtgXBvDd//0H83Ez5mQz5anlSJv2915bI+BEPj9g0sMEP/54o/ze66Ix0pJK69ZqAn6T/2cpNUHWfsiKh8x92nvWfi23foJPOAR3g5BuENwdQrpR7tuewzmWymGiex3vJ/oFvFwM9GjrQ692Prz29CTMqQexFmSecuiqI0/qwi7tpBalcqzgGAkFCZXvCYUJFJQXVJbrdwweXivxKbJxcERnTGP+QYhfOKEeoYR6hhLsFoyT3umsx5JSUrR2LZlz38JmMhG+9F94DByobbRZIfsApG2H1O3ae+Y+sDuuxSO4SgXQH8L6gkdAna+zqkJLIQlz5+D8+Rr+98QV/K9tIXvS9yCctQrFXm6n5FAJxfuK8c7z5sjfR9CJmjvg06ZPp+Db7whfsgTPy4fU65zO1XlptCgXlVYT8KGRbocrzJBzsLLVrr3Hax2pJ7j6VAZ0LcB3h+Cu2ud1YLXZOZJdzO6UAvakFLA7JZ/96UVYbFolYCsrxJJxhPL0Q1jSD1Gefghbcd4Z+5FSklOWw8G8gxzJO8Lh/MMczT/KsYJjlJ1oWQMBrgFE+UQR5RNFpHckUfoQ2ny8Ftv3v+Ac3YGw117DrVevc/s9nX5NeXkkjRmLNSuLyM+W49qtW/UFK8yQsefUSiDnMJXTK/lGnnoXENobXLxqPX7Z3n0k3nknPiNHEvb6awCs+HwFj770KDJI4tHRA49uHri21fpW/Fz8GBg6kCFth3BNxDV4OZ96jMp8fk4uUau+wykkpN6/m7pqzrSkcnFqVQH/nNjtkJ94MqCfaLXnHgXpGP2hd4agzlpQrwzs3bR8eyMPS7RY7fhEdMEQ3B7n0E64tOmIU1AkwtHx2sbHmU5tzQQG5CJcUsm3JXIk/xAms6lyH0FuQUT7RhPtF020bzTtfdoT5ROFj8vJiqj4zz9JnzEDa0YmAQ89RODUR9A5N86DZxXp6STeMwZpsRD52XJcOnSo2w+aCyF958kKIHUHFJyYcE1o/wbhgyD6Gmh/1RkVq7RYSBh9O7a8PNr/8D16n5PbT28EPPfqc7S9pC2b0jexKX0TWaVZOOucuTL8Sm6IuoEh7YbgoncBquTzu3cn4tNlTZ7PVy18paFUwAcoyTkZ0CvTMQe00Skn+BnPbLX7dwB983XancjhC2eBa7grbkYvPDoHEtQ9GKtHIXa0eWykXY/dEoIHkUR6RdM3pBtXRfViQEQ7nA3VpykqMjLIXvgOBatW4RwVRehrc3Dv27fRr6H82DGSxozFXlJC4ORJ+D/4YP0qlOJsSNvhqAC2QfImLX0m9CeDf8dhENKT7EXvk7N4Me0WL8Zr6NV1PoSUkj05e/gp4SfWJqzFZDbh5eTFMOMwro+6npiQGIq//4G0Z59rlvH5KoevNFTrC/h5SZD4x6kpmZKsk9vdA05trYd0h6Au4OLZeCdfjZrSTRabhcN5h9mXu4+9OXv5ZdcvFLsWI/TaHYTBaqBvu7508e9CF/8uhLl2oKDIj32pJew6ns+ulHxyirWKwNmgo3uYN33CfekT7kvfcD/C9BZMH3+M6bMVYLfjP3681qp3cWmya63IzCLz9dcp+vlnnKOiaPPyy2d90rVObBWQshWO/AaHf4WM3QCYy0NIWG3A+4oY2r7/MdTS51ATq93KlvQt/JjwI78l/UaptZRgt2BGRI3ghs+PIP77F+3XrMElKqph11ELNUpHaYjWF/DjlsEPj4PBVQvklYHd0ZHqGdzo6ZjaVLbczKW4tnXFLcoNr2gvOl3RiRxdDlZHJ6aviy/dA7vTPaA73QK60dW/K6EeoWd9qlVKSWp+GbuOF7DzeB67jhewOzUfm7mckcf+5K7D/8XDUkbGwKtwmTiJnjFd8XatX1A8V8UbNpAxazYVKSn4jBpF8DNPYwioX6fsGYoykYd+I+GpBVjziml/XSYGX1/ofAN0G6mlfgz1q9TMVjPrU9bz07Gf+CP1DzwLK1i4VKILCabbv1fh4lW3vhpFaW6tL+CXmrSXf9TJB4/OkzJrGXtz9jLm2TFYgiy4R7ujd9POyVZqQ2ZIJo6cSPeA7nQP7E6YR1iDpiyoSE+neONGijf8QdFffyNKijnesQ+xvW7kfzZfQKvrooM86RvhS98IP/pG+NIx2KvJppK2l5WR8+FH5H7yCTp3d4Kf/Ce+o0cjdA2foiJ36VKy5s2n7fy5eHfQQ/xqOLgWygu05xY6jdCCf/S12pPE9fDx5x8z86uZDG7nysz1BjZGw4E7hvPaPa/jZqjfPlsSdcfRsrS+gH8e5Zvz2ZG1gx1ZO9iWtY343HisdivSLilPLafkUAmlR0opO1aGJcuCoH6jL+wlJVRkZFCRnkFFehqWI0cp/nMjliNHATC0aYPn5UPwvvEmPAZpQyQLyirYdTyfncfz2Z6cx87j+eQ7nhb2dDHQO9yHfo4KoG+4X6PPIFp+5AgZM1+hNC4Ot759aTNzJq6dO9V7f+ZDh0i8bTSeQ4fSduHbJytKqwUS/gfxq+DAj1CWp00Z0WkE9L4bOgytc79MbGws999/PxUV2u/p0W6hTLb5sOxaHf8b4MmEfhO4s/OdZ4zwuVioPoWWRwX8JpRWnMb2rO1sz9ReRwu0gOukc6JHYA/6Bvelf0h/7rv2PpIO1n30hb28HPO+eCrS0rBmpFORlk5FuvaypqdjKyg4pbxwcsJ9wAA8Lr8czyGX4RwdXeudgpSSxNxSdiTnsSNZqwQOZBRhc8we1z7Qg74RfvSL9KVfhB+dQhp+FyClpGDVarLefBNbURH+48cR9Mgj1U5bfNb9VFSQcOedWDMyaf/D9xj8/asvaKuAxI1ayz9+lRb8PYKh5+3Q+y5o0/Os6b3TR80I4L2wtlzh6cnLVwkOX2rA08mTu7vczZiuYwhwa6R01QVCjRpqeVTAbyR2aedI/hF2ZGqt9x1ZO8goyQDA08mTPsF96Bfcj34h/egR2KNyaB/UvaUkbTbyYmPJfvc97MXFlZ/rvL1xCg3FqU0bDGGhOIWG4RTaRvssNBRDcDDCqeF5+VKLld0pBWxPzmN7Uj47kvPILdE6hD1dDPQJ96VfhC/9Iv3oG+GHj1s9O0jz8siaP5+Cb1ZiCAulzYszzml0Tfai98lZtIh2i97D69pr63hQCxz+BXZ9CYf+o80nFNwd+tyjvdzPrDSqGxfvqdPxf5FGPHU6An/4lH9lrOS3pN9w0btwW6fbGN99PG082tT5Wi5k6rmAlkcF/Hqy2CzE58azLXNbZZqm0KI9VRvkFkS/kH6VAb6jb0f0tfQX1JYLNcfHk/7Sy5j37sVjyBD87r4L54gIDKGh6D2bdgRRTaSUJJtK2ZaUx/bkPLYl5XMwoxC71BrGnYK96BfpR3/Hyxjgfk59EKVxcaTPnInlyFG8hl1LyAsv4BQaetafKdu7j8S77sL7+utoO3du/S6s1AT7voWdX0JqHOhdoPvNEPOANuTTcQ01tXCjnZ352hiFf79+RC7/lISyFD7Z8wk/HvsRBNzU/iYe6PEARh9j/c7vAqFa+C2PCvh1VGwpZmf2Ti09k7WdvTl7KbeVA2D0NtI/pD/9QvrRN7gv7TzbNdp88PbSUrLfW4Tps8/Q+/kR8vxzeF9//QW7+EpxuZVdx/PZnpTHtuQ8tiflUehYUczfw5l+EVrwjzH60bOtD65OZ68IpcVC7qfLyVm8GHQ6gh59FP97x1b7kJPdYiHxttuwFRTS/vs1pzxgVW+Z+7SRXbu+AkuRNqIr5gHodQexK384JYd/grOzM98++RTtv/0Wv3vuoc1LMwAtxffpvk/59vC3WO1WRncazZQ+U/B3rSHldIFTOfyWp8kDvhBiBPAOoAeWSinfOG37eOAt4MRUkYuklEtr229TB/zs0mwtNZO5g+1Z2zmUdwi7tKMXerr6d61swfcN6dtk/2GL1q8nY9YsrGnp+N5xB8FP/rNxglgzstslR7OLiUvSgv+2pDyO5WgPtDnrdfRs50OM0Y+YSH9iImvuDLakpJAxezYl/9uAS5cuhL4yE7fevU8pkzV/Prn/Wkr4v5bgefnljXsh5cWwdyXEfaxN+ubkDj1HszY7jHufnU9ubi5w6gphmW/OxbRsGWFvvoHPqFGVu8opy+HDXR/yzaFvcDO48VDPhxjbbewpab6WQo3SaVmaNOALIfTAIWAY2gLmW4G7pZTxVcqMR1u4fOq57LsxA76UksTCRG30jCNFc7zoOABuBjd6BfWqTM/0CuyFu9O5dSKeq4qsLDJfczyYFN2B0Fdewb1//yY9Zm0a8z92bnE525PziUs0sTXRxJ7UAiocM1VGB3sSE+lHjFGrACKrpIGklBT98iuZc+Zgzc7G9847CH7iCfQ+PtoKVmPH4nvbbYTOntVo112t1G0Q9wnsWanN9hnWFwY8BD1Gg9PJdQ6k1UryAw9StmsXxi+/OGP+oGP5x3h729usT1lPmEcY0/pN47qo6y7Yuzel5WvqgH8JMFNKOdzx/fMAUsrXq5QZTzMHfKvdykHTQbZlbmN71nZ2ZO2onHPGz8WvMjXTP6Q/nf0746RrngeR7OXlFHz3HVnzFyDLywmcMpmABx5ANNJcNvXV1Lfu5gobu1MKiEsyEZeo3QUUlGlpkkBPFwYYtTTQAKM/3cK80ZWVkfPeu5hWfI7ez4+gx6eRu3QpVFiJWrMGvadHg8+pTsryYffXWvDPPqCN8Bk8SUv5uPkBYM3NJeHW2xBOTkR982+++vHHMyrO6KHRzI+bz37Tfga1GcSLg19s8fl95cLU1AF/NDBCSvmQ4/t7gUFVg7sj4L8OZKPdDTwhpTxew/4mAhMBIiIi+lfXYVSbVUdW8drm1ypni2zn2e6U9EyUd1Sjt7DsFgvWrGysWVlYsx3vJ17Z2Vizs6jIysbuGE7pPngwoTNfxtlobNTzqK/m7pyz2yVHsovZmmhiW2IeW5NMlYvHuDnp6RPuS4zRj8G2HEKWLqRi316AWhcjbzJSwrH18Ne7cPS/4OQB/cfB4CngG07Zrl0kjb2X/Ihwhq1fT3E1Fefd99zNN4e+YeH2hZitZib0nMCDPR/EWX9+K3vl4tLUAf92YPhpAX+glPLRKmUCgGIpZbkQYhJwh5RyaG37rm8Lf1f2Ln489qPWig/qS4hH/ae1lRaLI2BnU5GVVX1Qz87Gll/NsoQGA4agIAzBQRiCgnAKDsYQHIxLp054Xn31BXVbfyEMv8ssNBOXmFd5FxCfXojNLjFgZ2z+XtqF+RM86iYGGv1p41PN8pHNJWMP/PUe7PlG+77HbXDZY+Stjydj5kw+yMnhvdycU36kasWZU5bDW1vf4qeEnzB6G5kxeAYDQwc280UoF6vzntI5rbweMEkpa+2ZbMpOW1lRgTUnB2tWlhbIKwP4qQHdlnfmHPTo9VogDwrCEByMIdgRzCu/177W+/k1yvQBzeFCHH5XUm5l5/H8ykpgW1IepRZt2upwfzcGRPozIMqfAUZ/OgR5NH8Fmn8cNn0A25eDpRjZ/mqWvbqVS3TePJKSwrqSk89RVFdx/pX6F7M3zSalOIWRHUbyVMxT+Ln6Ne81KBedpg74BrQ0zTVoo3C2AvdIKfdVKRMqpUx3fH0L8KyUcnBt+65vwLcVFGBJSqoM2hVVUyuOgG4zmc78Qb0eQ2DgKYH8xNdOVQO5v3+LCeR11RKG31ltduLTC9mamMfWBBNxSabKWUIDPJyJMWp9AAOj/OkW6o1B30z/RmV5Wo5/80fYCzLZvzYQS7Ge2xISSbBo/RQBAQF4enqe0SFutppZsnsJy/Ytw8PJgyf7P8nN0TdfUHd/SsvSHMMyrwcWog3L/ERKOUcIMQuIk1KuEUK8DowErIAJmCylPFDbfusb8E1ffEHmrNknP9DpMAQEnGx9V7bGgyq/dgoO1gK5/vxOtnY+tbThd1JKEnJK2JpoYktCHlsTTSSbtArLw1lPv0g/Bhq1u4A+4b61Pg/QYNZyNi15Ap/tX6LfEoRNSP62FzIvsZijFisWi6Wy6OmV6ZG8I8zeNJvtWdvpH9Kflwa/RHvf9k17vspFqdU9eGVJSaH80OGTrfSAgFYdyFuTjAIzWxJNbE0wsSXBxMHMIkB7HqBXOx8GRml3ADFGfzxdmmZhm9jPV3Bg0QvcIQ0YClwAQbaunNisAn4oKCLNqrX6T0+X2aWdVUdWMT9uPqXWUh7s8SATe01UnbrKOWl1AV9RTsgvtRCXmMeWRK0C2JNagM0u0QnoHnayAhho9G/02UGRkoot37J75iMEFHpQlqvtf6+5jJ+LivilqJjjlvIzfiy3LJd5cfP44dgPdPXvyptXvEmUT9MuuqJcPFTAVxSHknIrO5Lz2ZKQy+YEEzuO52Oxap2pnUO8KiuAQVH+BHs3zkggozGSDrpUXhnkThe7F7mJrtgLtODv2r07XsOH4z1iOM4REaf83Lrkdbz010uU28p5fuDzKrev1IkK+IpSg3Kr9kDYlgQTmxNMbEs0UeIYCRQV6MFAoz+D2muVQDu/+j19XbVDfGiUnjlDXejn60J2WiCWwgjMR7QZR1y6dcX7H47g73g+I7Mkkxc2vsCWjC2MMI5gxiUz8Hb2bpRrVy5OKuArSh1ZbXb2pRU6KoBctvMbdWwAACAASURBVCSYKieGa+vrxqAorQIYFBVwypQQtTm1QzycT5+/i6vsGyFzLxWuXSgUV1K0PZmyXbsAcOncGd8778DvrruwI1m2bxmLdiwixD2EN694kz7BfZrsd6C0bCrgK0o92e2Sg5lFbD6mpYC2JJgq1wcI8XZhUFQAA6P8Gdzenw5BnueWcrHbtSma170GpqPQtj8VPR+h6EAxBT/9hHn3bryGDSPszTfQubuzK3sXz254loySDCb3nsxDPR+qdUpupfVRAb8VamlDLFsKKbWZQTcd01JAm4/lklWkdbwGejo78v8BDGrvT6dgL3R1WSHMZoVdX8D6N6EwBSKHIIe+iOm/B8l66y1cunQm/P33cQoNpchSxOxNs1mbsJaYkBhev/z1i2axFaVxqIDfyrSEh6guFieWiTxxB7D5WC5pBWYAfN2dHH0AAQxu70/XNt5nrwCs5bDtU9jwFpRkQ8fhFHuOJHXWQoS7G+GLFuHWuzdSStYcXcOczXNw1jvzyqWvcE3ENc1zwcoFTwX8VuZCnCahNTluKq0M/psTTj4M5u1qcKR/AhjcPoCuod7VrxFsKWHHR5Npn7YGTyc7q3a7E50RiUupmdDXXsPnxhsASCxI5JkNz7DftJ87O9/JUzFP4Wo4j3MMKRcEFfBbmQthIjTlpLT8MjYn5LLpqIlNCbkk5WoVgJeroXIU0OD2AZXTQZy4Q3OVZcy4wplHBjhTXKIjbWdHdGnFBE6ZTODUqQidjgpbBe9sf4fl8cuJ9o1m7hVz6ejX8TxfsXI+qYDfyqgW/oUto8CsVQDHctl8zFS5Opini4EBRj/Wfb2E9N1/YMk4AtJOR38db17rws2dnEjf3ZaCgxKvfwwj7A2tMxfgz9Q/mb5xOsUVxTwV8xR3dr6zUcfsV2Rm4hRS/1lnleajAn4ro3L4LUtmoZnNCSZHBZDL0WytArCXl1KeGo85eQ/m5D1c6prAf58eiOl/x8ja6YNrdCTtln6KUxut0zanLIcX/3yRP1P/5Orwq5l16Sx8XX0bdG62ggIyZs2m8McfCXnhBfzvu7fB16s0LRXwWyE1SqflMnbpRbbwxTW8By4RPXEOdDyBW2Hmqm5tuc9jE33/XEzWOoHOzYV2b8/FbcgIQJuPZ0X8ChZuX4i/qz9vXP4GA9oMqNd5lGzaRNpzz2PNzsalfRTliUkYv/gCt549GutSlSagAr6itCCn36Hp3H3xju7HP8Y8gskQwOGsYtww80zZSoZsjMNm1uE+uj+hzyzCyUObTz8+N55nNjxDcmEyD/V8iMl9Jtd5GU97eTnZby/E9OmnOBuNhL01F+fwcI7deitCbyDq25Xovbya7PqVhlEBX1FamLPdoWUXlbPFkQJK3bODKT8vwCPXjEf3ctZffielve5jYHQbOgQ7MW/bm6w6sopeQb1464q3CPMMO+txzQcPkvb0M5QfOoTv3XcR8vTTlf0EpTt2kHTvfXhdcw1tF76t5vW5QKmArygXsWxTESlPP4brn5vwCi/DPMCV1xjDJkMMMcYAAkL28WfeRzjrDcy9ci6Xhl16xj6k3Y5p2adkL1yIzseHsDmv4nnllWeUy/34Y7Lemkebl1/C7+67m+PylHOkAr6iXOSklJg++YSsefNxCYTwS9I5HtyXudzLzznBCKccPMI/RzhnMthvLJP6PESvdr446XVUpKWR9tzzlG7Zgue11xA6axYGf//qj2O3c3zyZEr/3oTx669w7dq1ma9UqY0K+IrSShStW0fak0+hcxa0uzwXN488zD3u5s+Ih/k9zcqvWe9hdtlORUEfdLl3cF/JMa7/fQUGJCEvPI//6NtqTdVY8/JIuPkWdK6uGFeuRO/p0UxXp9TF2QJ+oyz6KYQYIYQ4KIQ4IoR4rprtLkKIrx3bNwshjI1xXEVRTuV19dVEfvklwsOXpF+8KXQdhWv8v7nm1+t4zW8dW8Z/wIPdp+DrvINn98xm1A8fcsgtiAeGTOPyPZ6MW7aVxeuPsD05jwpb9Q/pGfz8aDt/Hpbjx8mYObPah/yUC1NjLGKuR1vEfBiQgraI+d1SyvgqZaYAvaSUk4QQdwG3SCnvrG3fqoWvKPVjzc0l5dHHKNu+ncAHxxAYvh9x4AfwjaCkzf0kLPo30pTP2qu9GfL0B+QXhbLpmPYw2KHMYkBbFzjGeGIqCH96tPXBqcrC8Dkffkj2wncIfXU2vqNHn69LVU7TpCkdIcQlwEwp5XDH988DSClfr1LmP44yfwshDEAGECRrObgK+IpSf3aLhYyXXqZg1Sq8rhuBR3t/Sn/7hsKDFpz9nbDMmMpjJf8m35zPq0NeZbhxOAA5xSdHAVWtANwrKwBtRtCeYV5kPPwwpdu3Y/y/r3Ht1Ol8Xq7i0NQBfzQwQkr5kOP7e4FBUsqpVcrsdZRJcXx/1FEmp5r9TQQmAkRERPSvbooARVHqpmpnLlKi8/LC55JogoP/RGcrIidmHE/IDHbm7mVK7ylM6j3pjBx+TnE5m49pC8JsPnZyYXh3Zz1XBOqY9PlMnHx96PDNv3H19jwPV6lU1dQB/3Zg+GkBf6CU8tEqZfY5ylQN+AOllLln27dq4StK47AcP47Q6zGEhmoBvSQX1r0KccuwuPvzSueBrMnfx3DjcGZfNhs3g1uN+8qtcgewOcGEy57tvPbnEtYbB7Dp9ikMcswI2qudL86GRukmVM7B2QK+oRH2nwKEV/m+HZBWQ5kUR0rHBzA1wrEVRakD5/DwUz/wCIAb34b+43Fe+yyv7lhLx3adWZD4C8mFybw79N0aF1YJ8HThup6hXNczFABTyWAOvlHG0H9/RnpcN+Yd1qZecHXSEROpLQg/uEMAvdr54GJQK3SdT43RwjegddpeA6SiddreI6XcV6XMI0DPKp22t0op76ht36qFrygNV+u8SlLC3pXwyww2WE080yYUNxcf3rnmPXoF9arTMaTNRvL9D1C2Zw8BK75kGz6VdwD70wsBrQLoF+FXuR5A73BVATSFJh+HL4S4HlgI6IFPpJRzhBCzgDgp5RohhCuwAuiL1rK/S0p5rLb9qoCvKA1T3cypoK2Z8PDDD7N48eKTH5YXwx/zORL3IVOD/Mh2cuaVS2ZyY8eb63SsiswsEm65BUNgIMb/+xqdq7YYS16JhS2JpsrpoPdnFCIluBiqVgD+9A73xdVJVQANpR68UpRWqqa1EU6YPHnyqUEfIPcoeT8/yz+LdhHn5spDba/h0WsWoBO15+OL/9jI8QkT8L3jDkJnvVJtmfxSC1sSTJVTQsenaxWAs0FHvwhfBrcPYFBUAH0jVAVQHyrgK0orVdPqZyfo9XqsVmu12yoOrmXOhudY6WznauHJG/9YgnubnrUeM2vB2+QuWULY/Hn43HBDreULSivYkqgtCbkpIZd9aScrgD7hvgyO0tYF7hfhh5uzqgBqowK+orRStbXwgbNWCLKinC9+eZS52X/R1VLB++1uImDoDHD1qflnrFaS7htH+YEDRH27Emej8ZzOuaCsgq1VRgHtSyvALsFJL+jdzpdBjucA+kX64enSGONOLi4q4CtKK1VTDv+Es7Xwq1p/aBVP//0ygRUWPiyoIPLK6dDvPtBV3+KuSE8n4eZbMISFYfzqS3QuLvW+hkJzBdsS89iUkMumYyb2phZgs0v0OkGPMG8GRmkVwACjPz7udZvz/2KmAr6itGKxsbE8/PDDlJSUnLGt2hx+DXZn72bqr5PAUsKi9DR6+XaGEa9D1OXVli9at46UyVPwu+ce2rw0o0HXUFVJuZXtyXmV/QA7j+djsdoRAjqHeDHIkQIaYPQnyKv+FU1LpQK+0qKp5Robx5QpU1iyZAk2mw29Xs/EiRPrHOxPSCpMYtKvk8gpyWReoYUrc45D15Hwj9ngZzyjfOabczEtW0bbd97Be/g/GulKTmWusLHreD5bEkxsSTSxLSmPUosNgPZBHgyK8q+8CwjzrfmBsouFCvhKi6UWZL/w5JTlMPX3qew37edF/4HcvusHsNvgkkfg8n+Cy8nlD6XFQuLYe7EkJGj5/NMfAGsCFTY7e1MLtArAUQkUmbW0VTs/N0fw92dgVADGAPeLbuUuFfCVFqumTsfIyEgSExOb/4QUAEorSnnqf0/xR+ofPNz5Hh5JPYbY83/gGQLXvAy97wadNozTkpJKwi234Gw0Yoz9HOHs3KznarNLDmQUnqwAEkzkllgACPZyOaUC6BjsiU7XsisAFfCVFqumYYVCCOz26udrV5qH1W5l9qbZfHv4W0Z1GMXL4Tfg9MuLkBoHYX1hxJsQMQiAwl9/JfXRx/AfN46Q589YMqNZSSk5ml3C5oRcrR/gmImMQjMAfu5ODDCeTAF1C/NG38IqgKaeS0dRmkxERES1LfyIiIjzcDZKVQadgZmXzKSNexsW71pMTlkOC+5bjfuBH+G3l+GTf0CP0TDsFbyHDaN07FhMy5fjPmggXkOHnrfzFkIQHexJdLAnYwZFIqUkJa+MzQnaswBbEk38Ep8JgKeLgRijX+VdQM+2LXtCONXCVy5oKoffMnx7+Ftm/T2LTn6dWHztYgL1brBxIfz1LiDgsmnYB0wm6b4HsaSm0v67b3EKCzvfp12jjAJz5cNgWxJMHM7S1gQ4MR/QibuAvhG+uDtfWO1mldJRWjQ1Sqdl2JCygaf+9xT+rv58cO0HRPlEQX4y/PoS7PsOvNti6TGNhOf/hUvHjkSu+Azh1DLGzecWl7M18cRQ0Fz2pxdil2DQCbq39WFApB8DovwZYPTH36N5+yhOpwK+oijNYm/OXh75/RHs0s57Q9+jT3AfbUPSX/Dzc5C+i8KSnqR+n0vAhIcIfvLJ83vC9VRormB7Uh5bE01sTchjZ4r2LABAdLAnA4zaXcAAoz/t/NyadSSQCviKojSb44XHmfTbJDJLM5l7xVyGRjjy9XYb7PwCfn+F9PUW8o96EL7wdTxH1G02zguZucLGHsdQ0LhEE3FJeZVDQUO8XYgx+jMg0o8Yoz9dQ5u2I1gFfEVRmpXJbGLq71PZl7uPFwa+wJ1d7jy50VyI/fc3SZz9f1jLdETNugen658FJ9fzd8KNzGaXHMosIi7RxJbEPOISTaQXaCOBPF0M9I3wJSbSnwFGP/o0cj/ARRXwKyoqSElJwWw2n6ezUlo7V1dX2rVrh1MLyT+fL6UVpTyz4Rn+l/I/JvScwKN9Hz0ltVG+YwMJ903G1beMyFvcEcNfhW6j4CJ7EOqE1PwyrfWfqKWCDmYWISWVcwL1d1QA/Y1+BHvVv/K7qAJ+QkICXl5eBAQEXHRPyCkXPiklubm5FBUVERUVdb5P54JntVt5ddOrrDy8kpEdRjLzkpk46U9WlAVr1pD2zLMEDHAluMMxiByizc8TWreVtlqygrIKtifnsc1RAew8nk+5ox/grdG9uD2mfk8lX1Tj8M1mM0ajUQV75bwQQhAQEEB2dvb5PpUWwaAz8PIlL9PGow3v73yf7NJs3r76bTycPADwGTmSkk2byf3uOzyGTsMjawV8dIU2E+fQGeAZdJ6voOn4uDlxdedgru4cDIDFamdvWgHbEvMYGOXfJMds0BMEQgh/IcSvQojDjne/GsrZhBA7Ha81DTmmY38N3YWi1Jv6+zs3Qggm9Z7ErEtnsSVjC/f/fD/ZpScrzDYvTse5Q3tSP16H9Z5fYfBk2BkL7/WDP9+BitaRvnV2LPk44Yr2RAZ4NMkxGvrI2HPA71LKjsDvju+rUyal7ON4jWzgMRVFaYFu6XgL7w19j8TCRMb+NJZjBdqy1jp3d9q9/Tb24mJSZ8xBDnsVJv8NEYO1Mfzv9YedX2qjfJQGaWjAHwUsd3y9HGj546vqQK/X06dPH3r06MFNN91Efn4+AImJibi5udGnTx969+7NpZdeysGDBwFYv349Pj4+9OnThz59+nDttdc26TmuX7+eG2+8scFlWqLy8nLuvPNOoqOjGTRoUI2TrP3888907tyZ6Oho3njjjcrPFy1aRHR0NEIIcnJymumsW4fL213OsuHLMNvM3Lf2PnZk7QDApWNH2sx4kdJNm8j56CMI6gRj/g33rQGPQFg1SUv1HP4NLuB+xwtdQwN+iJQyHcDxHlxDOVchRJwQYpMQ4qyVghBioqNs3IWaJ3Vzc2Pnzp3s3bsXf39/3n///cptHTp0YOfOnezatYtx48bx2muvVW67/PLL2blzJzt37uS33347H6feKnz88cf4+flx5MgRnnjiCZ599tkzythsNh555BHWrl1LfHw8X375JfHx8QBcdtll/Pbbb0RGRjb3qbcK3QO78/n1n+Pr4suEXybwe9LvAPjceiveI28iZ9H7lGzZohVufyVMWAe3fQzlRRB7G3wyAg79ogJ/PdTaaSuE+A1oU82m6edwnAgpZZoQoj3wXyHEHinl0eoKSimXAEtAG6Vztp2+8v0+4tMKz+E0atctzJuXb+pe5/KXXHIJu3fvrnZbYWEhfn7VdmtUa+bMmSQkJJCens6hQ4dYsGABmzZtYu3atbRt25bvv/8eJycnfv/9d5566imsVisDBgzggw8+wMXFhZ9//pnHH3+cwMBA+vXrV7nfkpISHn30Ufbs2YPVamXmzJmMGjWqzudV1dGjRxkzZgw2m43rrruOBQsWUFxcTHFxMaNGjSIvL4+KigpeffVVRo0aRWJiIiNGjGDIkCFs2rSJ3r17c//99/Pyyy+TlZVFbGwsAwcOrPO1z5o1i++//56ysjIuvfRSPvroozNy6qtXr2bmzJkAjB49mqlTpyKlPKXcli1biI6Opn379gDcddddrF69mm7dutG3b996/W6Uugv3CmfFdSuY+vtUnlj/BM8NfI57ut5Dm5dexrx7D2lPPkXU6lUY/P21aZZ7jtYWWtm+XMvrf3E7hPSEIY9D91tqXGpROVWtLXwp5bVSyh7VvFYDmUKIUADHe1YN+0hzvB8D1gMXxf8om83G77//zsiRJ7sljh49Sp8+fejQoQMLFizgn//8Z+W2P/74ozKlM2fOnGr3efToUX788UdWr17N2LFjufrqq9mzZw9ubm78+OOPmM1mxo8fz9dff10ZwD/44APMZjMTJkzg+++/548//iAjI6Nyn3PmzGHo0KFs3bqVdevW8fTTT1e73F1dTJs2jWnTprF161bCqkx+5erqynfffcf27dtZt24dTz75ZOW0xkeOHGHatGns3r2bAwcO8MUXX7Bx40bmzZt3yh1QbdcOMHXqVLZu3crevXspKyvjhx9+OOMcU1NTCXcstGEwGPDx8SE3N7fGMgDt2rUjNTW1Xr8TpX78XP1YOnwpV4ZfyetbXuftbW8jPNxou/BtbAUFpD3zLLLqFNgGZxg4AR7bATd/ALZyWPkgLIqBbZ+Ctfy8XUtL0dBhmWuAccAbjvfVpxdwjNwplVKWCyECgcuAuQ08LsA5tcQbU1lZGX369CExMZH+/fszbNiwym0nUjoAX3/9NRMnTuTnn38GtJROdQGqquuuuw4nJyd69uyJzWZjxIgRAPTs2ZPExEQOHjxIVFQUnTp1AmDcuHG8//77XHXVVURFRdGxY0cAxo4dy5IlSwD45ZdfWLNmDfPmzQO0oa3Jycn1uva///6bVatWAXDPPffw1FNPAdr49BdeeIENGzag0+lITU0lM1ObYjYqKoqePXsC0L17d6655hqEEJXXVNdrB1i3bh1z586ltLQUk8lE9+7duemmm045x5rmzz/XMkrTczO48fZVb/P65tf5ZO8nZJZmMvvS2YS88DwZM18hZ9EiAh899YEt9E7Q5x7odRcc+AE2LoDvp8H6N+CSqdB/PLh4nrdrupA1NIf/BjBMCHEYGOb4HiFEjBBiqaNMVyBOCLELWAe8IaWMb+Bxz6sTOfykpCQsFsspOfyqRo4cyYYNG85p3y4u2qLLOp0OJyenyj90nU6H1WqtNlCdUFPAklKycuXKyv6D5ORkunbtWuN+7r//fvr06cP1119f5/OOjY0lOzubbdu2sXPnTkJCQiqfhj5xTSeuo+o1Wq3Wym21XbvZbGbKlCl888037NmzhwkTJlT7xHW7du04fvw4AFarlYKCAvz9/WssA5CSknLKHYvSfAw6Ay8OfpHH+j7Gj8d+ZPLvkzHccgM+o0aSs/gDUh+bhq2wmtStTgfdRmo5/nu/g4Bo+GU6LOyhBf9SU/NfTCNoyodhGxTwpZS5UsprpJQdHe8mx+dxUsqHHF//JaXsKaXs7Xj/uDFO/ELg4+PDu+++y7x586ioqDhj+8aNG+nQoUOjHrNLly4kJiZy5MgRAFasWMGVV15Jly5dSEhI4OhRrWvkyy+/rPyZ4cOH895771X+Ie3YseOsx1i2bBk7d+7kp59+OmPb4MGDWblyJQBfffVV5ecFBQUEBwfj5OTEunXrql20pKFOBPfAwECKi4v55ptvqi03cuRIli/XBo998803DB069IzKcMCAARw+fJiEhAQsFgtfffXVKak5pXkJIZjQawKvXvYq2zK2Mf4/49HPeILgZ56haN06Em6+hTLHnXM1PwwdhsL4H+DB3yDiElj/OrzdA/4zHQrTmvdiGsBqMpFw622UbN7SJPtvuUu3XCD69u1L7969K4PfiRx+7969eeGFF1i6dGktezg3rq6uLFu2jNtvv52ePXui0+mYNGkSrq6uLFmyhBtuuIEhQ4acMsJkxowZVFRU0KtXL3r06MGMGTPqffyFCxeyYMECBg4cSHp6Oj4+PgCMGTOGuLg4YmJiiI2NpUuXLg2+1tP5+voyYcIEevbsyc0338yAAQOqLffggw+Sm5tLdHQ0CxYsqBxymZaWVnnXYjAYWLRoEcOHD6dr167ccccddO+upQjfffdd2rVrR0pKCr169eKhhx5q9GtRqjcqehTvX/M+KUUpjP35XvJvvQJj7OcgBIlj7yX7/ferb+2fED4A7v5SG8ff9UbY9AG80xvWPAa51Y4TuWBIm43UJ5/EcvQoeq+mSUm1uLl09u/ff9Z0hNK0SktLcXPT5vf+6quv+PLLL1m9+oyum4ue+jtsWvG58Uz5bQoWm4W5V87lEq9epM94iaL//Aedjw+BEyfiN3YMuirpwmrlJcKf78KOz8FeoY3oGfIEtOnZLNdxLrIWLiT3w48InfMqvrfdVu/9nG0uHdXCV87Jtm3b6NOnD7169WLx4sXMnz//fJ+SchHqFtCNL274gjDPMB75/RE+P/4dbRe+jXHlN7j17kXWW29xdMR15H/7HdJ2lidw/Yxw4wJ4fA9c+qg2fv/DIRB7ByRvarbrqU3RunXkfvgRPqNva1Cwr41q4StKPai/w+ZRWlHKi3++yK9JvzKqwyheuuQlnPXOlGzaTNb8+Zj37MGlYzRB//wnnlddVftIq7I82LIUNn8ApbkQcSlc/iREX3PepmW2pKSQcOttOLVri/GLL9C5NmxdANXCVxSlRYmNjcVoNOLp4sm393/LEN0QVh9dzf3/uZ/U4lQ8Bg/C+H9f03bhQqSlgpTJU0i6915KaxmQgJsfXPm01uIf8SbkJ2lP7350Bez9ttnn67GXl5Py2GMAtHvnnQYH+9qogK8oygUlNjaWiRMnkpSUhJSSpMQkPpv0GbfobuFw3mFu+PYGnt3wLPtN+/EeMZz2P3xPm5kvY0lMIunue0h59FHKjx07+0GcPWDwJHhsJ4x6HyrK4Jv7YdEA2P4ZWC3Ncq2Zr75Kefx+wt54A+fw+s1/fy5USkdR6kH9HTYdo9FY47BeJ38nAoYF4HeVH3o3PYPaDGJc93EMaTsEWVpK7vLlmJZ+jL28HN/bbiPwkUdwCqlpiq8q7DbY/732EFf6LvAK03L+/cdplUMTyF+5kvTpLxLw8MMEP/F4o+33olrxSv1HUy4E6u+w6eh0ulofPtK56fC/yp+IWyKwOFuI9o1mXPdxXB91Pbr8InI+/Ii8r75C6PX4jxtHwEMPovfyqv3gUsLR/8IfCyBpI7j5a/PzD5ygpYMaiTk+nsS778GtX18ili5F6BtvLiCVw29kanrkC1tdp0d+4IEHCA4OpkePHs17gspZRURE1FrGXmYnZ20Ou6buYs6QOQghmPHnDEasHMGnad/h/vRUOvz0I17XXkvuRx9x9Nph5H76KXZLLakaIbQO3Pt/hAd+gfCBsG6O9hDXLy9CUcbZf74ObIWFpEx7HL2vL23nzWvUYF8bFfDrQU2PfGGry/TIAOPHj6+c50i5cMyZMwd3d/c6lbVZbIzsMJKVN63ko2s/Ito3mne2v8Owfw/j7fQvEDOfwLjyG1x79CDrjTc5NuI6ClavPvtQzhMiBsE9X8Pkv6DzdfD3+7CwJ3z/OOQcrte1SbudtGefoyI9nbYL38YQEFCv/dRXi1vT9hRrn4OMPY27zzY94bo3ai/noKZHbpnTIwNcccUVNbb+lfNnzJgxAEyfPp3k5GR0Oh22GgK03tE6FkJwadtLubTtpRwwHWD5vuV8deArvjzwJf8w/oPxc/9J5MEHyJo3n7RnnyP3k2UEP/UkHkOG1D6UM6Q73LYUrn5Be4hrZyxsW6ZN4dD3Xuh+c53z/LlLP6Z43TpCpk/H/TxMw61a+A2gpkduudMjKxe2MWPGkJiYiN1uZ/ny5eh01YeqiRMnnvFZF/8uvH7566y9bS33druXDSkbuPOHO3ms6GNSFj5G2Lx52EtLOT5hIsnj76dsTx0bjf7t4aaF8MQ+uPYVKMmG1VNgXmdt6oaUuLMuylKyaRPZCxfiff31+I0dU7djNrKW3cI/h5Z4Y1LTI7f86ZGVluNEi//hhx+ubKjodDoefvhhFi9eXOPPtfFow5MxTzKx10RWHlrJiv0rmLJuKtG+0dz3zgSGxJWS9+ESEm+/A68RIwh+fBrORmPtJ+QZrC28ctk07WndHStgz7+1xVkCO0HPO7QFW/yjKn+kiS3zjQAAD21JREFUIjOT1H8+iXNUFKGzZ53179FsNeOsd0YnGr89rlr49aCmRz5TS5seWWlZxowZQ3FxMVJKpJTYbLazBvuqvJy9GN9jPD/f+jOvDXkNndDx0tZZ3Om6nD8XjsFz0oMUb9jA0RtvIv2VV7DWdWlVISDyErh5MTx5EG56BzyCYN2r8G4fWDoMtvwLmZ9O6rTHsZvNtHv3HXQe1ad/jhcdZ/7/t3f/wVHXdx7Hn++EhDWBgCmo4JINAylS+RE5fhw/wkwrAYXi8aMUMNp6o0bQzpWpcneVQckx6Y3HyHAOk47xR4k0lYqAxcIBtpVRjhMChQgBQakgIVB+xZoYhEDe98euKcgu2WR/fHez78dMhuzyzXff37B558Pn8/2+vrueZ9yb49hesz3Yb02rWMMPgcUjx288skk8KckpTO4zmTcnv8mL+S+Sc3MOSw/+kund32DTc5NJmTqRz1e/ySfjJ3DmhRe4Ul8f/M5dGd4br/zzRu9VvHc/C5fqYeNT/PVHI7iwdy89f5RHx5uvbblN2sS2E9t44o9PMGntJFYeWMnw24bzLVdkFnPje0onBlwdj5yXl9c8h6+qpKamRjQe+etF2zlz5tCxY8fmeORu3boxZswY9u/fD3jjkefNm8egQYNQVbKzs1ucWgpk2bJlPPDAAzz//PNMmjTpmnjkyZMnM3ToUHJzcyMej5ydnX3DeOQHH3yQvn37kpmZ2fyLqaamhkceeaT5F9ns2bPZunUrZ8+exe12U1RUxMMPPxz2uk1sERFG9RzFqJ6jOHT+EGVVZZR9+jtW5Cg/eCaPaVsvcrbkl9Su+i3d5s7l5pk/RFJTg3+BrlmQ9zN05L9w9r8WUXt4DTcPTiXj81/BC7+CW77D33Lu5nfpafz21P/yWd1nZLoyeXTQo8z49gxuS/d3C/EwHbtdeGVaw+KRvex92L6c+vIUvzn4G1YfXk19Yz2TL/Zn5p8ukbr3ECluN93nzSNj4r1IgMXjb7pUfYKa+fO5sGcPXaZMocd/FCH11Rz6cCWvH/0fNmodF5KSyG1sYlbXAeQP+DGpfb5H+Rtrms9OysrKori4uHkNI1gRu9JWRGYAi/DexnC4qu4KsN09wH8DycDLqhrUaqs1/Njz/vvvN5/m2LVrV1599VX69u3rdFlRZ+/D9qnuUh1rP17LygMr+euXp7jndA/uf/cKrk9P0vE7/bnlySfpNHr0dV/3+OOPU1paypUrV5jYpQvF7l50SE5mSV0dvz5+lN4TepMzI4dqqnElu5iYdTezUm6j/2e74ZM/QuOXNEpHNhy6wNqqi/z+cCO1X0FaWhqlpaWtavqRbPj9gSbgReApfw1fRJKBw3jveVsNVACzg7mvrTV8E6vsfdi+NTY1sunTTZRVlXH4/Efc+3EnZr2nuM58QfqokWRMmkRSRgbJGV14bvlyXl71OpdUmdetO9O7duXDposU92qgtm8K6Xemk9IlhcYzjYy/dTyLf7iYLh27XPViX8HR9yl/5n6+2/MrenZO4sdvXeC1Su+6oMfjadX1IhHP0hGRrQRu+COBRao6wff45wCq+p8t7dcavolV9j5MDKrKByc/oKyqjB2fbWNSZQd+sB061l/0u30Tytp/FNaMTeZKstB4vpGGIw3Uvl9L/b56PFmBm3dSUhKo8g89k/j4XBN/872EiNDU1BR0zTdq+NFYtL0dOH7V42pgRKCNRaQQKITgMjWMMSZSRISRPUcysudIDp0/xGv9XuPR3A10rksm/StI/0pJb4C0uiZu+rKJfa7L7Kn9kgslF2g40sDlzy9fs78bXf+SlZXFsWPH2FXTdN3z4dJiwxeRPwD+lo0XqGowq3X+zocL+N8KVS0FSsE7wg9i/8YYE3H9MvtRPKaYJ4c+yRcXv6BTaifSU9Lp7OocMPrhm27UvIuLiyksLKShoaH5ubS0tIBX5bdFi0vOqjpOVQf4+Qj21Ixq4OpkfzdQ05ZijTHGaZmuTLK7ZNPtpm7c1OEmv/EO4I31uFpLzbugoIDS0lI8Hg8igsfjafWCbUuiceFVBZAjIr1FJBWYBayPwutGjMUjx7b33nuPIUOG0KFDh4AXZxkTLiUlJcydO7c5yC05OZm5c+eyYsWKVjfvqzOEjh49GtZmDyE2fBGZKiLVwEhgg4hs9j3fU0Q2AqjqZeAnwGbgIPCGqlaFVrazLB45tmVlZbFixQruv/9+p0sxCaKkpKQ5+uTy5cuUlJREvHm3RUiLtqq6Dljn5/kaYOJVjzcC11+nH6Lndj7HR+c/Cus+78i8g38b7j8/3R+LR469eORsXwBWoIRFYxKV/USEwOKRYzMe2RjjX1xn6bRmJB5OFo8c2/HIxhj/bITfBhaPfL1Yikc2xvhnDT8EFo8cm/HIxhj/rOGH6Op4ZPj7HP7gwYN5+umnIxqPPHDgQJKSkpgzZw4ul6s5HnnMmDF4PJ7mr1m4cCGNjY0MGjSIAQMGsHDhwja//rJly1i6dCnDhw/n5MmT18Qj79q1i6FDh1JeXh7xeOQpU6YEjEeuqKjA7XazevVqHnvsMe68886w12JMPLJ4ZNMqFo/sZe9DE6ucztIx7cju3buvi0c2xsQHa/imVfLy8qisrHS6DGNMG9gcvjHGJAhr+MYYkyCs4RtjTIKwhm+MMQnCGn4bWDxybLtRPHJZWRk5OTnk5ORQVlbmUIXGOMMafhtYPHJsCxSPfP78eYqKitixYwc7d+6kqKiI2tpah6o0Jvri+rTMU7/4BRcPhjceuWP/O7jt6aeD3t7ikeMnHnnz5s3k5+eTmZkJQH5+Pps2bWL27Nlt+l4YE29shB8Ci0eOr3jkEydO0KvX3++26Xa7OXHiRJu+D8bEo7ge4bdmJB5OFo8cn/HI/mJEAiWMGtMe2Qi/DSwe+XrxEI/sdrs5fvx48+Pq6upr/pdiTHsX6j1tZ4hIlYg0iYjfsB7fdkdFZJ+I7BWRXYG2izcWjxxf8cgTJkxgy5Yt1NbWUltby5YtW5gwYULY6zQmVoU6wt8PTAOCGcZ+V1VzA6W4xSuLR46feOTMzEwWLlzIsGHDGDZsGM8880zzAq4xiSAs8cgishV4SlX9jt5F5CgwVFXPtma/Fo8ceywe2cvehyZWxUI8sgJbRESBF1W1NNCGIlIIFIL3fGoTWywe2Zj41WLDF5E/ALf5+asFqhrs0G60qtaIyC3AOyLykar6nQby/TIoBe8IP8j9myixeGRj4leLDV9VQ84AUNUa35+nRWQdMJzg5v2NMcaEScRPyxSRdBHp/PXnwHi8i73GGGOiKNTTMqeKSDUwEtggIpt9z/cUka/P6bsV2CYilcBOYIOqbgrldY0xxrReSIu2qroOWOfn+Rpgou/zvwCDQ3kdY4wxobMrbUO0aNGi5sgCf9566y0OHDgQxYqMMca/dt/wy8vLyc7OJikpiezsbMrLy6P6+tbwjTGxol03/PLycgoLCzl27BiqyrFjxygsLAy56RcXF9OvXz/GjRvXfIOTl156iWHDhjF48GCmT59OQ0MD27dvZ/369cyfP5/c3FyOHDnidztjjImGdt3wFyxYcF1DbWhoYMGCBW3e5+7du1m1ahV79uxh7dq1VFRUADBt2jQqKiqorKykf//+vPLKK4waNYr77ruPJUuWsHfvXvr06eN3O2OMiYZ23fADRQC3NRoYvJn2U6dOJS0tjYyMjOYs/P3795OXl8fAgQMpLy+nqqrK79cHu50xJvY4PUUcqnbd8ANFM4Qa2eAvhvihhx5i+fLl7Nu3j2effTZgbG+w2xljYkukpoijqV03/OLiYtLS0q55Li0tLeDdpoIxduxY1q1bx4ULF6irq+Ptt98GoK6ujh49etDY2HjNG6Bz587U1dU1Pw60nTEmtkViijja2nXDLygooLS0FI/Hg4jg8XgoLS2loKCgzfscMmQIM2fOJDc3l+nTp5OXlwfA4sWLGTFiBPn5+ddEA8+aNYslS5Zw1113ceTIkYDbGWNiWySmiKMtLPHIkWLxyCZW2fsw8WRnZ/u9sY/H47nmVp1Ou1E8crse4RtjTLhEYoo42qzhG2NMECIxRRxt0boBSlipasAbdhsTabE8DWoiq6CgIK4a/DfF3Qjf5XJx7tw5+6EzjlBVzp07h8vlcroUY1ot7kb4breb6upqzpw543QpJkG5XC7cbrfTZRjTanHX8FNSUujdu7fTZRhjTNyJuykdY4wxbWMN3xhjEoQ1fGOMSRAxfaWtiJwBrr+07XrdgLMRLicW2XEnFjvuxNOWY/eoand/fxHTDT9YIrIr0KXE7Zkdd2Kx40484T52m9IxxpgEYQ3fGGMSRHtp+KVOF+AQO+7EYsedeMJ67O1iDt8YY0zL2ssI3xhjTAus4RtjTIKI+4YvIveIyCER+URE/t3peqJBRF4VkdMist/pWqJJRHqJyLsiclBEqkTkp07XFA0i4hKRnSJS6TvuIqdriiYRSRaRPSLye6driRYROSoi+0Rkr4jsavkrgtxvPM/hi0gycBjIB6qBCmC2qh5wtLAIE5GxQD3wmqoOcLqeaBGRHkAPVf2ziHQGdgNTEuDfW4B0Va0XkRRgG/BTVf3A4dKiQkR+BgwFMlT1+07XEw0ichQYqqphveAs3kf4w4FPVPUvqnoJWAX8k8M1RZyqvgecd7qOaFPVk6r6Z9/ndcBB4HZnq4o89ar3PUzxfcTvSK0VRMQNTAJedrqW9iDeG/7twPGrHleTAA3AgIhkA3cBO5ytJDp80xp7gdPAO6qaEMcNLAP+FWhyupAoU2CLiOwWkcJw7TTeG76/+xwmxMgnkYlIJ2ANME9Vv3C6nmhQ1Suqmgu4geEi0u6n8kTk+8BpVd3tdC0OGK2qQ4B7gSd807ghi/eGXw30uuqxG6hxqBYTBb457DVAuaqudbqeaFPVz4GtwD0OlxINo4H7fPPZq4DvicivnS0pOlS1xvfnaWAd3unrkMV7w68AckSkt4ikArOA9Q7XZCLEt3j5CnBQVZc6XU+0iEh3Eenq+/wmYBzwkbNVRZ6q/lxV3aqajfdn+0+q+oDDZUWciKT7TkpARNKB8UBYzsiL64avqpeBnwCb8S7gvaGqVc5WFXki8jrwf0A/EakWkYedrilKRgMP4h3p7fV9THS6qCjoAbwrIh/iHeS8o6oJc4piAroV2CYilcBOYIOqbgrHjuP6tExjjDHBi+sRvjHGmOBZwzfGmARhDd8YYxKENXxjjEkQ1vCNMSZBWMM3xpgEYQ3fGGMSxP8Df2WYwRewDggAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.kernel_ridge import KernelRidge\n", "from sklearn.model_selection import GridSearchCV\n", "from sklearn.gaussian_process import GaussianProcessRegressor\n", "from sklearn.gaussian_process.kernels import WhiteKernel, ExpSineSquared\n", "\n", "\n", "###############################################################################\n", "# Generate sample data\n", "X = np.sort(5 * np.random.rand(40, 1), axis=0)\n", "y = np.sin(X).ravel()\n", "\n", "###############################################################################\n", "# Add noise to targets\n", "y[::5] += 3 * (0.5 - np.random.rand(8))\n", "\n", "\n", "plt.scatter(X, y, c='k', label='data')\n", "\n", "for gamma in [0.01,0.1,1,10]:\n", " kr = KernelRidge(kernel='rbf', gamma=gamma)\n", " kr.fit(X, y)\n", " kr_pred = kr.predict(X)\n", " plt.plot(X, kr_pred, label='RBF model - gamma '+str(gamma))\n", " \n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you have notice, the performance is very different depending on the value of $\\gamma$. Tools like the [TPOP](https://github.com/EpistasisLab/tpot) may be used to select the right parameters." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Neural Networks\n", "\n", "Neural networks are modular models which pieces are connected according to some criteria. Pieces are called neurons and connections synapsis, due to the inspiration to the brain.\n", "\n", "The simplest neural network consists of only one neuron and is called a perceptron, as shown in the figure below:\n", "\n", "\n", "\n", "A perceptron has one input layer and one neuron. Input layer is responsible for receiving the inputs. The number of nodes in the input layer is equal to the number of features in the input dataset. Each input is multiplied with a weight (which is typically initialized with some random value) and the results are added together. \n", "\n", "The sum is then passed through an activation function. The activation function of a perceptron resembles the nucleus of human nervous system neuron. It processes the information and yields an output. In the case of a perceptron, this output is the final outcome. \n", "\n", "\n", "### Multilayer Perceptron\n", "\n", "In the case of multilayer perceptrons, the output from the neurons in the previous layer serves as the input to the neurons of the proceeding layer. Therefore, multilayer perceptrons, or more commonly referred to as MLPs, are a combination of multiple neurons connected in the form a network. An artificial neural network has an input layer, one or more hidden layers, and an output layer. This is shown in the image below:\n", "\n", "\n", "\n", "A neural network executes in two phases: Feed-Forward and Back Propagation. In the feed-forward phase, the input is presented to the input layer. The input signal is then propagated till the output layer. During the training phase, the output is compared with the correct value, and the weights of different neurons are updated in a way that the difference between the desired and predicted output is as small as possible.\n", "\n", "Let's try a neural net for predicting some properties of the molecules. MLP networks allow us to predict more than one output, so let's try." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
feature_0feature_1feature_2feature_3feature_4feature_5feature_6feature_7feature_8feature_9...homolumogapr2zpveU0UHGCv
O3NC4H5134.38304973.27743165.56522041.20533937.97136028.56217022.14847519.5414560.3461830.246778...-0.2528-0.00770.24511012.18640.095653-435.725372-435.718189-435.717245-435.75726626.019
FONC6H4135.76212285.19293461.37565438.78915036.30910926.51595423.00382220.93152518.6641590.345873...-0.2604-0.07890.18151188.64410.089534-460.774434-460.767413-460.766468-460.80637725.848
O2N2C5H4139.21411374.41726949.82891645.67417037.00023826.23330924.53920423.07953619.0392770.343824...-0.2324-0.04680.1856945.79880.092456-452.744414-452.738132-452.737188-452.77571623.374
O3C6H4137.46688773.56536461.51838346.47332532.47677128.00542623.11813820.29464519.7725880.349248...-0.2670-0.08230.1848944.04260.090391-456.613010-456.606412-456.605468-456.64439224.902
FON3C4H4146.43197883.54877159.49413250.05566143.92324231.13762023.79579122.15140019.0936720.331518...-0.2192-0.03090.18821074.30780.090835-494.100913-494.093709-494.092765-494.13247127.708
\n", "

5 rows × 29 columns

\n", "
" ], "text/plain": [ " feature_0 feature_1 feature_2 feature_3 feature_4 feature_5 \\\n", "O3NC4H5 134.383049 73.277431 65.565220 41.205339 37.971360 28.562170 \n", "FONC6H4 135.762122 85.192934 61.375654 38.789150 36.309109 26.515954 \n", "O2N2C5H4 139.214113 74.417269 49.828916 45.674170 37.000238 26.233309 \n", "O3C6H4 137.466887 73.565364 61.518383 46.473325 32.476771 28.005426 \n", "FON3C4H4 146.431978 83.548771 59.494132 50.055661 43.923242 31.137620 \n", "\n", " feature_6 feature_7 feature_8 feature_9 ... homo lumo \\\n", "O3NC4H5 22.148475 19.541456 0.346183 0.246778 ... -0.2528 -0.0077 \n", "FONC6H4 23.003822 20.931525 18.664159 0.345873 ... -0.2604 -0.0789 \n", "O2N2C5H4 24.539204 23.079536 19.039277 0.343824 ... -0.2324 -0.0468 \n", "O3C6H4 23.118138 20.294645 19.772588 0.349248 ... -0.2670 -0.0823 \n", "FON3C4H4 23.795791 22.151400 19.093672 0.331518 ... -0.2192 -0.0309 \n", "\n", " gap r2 zpve U0 U H \\\n", "O3NC4H5 0.2451 1012.1864 0.095653 -435.725372 -435.718189 -435.717245 \n", "FONC6H4 0.1815 1188.6441 0.089534 -460.774434 -460.767413 -460.766468 \n", "O2N2C5H4 0.1856 945.7988 0.092456 -452.744414 -452.738132 -452.737188 \n", "O3C6H4 0.1848 944.0426 0.090391 -456.613010 -456.606412 -456.605468 \n", "FON3C4H4 0.1882 1074.3078 0.090835 -494.100913 -494.093709 -494.092765 \n", "\n", " G Cv \n", "O3NC4H5 -435.757266 26.019 \n", "FONC6H4 -460.806377 25.848 \n", "O2N2C5H4 -452.775716 23.374 \n", "O3C6H4 -456.644392 24.902 \n", "FON3C4H4 -494.132471 27.708 \n", "\n", "[5 rows x 29 columns]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.head()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "## variables to be predicted\n", "outputs = ['homo','lumo','gap']\n", "\n", "molecules_X = data[feature_names]\n", "molecules_y = np.array(data[outputs])\n", "\n", "\n", "# Split the data into training/testing sets (we will discuss this latter on)\n", "molecules_X_train = molecules_X[:-2000]\n", "molecules_X_test = molecules_X[-2000:]\n", "\n", "# Split the targets into training/testing sets (we will discuss this latter on)\n", "molecules_y_train = molecules_y[:-2000]\n", "molecules_y_test = molecules_y[-2000:]" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "from sklearn.neural_network import MLPRegressor\n", "\n", "## two hidden layers, with 20 neuros each\n", "mlp = MLPRegressor(hidden_layer_sizes=(20,20),max_iter=1000)\n", "\n", "mlp.fit(molecules_X_train,molecules_y_train)\n", "\n", "molecules_y_pred = mlp.predict(molecules_X_test)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3MAAAJOCAYAAAAd2l69AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdf5hb1Xkv+u8rjWw0TmLZidvEwsa+lNgH18FTpkDrp21MkpocLmYKBEPgNulNy+3t4enFoXM7abjBJuQwrW9KTk95espJc5oGAoMhmdo1rfPDzjnnujHF7oxDh+ADIWBb5jRubLnBI2zNzHv/kLa8Z2ut/UPa0ujH9/M8CSNpS9qe0V5a71rvepeoKoiIiIiIiKi9JOb6BIiIiIiIiCg6BnNERERERERtiMEcERERERFRG2IwR0RERERE1IYYzBEREREREbUhBnNERERERERtiMEcxUZEXhORD871eRARmbCNIiKiTsNgjoiIiIiIqA0xmCMiIiIiImpDDOYobutE5HsickZERkTkIgAQkd8SkVdE5JSI7BSRpc4TRERF5HdE5GUR+YmIfFZELhWR74rIv4rIUyIyz3W89bWIiIKIyF+KyIOu2+8XkeOu26+JyGC5LTsrIn8hIj8tIn9bbqO+JSKLXMdvEpEJEcmLyHdE5N80+99ERJ1FRH5ORMbKbc6Ocp/qQRFZJCJ/IyInReR0+eeLXc/7jog8JCL/UO6L/bWILJ7Lfws1FoM5itutAK4DsBLA+wB8XESuBfBQ+bH3AHgdwJOe510H4EoA1wD4vwE8CuAOAMsA/CyA2wEg5GsREdXrZgAfAvBeADcA+FsAfwDgXSh9d/4uAIjIewE8AeAeAEsAPAtgl3sAiogoinL78XUAfwlgMUptzK+VH04A+C8ALgGwHEABwJ96XuLXAfzvAJYCmALwJw0/aZozDOYobn+iqidU9RSAXQDWoRSUfUlV/1FVzwH4FIBfEJEVruf9oar+q6pOAPgnAN9Q1VdV9QxKnai+8nFhXouIqF7/UVX/WVVzAP47gOdUdazc7nwdF9qkzQB2q+o3VbUI4P8FkAbwi3Ny1kTUCa4B0INSn6qoql8D8A8AoKo/VtVnVHVSVX8C4HMAfsXz/K+o6j+p6lkA/w+AW0Uk2cx/ADUPgzmK2/90/TwJ4G0ojQy97typqm8C+DGArOvYf3b9XDDcflv55zCvRURUr1rbpBkAx8A2iYhqtxRATlXVdd8xABCRXhH5cxF5XUT+FcB/A5DxBGvHXD+/DiCFUlYBdSAGc9QMJ1BKBwAAiMgCAO8EkJvj1yKi7nQWQK/r9rvreC1vmyQopYezTSKiWr0BIFtuTxzLyv+9F8AqAFer6jsA/HL5ftOxQCkVswjgXxp0rjTHGMxRM3wVwG+IyDoRmQ/g36OUsvTaHL8WEXWncQD/VkQWi8i7UVrvVqunAFwvIh8QkRRKHa1zAP4+hvMkou70XQDTAO4WkR4RuRHAVeXH3o5SdkC+XNjkfsPz7xSRy0WkF8ADAJ5W1elmnDg1H4M5ajhV/TZKOdvPoDTadCmA2+b6tYioa30FwGEArwH4BoCRWl9IVY8AuBPAf0Rp5PsGADeo6vn6T5OIulG5/bgJwCcA5FFqY/4GpYGiL6C0LvdfABwA8HeGl/gKSsVT/ieAi1Au2ESdSWan4xIRERERUSsRkecA/CdV/S8Bx30HwGOq+sWmnBjNOc7MERERERG1EBH5FRF5dznN8mMobfdkmoWjLtcz1ydARERERESzrEJpTe7bAPwAwC2q+sbcnhK1IqZZEhERERERtSGmWRIREREREbWhlk6zfNe73qUrVqyY69MgopgdOnToX1R1yVyfRz3YPhF1HrZNRNSK/Nqmlg7mVqxYgYMHD871aRBRzETk9bk+h3qxfSLqPGybiKgV+bVNTLMkIiIiIiJqQ7EFcyJynYgcEZFXRGTI8PgnReRFEfmeiHxbRC6J672JiIiIiIi6TSzBnIgkATwC4MMALgdwu4hc7jlsDEC/qr4PwNMA/iiO9yYiIiIiIupGcc3MXQXgFVV9VVXPA3gSwI3uA1R1n6pOlm8eAHBxTO9NRERERETUdeIK5rIAjrluHy/fZ/MJAH9rekBE7hKRgyJy8OTJkzGdHhERERERUWeJK5gTw33G3chF5E4A/QC2mx5X1UdVtV9V+5csaevqwERERERERA0T19YExwEsc92+GMAJ70Ei8kEAnwbwK6p6Lqb3JiIiIiIi6jpxzcw9D+AyEVkpIvMA3AZgp/sAEekD8OcANqnqj2J6XyIiIiIioq4USzCnqlMA7gawB8D3ATylqhMi8oCIbCofth3A2wDsEJFxEdlpeTkiIiIiIiIKEFeaJVT1WQDPeu77jOvnD8b1XtSeRsdy2L7nCE7kC1iaSWNw4yoM9GWt9xMRERFRd2B/sDaxBXNEfkbHcvjU115AoTgNAMjlC/jU117AwddP4ZlDuar7ATT0AmaDQUTUnth+E3UeWz8RaGx/sBPEtWaOyNf2PUcqF6ijUJzGE88dM96/fc+Rhp2L02Dk8gUoLjQYo2O5hr0nERHVr9PabxG5TkSOiMgrIjJkOeZWEXlRRCZE5KvNPkeiZrD1ExvZH+wUDOaoKU7kC8b7p9W4g4X1+DiwwSAiak+d1H6LSBLAIwA+DOByALeLyOWeYy4D8CkA61V1DYB7mn6iRE1g6/c1sj/YKZhmSU2xNJNGznBBJkWMAd3STDr0a0dNuWGDQUTUnjqs/b4KwCuq+ioAiMiTAG4E8KLrmN8C8IiqngYAVgOnTmXrJ0bpD3YrzsxRUwxuXIV0KjnrvnQqiduvXma8f3DjqlCvW0vKja1hYINBRNTaOqz9zgI45rp9vHyf23sBvFdE9ovIARG5zvRCInKXiBwUkYMnT55s0OkSNY6tnzi4cRVGx3JYP7wXK4d2Y/3w3rZNq24UBnPUFAN9WTx001pkM2kIgGwmjYduWosHB9Ya7w+72LWWlBu/BoOIiFpXh7XfYrjPm6rSA+AyAO8HcDuAL4pIpupJqo+qar+q9i9ZsiT2EyVqNFs/EUBHrZNtBKZZUtMM9GWNQZrtfi9TOmUtKTfOe7EaGhFRe+mw9vs4gGWu2xcDOGE45oCqFgH8UESOoBTcPd+cUyRqHlN/cP3wXuugve2677aKtwzmqC3YStYuTKeQLxSrjg9KuQkbQBIRUWvpoPb7eQCXichKADkAtwH4qOeYUZRm5P5SRN6FUtrlq009S6KY1BJkRR2078YtDphm2eXaJQ/Zlk4pgk5KuaEGCVP+u3zcLSKiItLfzPMjou6jqlMA7gawB8D3ATylqhMi8oCIbCoftgfAj0XkRQD7AAyq6o/n5oyJalfrtiJR18l2UsXbsBjMdbF22q/HNgKTnyzWteaOOl+Y8t/l494O4HcBPNfcMySibqWqz6rqe1X1UlX9XPm+z6jqzvLPqqqfVNXLVXWtqj45t2dMVJtag6yo62Q7rOJtKEyzbHFx5v16X+vsuanIechzxa9kbQel3FBjhCn/DQCfBfBHAH6vuadHRETU2WoNsqKuk+3GLQ4YzLWwOPN+Ta9l04qjF4MbV806f4DplBSaqfz31e4DRKQPwDJV/RsRsQZzInIXgLsAYPny5Q04VSIios5TT5AVZdC+G/uLTLNsYXHm/Zpey6YVRy9sJWs5I0ch+Jb/FpEEgIcB3Bv0Qiz/TUREUbVLfYJGata2It3YX+TMXAuLM+837HNaefSC6ZRUo6Dy328H8LMAviMiAPBuADtFZJOqHmzaWRIRUcfpxuqKJlHTJetZZtRt/UUGcy0szrxf22st6k2hd15P1+zFQV3Jt/y3qp4B8C7ntoh8B8DvMZAjIqJ6+WVZdVt/K8q+wgyAw2Mw18LizPu1vdb9N6zhhUEdTVWnRMQp/50E8CWn/DeAg07VOCIiorh1Y3XFejEAjobBXAuLOiXt5Z2i/rnlC3Hg1dOYVkVSBDdf2X7T0HFW96TuoarPAnjWc99nLMe+vxnnREREna8bqyvWiwFwNAzmWlyteb+mKWp3YzKtimcO5dB/yeK2CYY47U5ERETtpBurK9aLAXA0rGbZocJUrwxTGbOVKjDFWd2TiIiIqNG6sbpivZpV+bJTcGauQ4WdivY7rtVmwjjtTkRERO2m26or1qveZUbdhsFch7JNUZuOs2m1BaicdiciIiLqfAyAw2OaZYcyTVF7BU1Zt9pMGKfdiYiIiIgu4MxchzJNUW9YvQT7XjoZesq6VWbC3BUsM70pzO9J4EyhyGl3IiIiIupqDOY6iKls//6ha2t+Pb8KTM3aIuC+0Rfw+IGj0PLt05NFpFNJPLx5HYM4IiIiIupqDOY6RCOKldgWoAJoSmGU0bHcrEDOwY0jiYiIiJqLe/22JgZzHaJRxUpMC1DXD+/1fa+4Lvbte45UBXIOVrAkIiIiao5Wq3BOF7AASodoZrESv/dyLvZcvgDFhYu9lv3p/M6dFSyJiIiImoN7/bYuzsx1iGYWK/F7rzhnCG3vI4C1giVTAIiIiKjZOr3/0WoVzumC2GbmROQ6ETkiIq+IyJDh8fkiMlJ+/DkRWRHXe1Nzy/b7vZftos7lC1g/vBcrh3Zj/fDeUDN1tu0VfvHSxcYGMs5ZQSIiIqIwuqH/YZsc8Js0GB3LRe77UXSxBHMikgTwCIAPA7gcwO0icrnnsE8AOK2qPwPgYQB/GMd7U8lAXxYP3bQW2UwaAiCbSeOhm9bGPirkjDwVitNIigCe97Jd1AJEbuQG+rK4+cosxHP/Px49Y3wuUwCIiIio2bqh/xF10qAbAtxWEVea5VUAXlHVVwFARJ4EcCOAF13H3Ahga/nnpwH8qYiIqtpqXBCiTdubipXEfS7uxa/TqpUL2Xlf03YGAgRWpLT9O/e9dDJ0NUumABAREVGzdUP/w1bh3NbvbFRhPqoWV5plFsAx1+3j5fuMx6jqFIAzAN7pfSERuUtEDorIwZMnT8Z0eu2p1UY1wow8mWYIgypS+v07ozSQtaQAEBEREdWjW/ofA31Z7B+6Fg9vXgcA2DIybk2f7IYAt1XEFcx5M+GA6smYMMdAVR9V1X5V7V+yZEksJ9euWm3aPuyF6b3YbRamUwD8/51RGsiwKQDM4SYiIqK4NLNuwVwLO9HQLQFuK4grmDsOYJnr9sUATtiOEZEeAAsBnIrp/TtSq41q2C7AhEhVYOS+2G3Onp8KnH2L0kCGWTfYarOdRERE1N68/Y9FvSnM70n4zly1q7ATDbb+24bVSzigHrO41sw9D+AyEVkJIAfgNgAf9RyzE8DHAHwXwC0A9nK9nL9mbjcQhmk9HFBaOwfM3kDSdLF7Fae1Mvtm+3eGydGOsq6QOdxEREQUN6duQadvrh0lSwuY3X/bsHoJnjmU69jfzVyJJZhT1SkRuRvAHgBJAF9S1QkReQDAQVXdCeAvAHxFRF5BaUbutjjeu5OZgqe5nLb3XpgJkUog5ygUp7Ft1wROTxZDveaJfAEPb17n++/0K+xiajTvGRnHtl0TuP+GNSySQkRERE3T6YPGUSYavP239cN7O/p3M1di2zRcVZ8F8Kznvs+4fn4LwEfier9uELVyULPOyXn/lUO7jceEDeSA8LNvNrYZwNOTReNoT6vNdhIREVHnmItB42ZuWF7PRAMH1BsjtmCOGqPR2w3UwxYYhRV29s1mdCzn+/6m0Z5Wm+2k5hCR6wD8B5QyB76oqsOex38bwL8DMA3gTQB3qeqLVS9ERBSjoLbJddwtAHYA+HlVPdjEU6SImj1o3Oy0Tr8B+KCgkgPqjcFgjmpmW0Nnc+c1y7HvpZORRo5sDcPoWA6DOw4HvmcuX8DKod1V79dKs53UWCKSBPAIgA+hVIjpeRHZ6QnWvqqq/6l8/CYAfwzguqafLBF1jZBtE0Tk7QB+F8BzzT9LiqreQeOos2y2tM6tOyca1rcxDcCHCSo5oN4YDOZaSCOnyet5bdtzTYHR2XNTyBeq0yyzmTQeHFgb+ZxtDcPWnRMozoSrn+OuWgm09mwnNcRVAF5R1VcBQESeBHAjgEqHSVX/1XX8Ahi2TSEiillg21T2WQB/BOD3mnt6VIt6Bo1rmWWzpSjmC0WMjuWa1t8Js1aQA+qNwWCuRTRymjzMa/vNgNmeC1RfkABiG3XxaxhMAWMQLrLtWlkAx1y3jwO42nuQiPw7AJ8EMA/AtaYXEpG7ANwFAMuXL4/9RImouet/5lhg2yQifQCWqerfiIg1mGPb1HhRPpdhBo1Nr1dL8RS/JS/N7PNEqXLZodfznGEw1yIaWf3I9tr3jIxj+54jvqVi/abvz03NVD3noZvW4qGb1sbyRdyIhbJcZNuVxHBf1cybqj4C4BER+SiA+1DaSsV7zKMAHgWA/v5+zt4RxazTy7p7+LZNIpIA8DCAjwe9ENum+LmDrUxvCm++NVXJCKr3c2n7nNuWrdj6LqNjOUyen7K+Tz11DaIKux6uiwZrmobBXItoZIUfv9fI5Qt4/MDRqp6tE0j6Td97Oc/ZP3Rt1T5w64f3Bl643gs805syVsZcmklj8vyUb9VMgTlPjotsu9JxAMtcty8GcMLn+CcB/FlDz4iIjDq9rLtHUNv0dgA/C+A7IgIA7wawU0Q2sQhKY3mDLVN/o57P5bZdE8bPuQhg2oF5YTpV1Y8CqjOhvJJiGi9ojDDr4bpssKZpGMy1iEZW+AmqOmkbwnMajSgjO97gz3ThbhkZxz0j48i6GiTv3nS5fAGphCCVFBSnL5yhlB/LpFNIJgTTlnVziuqAjotsu9bzAC4TkZUAcijtcflR9wEicpmqvly+eT2Al0FETddlpct92yZVPQPgXc5tEfkOgN9jINd4tm2PvPxmzGyzT6NjOetgtCmQSwA4e/5CPQInAJrfkwg8R/dewI2eEQuzHi4oU4yzdLVhMNciGlnhJ2rVSUemN4Wz5+zT97bnuJkuXKdpyeUL+ORT47DVMSnOKDLpFBbM70EuX5gVnOULRaQSgndYZu+c98lm0pzK73KqOiUidwPYg1L57y+p6oSIPADgoKruBHC3iHwQQBHAaRhSLImo8bqpdHnItonmQNjBg4RIpWL2htVLsO+lk1Wf31y+UKm+PdCXxdadE9FORjBrUBsoBUBh+nTZ8nXTrBmxoPVwQZlinKWrDYO5FtHICj/Oa9wzMh7peWcKRWugZX3O5OzqSUGzekGvf6ZQxPj9v4r1w3urXqs4o+id14PeeT3G98lm0tg/ZKxjQV1GVZ8F8Kznvs+4fv6/mn5SRFSl20qXB7VNnvvf34xz6iS1zkaFzUpyZr5y+QIeO3DUelxxRnFPOSspqqj9MIf7upnr9GXn7xD0T+nglOqGSsz1CdCFNWVbyhf5w5vXVa07q9dAX7YyQhNWLQ3IDEqNhqPefG1nNNY2mpPLFzC4cRXSqeSs+4O+/J3f+cqh3Vg/vBejY7m6zpOIiOo30JfFQzetRTaThqA0KPfQTWvZuaPInNmoXL4wa4uiMN/3pn5FKinIpFMQNHctms2i3lTVOTqc6waAcTDc0Yz0ZfffIYwOTaluqK6emWuFijpBU99xnmOt6ZZRuS/EaVMCeAQbVi8B4D9KtmVkHIpS4zqtWlmLZ/s9cQEuEVHrYulyikM9s1FB2VIrh3Y35qRDSiUE99+wBgdfP1VVxM49mB3U51uaSftuTRVH/zPs+kP3OVE0XRvM2Tr0B18/hX0vnWzaxt22xmbbrlJOdS1BR5hNvr1r0MJKpxJYvGB+YEGV9cN7MbhxlbUyU1j7XjoJoBSIOkGb6f2AUuDoNGJ+1TJN586pfSIiotbh/s4OO1jrVutslLcP9fDmdVXvF7U4XNzedlEPBvqyxtRFpz/j/GyTSgo2rF5iLVLn7iPWM+gddaatU1OqG6lr0yxtQdTjB47WNCUfxDbdb2sMTk8WraVr3WmMYd7nnpFxrPzUbtw3+gIG+rLYP3Qtspm0byCXSggShiyCqRnF4MZV+MLmdUgl7WkGzr+vzom5SiMw0JcNFXjafj9hpvmDGmamZhIRETWe9zvbvTbN1i9zf0ev2/YN62v7zfyETc10sobmSr5c+M2v+mtgEKXA7u+9YS1S5xckRhFlpm39pYs5qF6Drg3mbB/yuD68Xrbg0S/v2lalMZcvWAMJ23S2KvDYgaO4b7Q0suJ3kWczaWz/yBVYmE5VPVac1soM1vZbrsCi3upjHHGkc7obgbBr/nL5QlWwFXaa3/Z7rSf3noiIiMLz+8429ctGx3IY3HG48h1t2gsXKG1Z5J75uW/0BVz6qWexYmg3Lv3Us/j016vTEp33c4LFFUO78bhPsZNmcKpoJix9yKWZdGAQVZxR3/16TWpZz2ara7D+0sWVPnBSBHdesxyP/9YvRH596uI0yyhT5I3cuLvWNWW2dXVBr/bEc8fQf8liJMopC17uCpBbLFWX3LNlzgjKijryxxPlufwZw2Nnz01VqmNuWL3Et1qUmzclIOzf2pZqOdeVoIiIiNpFveutgvpd3se37pxAMUTVNsWFIm0HXz81q08xrYqz580BpJN+aJu1ctSyfKUWTv/N1od09uONW0JkVsVyP/eNvoAnnjuGaVUkpLRM563iDLeKaoCuDeZMxUBsF2GjN+62rSvLpFM4NzVjHJ1yj0xFKWoyrYpPfe0F3wZg5dDuWQVFqs7LMxs3Opaz/u4y6RTOnp+q2iPFza/9zReKlfztqNWj3MFW2LV7UXPsnVlSNkpERETxFBkLGnD39stsM3Emzvmcm4qWPRQmSGtGIBdWlN9JWE4fEvD/W943+sKsQHlGgUJxBndesxwPDqyN/by6XdemWZrKH99xzfLIJe69vOuq7ht9oVIW1haKmIKMdCqJrZvWVErLmuTyhchVgoDg9Ed3QRGTN9+aqkphtDVgxekZ30AujKDz8XOiHGyFfaotYPQL6ONMtwxal8d1e0RE1Mr8MlnCMqXmOeLYd7BQnK55/7ZuF6Z2gy2L6onnjjXqtLpa187MAebyx/2XLK45NcA0GuX+QIdtNwTAzVdeOLd7nzpsDGSSInOyH4ez+eX2PUcwuHGV7+iZLWWhWZZm0pG+QGwBo9+2DoXiNO596jC2jIzXlT4QZpsKbqlAREStanQsZ+0TRK3+eFEqUfm+c7J/vNUsnXTOTjEvKVCg7kHwRrNlJTn9FJt6t6sis64O5kzq2d+mllkyE8WFkvyA/cM/rYpMOmWdSk9g9jq0BfOSmFFFoWhanRadUymzVaWSUtnSICwBjA2Uc9v27/VW2nKeE2X/lqB1eVy3R0RErSqoIx92qYR34BIALkolqzaPv2/0hao91trd+RYP4txMg8lB/eBW2Gy9EzGYi1Gcs2Tu11rUmzJWHHLWo9l4Q7a5niVrtgXzSvuwbN05ETp3XGFuoJzbtn3q3NwpCINPH66MsOXyBQw+fRgHXz+FZw7lqmbYbA2gU50zaD1fnBvMExERRRHUkQ87K2MbuLynvH4eAJICtFHc05FMg8lB/eDbr17W6NPqSl27Zq4R4ty13nmt0bEc3nyrOmBLJQUirT8VP5fOlAO4qANBfnvVTfoEz24n8gVs2zVR9fcpTiu++tzRyNtU+K25XJpJl8oyP32hLLMTOHJNHRERNUNQRz5oeyFnTXiYdEx2fRonlRBrf8PL+zf36wez+EnjcGYuRoMbV2Fwx+FQ5XGBC9UivZUg3Yt7t+85Yny9BfN6KptGklmmN4XRsVzkfVSA2bNhtr+TH4V9n0Dbx2NaFelU0jqyqaiuuOp8VmyB47ZdE5ydIyKihvOrQBlUtMQZkOQA9dxKCLBgfk/obCZv8GaqL5A2pMhSvDgzFxMnxS1sICcodd6zmTR+0bNxorv4ia1hzBeKCD100qXOFIoY3HG45uc7v3snNaTRXzHZTBoP3bTWd4bOWQAuruMH+rLWwLGWQJaIiCgqWwXKRb2pWZ15U9XvLU+NM5BrAUmR0IGcKUA3VYpnINd4nJmLgWmxrkm2PGrlnl3J5QuzArZpVTx+4CgeO3A0cMNHFgXyN6PATJN+SfVuFOo0igN9Wd+CLe5N3R1MpSTqHlwbS63K+Rz6fT6Dqn7T3Ao7IQHAGqR5Cwk6wTvbrMZhMGcQ9csyTBVLp5yubZsBN+fRRmz4SNXWX7oYr/24UPl7Ry2fDJRSOmuZBROg6jNmOwcBqkbBgqqHBQ0IEFH74PYk1OqCKoLHVfWb5tai3hS27zli3JLJ3YdeWC7U5y4ExzYrfkyz9HC+LN2FJII2hQ5a9JtKCPKT53HPyDj32GhB+39wqhQ8CbBh9ZKaXqPWdEbvdgW2DeYFwB3XLK9q/Py+GFMJwdZNa2o6LyJqPdt2TdS9GTPRXJqLvXEpXqmk4M23poz9ZG8fOl8oVqXPss2KH2fmPGrZy8tvNsfZPqDbtgVoR6rAYweOYn5PAuem4tmLD/BPwXQaQe92Be5iJ1nD/nTuwiw22z9yBUe+iDrA6FjOd4sVdpCpXSz02RuXWtO8pJSKokwWsTSTxtlzU1V/Q3eAFmbm1dZnZhp5bRjMedi+FP2+LDesXmLM+b7zmuXY99JJNlxt5vzUDJIJwXSE3HE/Qa9SKE7jieeOVQVmTiC3YfUS3PvU4aoNy/0COdbGIeoMYdZkx7ktDlE9gjrj56c4sN1uzk8r9NwUHt68DgN9Wawc2m08LsqgkqnQG9PIa1d3MCciiwGMAFgB4DUAt6rqac8x6wD8GYB3AJgG8DlVHan3vRvBNsvm92W576WT1vs5Ytp+FIgtkAvLFpjVujhcAd/ZZCLy5+2Ubli9pNKmN3PEOMwaI7+S70TNEqYzPlmML+uFmqc4rZU+RVA/Odw+gdV9nloy46gkjpm5IQDfVtVhERkq3/59zzGTAH5dVV8WkaUADonIHlXNx/D+sbLtkeH3Zek3m+eXgplKCDZftWxWel0U9VZQpM7GgQSi2gRV3DN1Ut3BX6Y3BdXS9ij1BoJB13EmnWJHh1oCO+OdzWmLgvrJYau7214/7P10QRwFUG4E8OXyz18GMOA9QFX/h6q+XP75BIAfAait0kSD1bJHhm3WzvnSNu27AgAQoOYiuw4AACAASURBVP+SxZX3i4qBXGezfm5CSohw24IyEblORI6IyCvlQSfv458UkRdF5Hsi8m0RuWQuzpNaQ5jZMPcaEe+i/9OTReQLxUpxgMcOHI1UVMvNLysknUqyyBG1DFunO5cvVPaTo/a1NJOuDFoVitOVVEl3P9ndh/Zz9txUZZ9Bpy3060uTvziCuZ9W1TcAoPzfn/I7WESuAjAPwA8sj98lIgdF5ODJk+b0xUYb6Mti/9C1+OHw9dg/dG3giJIpYHPvG2bbCNo9bb1/6NqaAjrqTIt6U5jfU9/lOa0aqdPYqUQkCeARAB8GcDmA20Xkcs9hYwD6VfV9AJ4G8EfNPUtqJWFHgp3jopZbj1LNLexGzERzza/TncsX8Dj3k2trp8+ew+COw5VsMydVcvL81KzjnD6t39p992CX00/x60uTv1C9RRH5loj8k+F/N0Z5MxF5D4CvAPgNVTUmTqvqo6rar6r9S5a05ORdlaDZvIG+rHXzanenwXcWj7rKW8WZWArnsAQwAOAqAK+o6quqeh7AkyhlFFSo6j5VnSzfPADg4iafI7WQsCPBznG1pAGFfY7p++ULm9dh7DO/ykCOWkpQH4bZRO3DtEftZHHGuKn46cmiceA4bDtaKE7j3qcOA0DkzDgqCbVmTlU/aHtMRP5ZRN6jqm+Ug7UfWY57B4DdAO5T1QM1nW0LC9ooM0xhFef5YTYW91owL8ntDzpInJuqMt8cWQDHXLePA7ja5/hPAPhb0wMicheAuwBg+fLlcZ0ftRjTmhAv94ix39pomyipQ0HfL0TNEFSp0vnZ2T6H2lM2k8b+oWsr+96GYVobGaYddTiZRA/dtBb7h66t+dy7VRxpljsBfKz888cA/LX3ABGZB+DrAP5KVXfE8J5tJ+z08UBfFp+/9YpIM3RJEXzu19biC5vXxXKu1FkSIlW56V3GlO1hHC0RkTsB9APYbnq8HTMHKDq/9Hig1Oa6R4xryarYsJqfH2of3nWhtrWfXDbS3tz90qgDwd7jTVkFphk/BzOJahdHMDcM4EMi8jKAD5VvQ0T6ReSL5WNuBfDLAD4uIuPl/3VV5BGlsMpAXxY3X5m1diS8nBENIpNp1ZoKL3SQ4wCWuW5fDOCE9yAR+SCATwPYpKrnmnRu1KL80uNnVKtmJB66aS0W9do7Kl62LW3iMjqWw/rhvd0+kEMx8atU6TU6lsPps2xCW52gtB+yrV8atfCI6XhvDYqtm9b4Dnwxk6g2dQdzqvpjVf2Aql5W/u+p8v0HVfU3yz8/pqopVV3n+t+4/yt3HudD/XB5Bm3LyLjxS3Z0LIdnDuVmpVo6Yd2i3hRMMV6hOI2tOycaderUZgTmTTn9vnw7uOP3PIDLRGRlOUvgNpQyCipEpA/An6MUyBlTxan7RKmuNtCXRe+88Lv9OBX+GnGthZ1FIQorbNl457PH/eRan6I0qOQOtgBU+gKT56dCBwlhC5UEZT2wcmVt4piZowjCfMmaRsAUpVGTsc/8KmzL6eIomEHtL5tJ44fD14cqugN0fsdPVacA3A1gD4DvA3hKVSdE5AER2VQ+bDuAtwHYUc4c2Gl5OeowfgMZUaurRR1VbtS1FjSL0uGDN22lXbZNCTuwEbWyK82tXL5Quf7vG30BW0bGZ22xEiYk96adB7EtJ2LlytrFsWk4heAsHDYtJvUuHA3aqyUhgKGgEBEABBZlCPPl22kbvarqswCe9dz3GdfP1iJPVJugYgmtcC6mzcHdm4G7CzqE+XfYrjmBvZJfI641v1mUoH8zNY9r25QPoZQO/ryI7FTVF12HOdumTIrI/4nStimbm32uQRtFO5gm134Gnz6MP/ja92qaTU2nkjVVnIzatpI/BnNN4P3yNHE3gH6V0VghioK4izLU8+XLL2WqVSsFDH7nEmYgI0olSdM15xfIOeK+1vwGcrph8KaNVLZNAQARcbZNqQRzqrrPdfwBAHc29QzLBvqyOPj6KTzx3DFMqyIpgpuvvHBtjI7lsHXnBLcfaEPFaUVxOtpfToC6A7BmVOltpUHFRmIw1wRh0g7csyWDG1fh3h2HMc3pN4qoN3UhczrsyFfYGTyisBodMNw3+sKsTuXtVy/DgwNrI59LlIGMMJ0Cb2n2MIEcEP+15jeQs2XEvFydgzdzom22TfGu5Z9WxTOHcui/ZDEAYHDHYeMeZNR5nK0L5lpQm9xKg4qNxmDOoxFRfNCXpHe2ZKAva/3CJfLz729636zbYUa+ws7gEYXVyNne+0ZfwGMHjlZuT6tWbpsCOr9zCTuQEaVT4FxzUfZoGty4KtbvHr+BHFu6Pwdv5kQt26b8iulxVX0UwKMA0N/fH3tUFbQOk4Fcd2iVvkGYNrmbshBYAMWlUYUg/L4kbVsUsFmkqBb1pmpqoKJsm0EURpQqkFE98dyxSPf7nUvYAidRy7JHCeQccX/3eEuC++2J1yodtC7UNtum+A2KcFa3OyzqTcXeN6i1GFOYNrmblpBwZs6lUVG8beaj1osibOoOdRfVUsNYa0DH4I3i0sjZ3mlLlVbb/X7nEjYVOWpZ9qjV/O596nDV+TdqBJmFB1pKZdsUADmUtk35qPsA17Yp183ltil+s9hnz02xmnYX6J3XE3sgZ5tdA/zbqDBtcjctIWEw59KooiNxf3kykCOTfKHYsfng1F4aGTAkRYyBm3ffInfa4sJ0ChelEshPFrE0k8aG1Uuwfc8RbBkZD3Vumd4UTk9Wd1bjKstuC0QbNYLMwZvWoKpTIuJsm5IE8CVn2xQAB1V1J2ZvmwIAR1V1k/VFG8Q2KLJh9RKMPG+eFafOEnd7ZJtA2bpzAuemZnxTKMMEat20hITBnEvYTkIton55Zi0f1Gz5g8qqlt1pwbwkUskEzhSKSBg+r52aD07tp1EBw+1XL5u1Zs59v2N0LDerIEO+UEQqIXh48zoAqBoN3jIyjntGxpG1LKJ/862pqvdLJaXhZdmjjCB3S9W2TtMu26aYBmg2rF5SKUREnS/uGS1be2ma5fX2bcIEat2UhcBgziVq+k4j2Upcb1i9BP2XLK4plYfa3/mpGXzu10rpuSuHdhuP6cR8cCKHU+TEr5rl1p0TVQUZijOKrTsnsGB+T1Xb6RxpW0RvKu6wwJBy5LetTFRRRpC7qWobzR33AI3zmWMg1z3intGK2l66+zZhA7VuyUJgMOcSNBsWVhwjpM6eLo8fOFrpaChQKQX80E1rK+9hmqGhzlSc0croVDflgxO5PTiw1roVwehYzrp+J18o4kzA2h7vCLDf6PH64b2z2nnTIFwtMukUtm5aE/p7o9WrtnHWsDO4/45R+x2ZdArF6RmcPc9B6EZrVF2FKKnpYdhm1y5KJUKltXdLoBYGq1m6xFHlK86KmPteOll1Qbq/oJ1KZTMM5LqK07lkVTqi2Zz210+YwQ7vInoTAaraeQCVyrD1+MlbU9gyMh66uptfMYBaq8XFpVFVoqnx3J+dvge+gcEdhyt/x6gDyPlCkYFckzSiR5iQ6vau3mvYVkn7/hvWsG8TEWfmXOLIr41zhDRsBTXb4nzqTE7nspvywanzRZm9sR0bVIBkUW8q1OxZ0CJ608h3oTiNe586jBlVLM2kcec1y/HMoVxdBVH80iXDzJJkelNznn7Z6rOGZOZN3WUfo7t5M83juob9ZtfYtwmPwZxHvdO2ce5rETaNjhNz3SOVkKoFvmzgqN1FWfM1OpbD4NOHUZy+EPAMPn0YgH87m0oK7r9hzaxBEFP76i1sYho0sa3zcAdhzxzK4eYrs8ZiLQ5b0S03b6dpdCyHrTsnZqWSml4jnUpCFZECqUakQ3bTXk+dpNbKrNQ9GnkNs28TDYO5mMW5jilsWdWgNSDUflIJwbyexKy0lKjraIjaRZTZm227JiqBnKM4rdi2a8KapZAQYPstV1Rey+koeKteAjDmKHk7FmE2BS8Up7HvpZO+AdvbL+rBT85NYdpQYMUtly9U3tNvPUxSpDIzOLhxFbaMjBuPM3XCGlVEhWt721PYjrrzmeOYcvta1JtC77yeyMWb/K5hrpNtLq6Zi1mc65icfOJMOlW579zUNO7xrKVY6Hqc2l8mnQIEswK5dCrJQI46VpTZG1u61+nJojVLYUZLQaB3jYepUqVTZMiPqZ03OZEvzNoywStfKAYGcsCF9XmA/3qYGVX8cPh67B+6tlIkyWRpJl21lm7brglrQF0Pru1tT2GC7XQqiduvXsbAvM3df8MaDG5chVQi/DZcftcw18k2H4O5mNkWdNbTCT83NVP52fned18cMWyD13GSERqlViIAFszvqZp5iKNTRdRK3MFEwtKIRe0k+mUpnJ4sYvDpw5UOxehYzjoS7Q0ivYEPUCp0EmRpJo0HB9bizmuW17xfaZTKdN7fly2Q2rB6SVVnyxYk15tK1YjvRGo802cnlRRk0qnK3/HmK7N45pD9OqLWt/7SxZXMg7ddZE/WSyUEi3pToa5hv0wLm7ku1NTumGbZAHHm+vrlrReK01VrJ6gkAaAds/2XZtJcY0Idz5vSZ1vzZRr5zaRTxjYvk05hwXz/VKHi9IVZN7+qlwkRrBzaXdkY2V3IxNlk/I5rllu3swFKbdDk+anK69g2O7cRRN+HacPqJbNue9f7LUynIIJI5xHHrAvXv7SfMAW21g/vtfZPsuXjt+2aMA4USHmUglsrza3Hf+sXKj/7FbnZ/pErQl/DUfsw3CezfgzmWpA71zioiWMgV00A4ya/rc4pbmIrzMBUFuoUtkEq75ovU/ET2wRXmD3kgFKHIqi4g7uQiXuvT4eiFBD5zbXN4ELnKJcvRAqgspk09g9dCyDc+jyHsw+p+/fmXh8YdQ88pkN2t6Ag3NY5F6Dy+bWt24QCPxy+vqbPJcVjUe+FJTqjYzlrFkA2k44UVEVdJ8uKt/VjmmWL8eYaxyGVbM+Uw1q1XxhXmlVwRr64xoQ6na0T6F3z5ea0jX6jx2Gufb/Z76iv2ai2ZvL8VCXNKOz6PMA/lSlMdcJMOsV0SArNb01m2GPcabgAfAdI/NxZnimn8N5860I7s33PEWt75p3xD2Jbf3f23JQxfZLZSPXjzFyLiascsHuEpSchVWuwqDUISqOTbtw/jjpdLRUO42gbkwnB2XNTLT/gc3qyOCsNNMoS4Fo7RiyyRFGFqbgd5hj3DKCTmWTbNiSVEEwWZ2bfnxD0X7IY/ZcsxpaR8Za/vhslIbP3g0tIqc2z9f+cYk8DfVnf9sE04x/I0GblC0Vj+iQr3taPM3MtJugL11lEn82kZ02Ru6VTCVzkGskteBq+Vhd2FLpegtLvKm6LelOhCx7YGquBviz2D11rnaUgame1zD7XO0q7YF4SCfinprdS3aRCcRrbdk3g3h2HZ1W2DaIA7hutXg/o1zHiLBzVIkxxm6gFcJzvvteGr8cXNq+b9bztt1yBRQvmVz3HHZTccc3yxvxjm6TWQklA9cbeMwosmNfjO2PptKt+7UPUAmzb9xyxBpCm12I2Uv04M9di/Ba8Zz0zNMY9ktB+wZtX0Ea7cVHE/7v6wuZ1lb9P/yWLfdcCsLFqHhG5DsB/AJAE8EVVHfY8/ssAvgDgfQBuU9Wnm3+W3aOW2eeoxUCA6GvPWm2prV9KqR+n/Xxw4ELFTdsMCYM4qkeY4ja1FsAxPS9o78QHB9YG9h8y6RTOFIotOYMXdzGYfKGI8ft/1dr+LUynQu1hGWUwLehY7+PMRqofZ+ZajN/6CO9eHUGlZNvVvpdO+o4k1TNy1UjeRcLeEclMOhW6tC/FR0SSAB4B8GEAlwO4XUQu9xx2FMDHAXy1uWdHYW1YvSTSehrvYEm3rb944rljs25ziwDqBGHW6QUpTs+0ZCDXKPeNvmDeaiIhOHt+KtQellF+v0HHmh5nNlJ9Oi8SaHPuEQrTKIq3wk++xpHbVnYiX8DDm9f5jiJHqfAGVOeSOxb1pvBWcabutTi2WTaW5G4JVwF4RVVfBQAReRLAjQBedA5Q1dfKj7X3tHYLcFfj9atKGaUU9ehYDs8cygV2wJLlMufeLAagtpm9ILZ2pRWYRvjjbI/C/J2J4hZmDV7Q3oxR0pY7wRPPHavM0ruv2cnzU+ZtIzD79xc1i8j0N7K9FtuReDCYa0HOF+7Kod3GBsk9wlxLB0UA/OKli7H/B6fqO9EGWeqa4bJd5BtWLzGmUvSmEigUZ6qON5U/TqeSuP+GNQBg3QunN5XA/FTSN90pk06xcEBrywJwT1McB3B1LS8kIncBuAsAli9v77UZjRA2SAtTitqvEIJJQoDP32reC2l0LIf85Pma/k1+3rOwlMZ56aeetaZHLepN4c23ppq+XUojMxi4LxQ1m7s9cK9tNX3/3nHN8oYt1UigtO1Is2TSKZybqm/AeVoV64f3YnDjqkraOQCsHNptPF5RmrmvNcDyTkrYBtnYjsSHwVyT2UYhTPeHqfAzuHEV7rHt44JS4HZROcBxKIB/PHoGac/9jkW9KfTO8998t5FOnT2H0bGcdRR5dCxXlULkWLRgPl50NVaOoODQ9jdwv//oWG7WJu2LelO4/wYGcW3A1KutqWetqo8CeBQA+vv7W3ROJpxaR0T9nhd2vyBbyqPT5tSy99SMwtgRaOQ+Vrl8AeuH9/quc7n/hjU4+Pop4351jXT71csqP8c9+s19oaiZvNewe1zk3FR1H8aZhao1oBMAmd4U8pNFa1/ANgAcp1RCsHVTacA5ysCWiSlQ8psM2LB6yaw1t1GFyQJgOxIfBnNNZBuFOPj6KTxzKFd1/81XZmfdD5hL+roDDDdn8b8pJbFQnC5XwxRjJ8d5D9NsFqA1FQ4JO6JVKM7gnpFxbN05MashO5EvIFMe5bZ1nvzWxQQ1LvU+Ti3rOIBlrtsXAzgxR+cyJ7yd+Q2rlxjbHMCe5uh0JtwpON7n2ToGuXwBK4d2Y2E6BRH/SPqO//xdvPjGT2oKvkwdgbi2ezERwLeDlUmXKg6HSRENI5NO4ez5Kd+tZpIiuP3qZZWOWCNGv7kvFDWT3zVs6/w/OLA2sAiZjQK+A7UDfVls33Ok4cGcMwwZlK3lHJpOJaq2bXDz/q4GN66ybuXw+IGjobcjqHWwiO1IfOougCIii0XkmyLycvm/i3yOfYeI5ETkT+t933ZkG4V47MBR4/37XjoZasH61k1rfMu62i6M/GQRD920tqo8v3uPI9P7P3TT+2r64EjEtJ98oYjBHYcx+PThyibqpyeLvulK3JeEDJ4HcJmIrBSReQBuA7Bzjs+paZzOvHMN5fIFPG5pc9wlo0fHclg/vBcrhnZjy8i4dZG8+3l+qX2K0jUd1AHa/4NTdXWSvMFVozoGQetynH3b4goms5k0FszvsQZyqaQgk05hRhX7Xjo5azPgoL91VH5FKJzPzcqh3Vg/vNe4STBRFFGrIzqcoj+1CLo+mhFwFKd11nnYrrtsJo0fDl+PFz/74cBtkdznPdCXtbZhiuDfAWD+fnEX6vMTRzEbKomjmuUQgG+r6mUAvl2+bfNZAP81hvdsS1Ev/hP5QqgKP0FVyoIumLcMIznuERzv+w/0ZfHHm9dVRp0dAv99mmopuVuc0dAbnqcSwlL/VEVVpwDcDWAPgO8DeEpVJ0TkARHZBAAi8vMichzARwD8uYhMzN0Zx8vUmQ8qP+3+gvY73vu8uMtq12rdtm9UOhMZy36cQCngWjAv/L6WSZFKG+v3L3XaQncQXA9BaRTd9ztES8Gyt0PViNFv275QG1YvqbljR2SzMG2/hgH/zv9AX9a3OraN6bp1D1Qk6liT2ptKVM4p6FXc12mY/dgeHFiLHzz0b63/Zu/vKswedH7qGSzi/nLxiSPN8kYA7y///GUA3wHw+96DRORKAD8N4O8A9Mfwvm0narGSKKMTfimAftWftu85UtO+Irb3u2/0habsEWfytot6mAZJRqr6LIBnPfd9xvXz8yilX3acKJ12p82JOpvkPC+TTvluyt0s+UIRg08fxsHXT+HNt6aMxwiAn1u+EH8fshCUu5ru6FjOmp4ElILfOCvmXVTOnvD7Dqneb3Qa9z51GJnelHGm0y/IDWJbg8w1MNQIk+fN1zAQrvPvV13RRoDK2n2gOl3ZNHAVtrptoThTWdsfVOjJ3Q+Msh9bmKqfznG2tixMH7SWwSJ3WmamN4X5PQmcKZjXJ1I4cQRzP62qbwCAqr4hIj/lPUBEEgA+D+B/A/ABvxfr5GpxURqUOEcn/BoA2wacQPWFHKZ4Sy2jVcmEYDqGSm+duE0DUb3CDiKFSc02EZQWy4+O5XDWp9PVbMVpxePPHYVtslCBwIq+topufoNg9cqU1xW6A7BCcca6jtrPtCrefGvK2Ma++dbUrM5qVLVs6ExUi/M+2Tm2vRK9/ZWbr8xi30sncSJfqKzdzU8WrYMdTpqhX3Enr6QILuoR33VrwIV0ZPf53XnN8sAaCUD4tfthA7+BvqyxQFPYPmiYQn1u3qD49GQR6VQSD29exyCuDqGCORH5FoB3Gx76dMj3+R0Az6rqsaB1U51ULc7Le3ElyuVavZIisW/mamsAbBeik9bjCFu8pZY0q89/5ArfipyppIRKtWSeNVG1wY2rMLjjcGBp/Itca2ejZBEoSovlnzl0PHRKdLPUk/W5yDVzdfbcFLbtmsCWkfGG7FfnJmI+b/c66iiV7YozakzlKs5o7DNmUTt2RPXavudI5bp0DzB7+yvPHMpZ+1UrLCX63YMQYQYkijOKBfN7oJbCcsCFoknu2TDn/NwBZxyzVGEDP6dYTC1FTMLOADo4e98YoYI5Vf2g7TER+WcReU95Vu49AH5kOOwXAPySiPwOgLcBmCcib6qq3/q6juS+uGx7n8UdyPkxXYiC0j4tYfaFeuK5Y3Wtk8mW95S796nD1rSF7bdc4VuJD2CeNZGvEBPm7sJHtnbBL63Qr8JtULGQVnRm8kKxFnfqaKO3bPEr/uKsow6qbOdVSyp9LaJ27IjCsA1wABeuR3eV1rABgzM7ZrMwncL64b2+g+9eZwpFPLx5XaXPstA10+5uB02FpPa9dHLWPnDNVGu17iipnwArWDZKHGmWOwF8DMBw+b9/7T1AVe9wfhaRjwPo78ZAzivqRTCX52C70OoJ5Nxf8tf8L4uMKU8fvXp5VSMzOpbD7u+9UWmsuWk3kd32PUdCz5g5HR6nQxE0iBJGMmQnqNU0c2PgsNwzXFFmCG1/g7hnzFrhO406zx1Xh9sE3Gm/wgQMQftPphKCs+enKgM5YduwpeUBau9n3rRFlN/5ecW9V2ScogSCCy3rqjl7X584grlhAE+JyCcAHEWpGhxEpB/Ab6vqb8bwHh2rFfYuC3MOto5DPR01dzGBfzx6purx9Zcurtq00tQAmzYNJaKSqCOeznXutAtOJ6KWwkbpVLJhe7y1kmwmjR/9awE1bL8ZWkJKhSBWDu2OHFzffvWypmUztMJ3GnUWpx/gZAL59Tv8AiZ3wOC3Bi6bSWPy/JRxljwpghlVLDTs+eh3TYVph8OuM4tjr8hGB4em1wdgXFfNSuT1q3trAlX9sap+QFUvK//3VPn+g6ZATlX/UlXvrvd9qblsJWRvv3pZ1f1AqfSu34fLSa8E7I3qaz+ubvwasWcSUSeLOuLp3qPIu0VBlOc7W6TUUha8nVTWF9dRqjyMGS2lajkl/501Ns6WNLa9pRb1pvDgwNpQe5ZS84nIdSJyREReEZGqjCURmS8iI+XHnxORFc0/y7nnlNx/bfh639L7Nt5AyxZcCYD9Q9daC6pNq2JpJl3aQ/KWK0JfU0HtcK3rzGpRz95w9bz+H3zte8YsEVYir18cM3PUBfzSZ/ovWYxtuyZmjWJNFmeQSgrmJ6RqLU3YRtV0P/OtiaLZsHpJVaUyP+4R71o2vJ5WrVzjTrsRtSx4O1maSUdKZY2Ld42NbQ32/TesAdB6M2atnDbWLCKSBPAIgA8BOA7geRHZqaovug77BIDTqvozInIbgD8EsLn5Z9taolQHzxo+X0HFevzSmJ3g5KGb1oZe4+a3Dtl0fm5x93saXYTE9vo2rERePwZzFJqtMzDQl8X2PUeqUhKK04qfevtFlf2HbF/aUSqgsVoaUXijYzk8cygXqfiIe8S71s6Cu2PgXOu2IkcOvyIHrcqpTDdX3H8f94BbLl9AUmTW6H0rBUqNSBtrU1cBeEVVXwUAEXkSpb173cHcjQC2ln9+GsCfioiottvVEi/vALPtl+HMtHkFFesJChajBj/1rCeNu9/T6EHxqK/D/lv9GMxRLPwah6AR4SgV0FgtjSj8rEbUmTXnWnJev57eYi5fwIqh3aHX1bZj13SuT9nbCTLNhAYFSqNjOWzdOVEpStCbSmB+Kon8ZOM28WV58oosgGOu28cBXG07RlWnROQMgHcC+Bf3QZ28R6+Nu29hKzBiCxSCgivv4IhJ1KCl1tnxuPs9jR4Uj7p9C/tv9WMwR7Gop3GIMmLFamnU7aLMakTpbCzqTVVS8uJMi2zHSpbtwLsXqMMWKG3bNWEsSODdg3CyOFPZ9LhRM2ZMl68wLXT0XjBhjunoPXrDqCXgCQqunMejBopxi7vfs2H1EmNBqw2rl9R1no4oW9tk0in232LAYI5iUe/IUZQRq1Zb+0HUTFFmNaKMkPbO66l0XDp1fVu78naETHuBOmwB0WnXvnm5fAGffGocqsGzi4XiNO4ZGcc9I+PWtT1R178xXb7iOIBlrtsXAzhhOea4iPQAWAigeh+fLhdHwGP7HLdCRlCc/Z59L52MdH9UppTvadWqdiydSmLrpjWxvGe3YzBHseCMGVFzRJnViFIkwHl+F86OtKx0KomHbiqVZQ/btoYN4GdqmLsxzdTVsv6tFTrHLeJ5AJeJyEoAOQC3Afio5xhnL9/vArgFwN5uXy9nU0/AE+Zz3Cn9m2bMjJtSvhXhi75QNAzmrtBlEwAAIABJREFUCEA8lcU4Y0bUeFFmNcIWH3E/3y8YSCWl6VUbu5UAuPnKC21q2LY1SgBfC/dMnd8xfuvfOq1zXKvyGri7AewBkATwJVWdEJEHABxU1Z0A/gLAV0TkFZRm5G6buzPuXEEZD3H2b+a6kmuzZsZNv1MnkAtbBZTCqXufOWp/jd5zhIjiY9vz0TarMdCXxUxAIOd+vt+6iZ5EY/dSowsUtaU9DfRlq/aUmwtBo/wDfVnsH7oWPxy+HvuHru26QM6hqs+q6ntV9VJV/Vz5vs+UAzmo6luq+hFV/RlVvcqpfEnxatY6zlbob5m+Q1A+l3XbvhHbuXBtbPMwmCNuxE3URkyd9aANoP1GXN0zQM5WBjbePSOpsWrt9HgDpUW9qdDP7U0lYtn/vAvXv1Ebs31emzFb1ez+lvMdkklXtwv5QhGDOw7HEtA163dKDOYIHD0hajdRZzVsI7HA7BmgWjYJp8aJq9Nz/w1rkEqGi9Be/OyH8fCt66yflzC6dP0btbGoGQ+1apX+1kBfFgvmm1daFWc0luCyWb9T4po5AiuLEXU6J9izrXM6kS9gdCw3pxtg02yC+EqFe9eoJSx7/zkpmbZqdGHOuVvXv1F7a9Y6zij9rUavrfMLIOMILrk2tnkYzBErixF1sNGxHLbtmqiUpTdRAFt8ClpQ8ymAkeePof+SxbF0ftwFHLyV+4DqNt9b8MH0HLf1ly7G47/1C3WfJ9FcaXQRt9GxHM6em6q639TfqqVKbFR+xa7iGsxnYbzmYJol1bQGh4ha3+hYDoNPH/YN5BysUdl6itOKbbsmQh8/OpbD+uG9WDm0G+uH91rXvXjb/EW9KczvSWDLyLj1ee7neDGQI/LnBGf5wuy2eFFvytjfsq2tu/epw4HXd1iDG1cZ069TCYllMD9se0T148wcAeDoCVEn2r7nSEdvJdCbSqBQnOnoQNQJxINSrqKO5DttfpTn2b4nnE4bU6mIzGzrkXvn9RivFVuao5Pu7FynB18/hX0vnazp2nOOc2duZNIpbN20pu7rtxkzi3QBgzkiog7V6UWMJrukuuZ9oy/g8QNHK0FrLl/A4NOHsXXnBM4UiliaSWPy/JTvPlk2QftrBWGnjShY1MInfimQjkJxuqpdiHrtNWogv952haJhmiURdTwRuU5EjojIKyIyZHh8voiMlB9/TkRWNPsc40xJcV6rk2esukVvKjGrw+YoTivyhWJlrypbKm1QQF9vdb1WKLVO1Ooylu1BbPf7VSB287YLrXLttUrVzm7BYI6IOpqIJAE8AuDDAC4HcLuIXO457BMATqvqzwB4GMAfNvMc49xI1v1aUXFL8NaSkFJnrZ6gPKiQQb17QbHTRhTMVgzWdr93XWsywuaPrXDtcY+55mIwR0Sd7ioAr6jqq6p6HsCTAG70HHMjgC+Xf34awAdE4tg6OZyosxt+s3i17hWXzaTx8OZ1kZ9nsv7SxXXtU0YlF/Uk6tqoPZ1KYsPqJb4zvvXuBcVOG1GwMwXzzLntfmD2fqKfv/WKquvU9gXVCteeX7vCwijxYzBHRJ0uC+CY6/bx8n3GY1R1CsAZAO/0vpCI3CUiB0Xk4MmTJ2M7wSizG36zeLXuFZfNpLF/6NrIz7OZOPET3HxlNtJoMlWLuiYwk07Nqkp885VZPHMo5zvjW281Y24MTBSs3kEP03V6xzXLW/bas7UrAGLLQqELWACFiDqdKaLwJreEOQaq+iiARwGgv78/tiVpUTaStc3ibds1gTffqt7DKAxntHTw6cM1Pd8rXyjiqweOojvKk9Qnm0kjP3keZ89Hn011S6eSVVXo1g/vDVWEoJ4iCNwYmChYHPv5mq7T/ksWt+y1ZzrfsG0SRcNgjog63XEAy1y3LwZwwnLMcRHpAbAQwKnmnF60L3rbLF6YveRM5vckcPD1U3jiuWOVstdxYCAXLJUUrHhnGvsNf9NkQvD2+T1V+1KZ2MqJN2s9G7e2IfLXqEGPdrv2uMa2MRjMEVGnex7AZSKyEkAOwG0APuo5ZieAjwH4LoBbAOxVjTGyCRDliz5Myeoozk3N4LEDR2N7PQpvalrx96+axwxmZhRbN63BPSPj1udnfT4no2M5JESMAXorrKkh6jbtFng1QpQsFAqPwRwRdTRVnRKRuwHsAZAE8CVVnRCRBwAcVNWdAP4CwFdE5BWUZuRua/Z5hv2it83i1VL0hOaWVv7P/NhAXxaf+tr3jEVQMumUdZ2js67SFMi1ypoaIuo+caSbUjUWQCGijqeqz6rqe1X1UlX9XPm+z5QDOajqW6r6EVX9GVW9SlVfndsztrMtLF9k2a+I2pNTPOahm96HVGL2kk7npq0anK2iaVIkUnETIqI41Vtwicw4M0dE1GZss3iDTx9GcZpbhXeC268uLfN0p+Dm8gUkBJhRVNbSOdXg3Mfa1p/MqLLTRERziumm8ePMHBFRBxjoy2L7LVfMGvG885rlSCVnz+okE8KGv80M9GUrWwDMGGL1QnEaW3dOVG77lUHnHk9ERJ2FM3NERB0ibOnqbbsmaq5+Sc3x+IGj6L9k8ayZOb91kflCEX0PfAP337DGui5lw+ols+43zeoREVF74QAtEVEHG+jLYv/Qtfjh8PUY3LgK2/ccYSDXBhTAtl0TlVm0MBVMT08WK8GZaV3KvpdOWvd4IiKi9lR3MCcii0XkmyLycvm/iyzHLReRb4jI90XkRRFZUe97ExFROE6Fwzi3NaDGOj1ZRC5fsBW8NHJvwDu4cRWWZtI4kS9U1tyZcI8nIqL2FcfM3BCAb6vqZQC+Xb5t8lcAtqvqvwFwFYAfxfDeREQUQlCaHnWOE/nCrOBdAd8gnns8ERG1rzjWzN0I4P3ln78M4DsAft99gIhcDqBHVb8JAKr6ZgzvS0TU9UbHclVr4oDqDcg5+9I9Mr2p0ME793giImpvcQRzP62qbwCAqr4hIj9lOOa9APIi8jUAKwF8C8CQqlZ904jIXQDuAoDly5fHcHpERJ3JmX1xF7S4Z2R81jG5fAGDOw5DBDDsIU1tJptJB6bKnikUQ62L5L5zRETtL1QwJyLfAvBuw0OfjvA+vwSgD8BRACMAPg7gL7wHquqjAB4FgP7+fnY92phpxoCdBqL4hJ19KZrq2VPbSSUFgxtX4d6nDmPaJzKfUVT2o/PTivvO8XuDiCiaUMGcqn7Q9piI/LOIvKc8K/cemNfCHQcwpqqvlp8zCuAaGII56gymGQOWwCaKF1Mnu0tPQjDQl8XB10/hsQNHfY+d0VIKpV+w32pr5fi9QUQUXRwFUHYC+Fj5548B+GvDMc8DWCQiS8q3rwXwYgzvTS3KNGPAEthE8RkdyyEhEnwgdYxCcQYA8ODAWtx5TfAyBGd7AgDwflJaca0cvzeIiKKLY83cMICnROQTKKVQfgQARKQfwG+r6m+q6rSI/B6Ab4uIADgE4D/H8N40h/zSYWwzBpxJIKqfM4Phl2pHnWt0LId9L530PSaTTs3acDyXLyApgmlVZFs0fZHfG0RE0dUdzKnqjwF8wHD/QQC/6br9TQDvq/f9qDUEpcMstSzSb7W0HqJ2xG0GutOi3lRV22uSALB105qqY6dVKzNyrRbIAeiq7w0RWYxS/YAVAF4DcKuqnvYcsw7AnwF4B4BpAJ9T1ZHmnikRtbo40iypCwWlwwxuXIV0Kjnr8VZM6yFqR5yp6E7337AmMJBPpxL4483rMNCXtbbT94yMY/3wXoyO5Rp9ypF02fdGmD16JwH8uqquAXAdgC+ISKaJ50hNNjqWw/rhvVg5tLslr1FqTQzmqCZB6TADfdnKeg1BqZw2S2ATxcM2U5HkGrqONtCXDRHIX/gM+B3rZFO0Umexy743bkRpb16U/zvgPUBV/4eqvlz++QRKBeaWeI+jzuDMpOfyBSha8xoNwmB0bsSxZo66UJh0mIG+bKd+CRPNqcGNq6pS7QTgGroO5hQysbW9DidDwi/d3XRsq+ii740we/RWiMhVAOYB+IHlce7R2+b8Mp7a4ZpgNdq5w5k5qkmXpcMQtRT3DAZQCuTiCuNS/FZoOe621dT2ejkzclGOpfiJyLdE5J8M/7sx4uu8B8BXAPyGqs6YjlHVR1W1X1X7lyzh5F07avcCQKxGO3c4M0c1cVdJ4+auRM3nzGCsH97rO/sSVdHYVaS54q086a1QaeJkSEQ5luIXwx69EJF3ANgN4D5VPdCgU6UW0O4FgNo9GG1nDOaoZp2YDuO33QJRo9g+d2E+j/yi7FyZdMr4N3faXlNlS2+GRJRjqamcPXqHYdmjV0TmAfg6gL9S1R3NPT1qNlP6fDtdo+0ejLYzBnNEZcz3prlg+9wdfP0UnjmUC/w82r5AM+kUFszvwYnyYnpqP/lCEYM7DmPbrgnkJ4tVAX2UDIm5yKbg4JivwD16AdwK4JcBvFNEPl5+3sdVdXwOzpcarN0znto9GG1noi28YL6/v18PHjw416dBXcKWrpbNpLF/6No5OKPOJSKHVLW/Ce8TuJdT+bi/A3ANgP9PVf/XMK8dV/tk+9w5Gzx7eT+PthkXdxXAuFMxae54/7atKsznshU1q21qJPadaK5wAKdx/NomzswRlTHfuyM5ezkNi8hQ+fbvG47bDqAXwP/RzJMD7J8vW2VK0/EJ144EAuDmK2enQA9uXIXBpw+jON26g3cUTjOr29XTMWv3ynxEFF0nLr9pBwzmiMqY792RbgTw/vLPXwbwHRiCOVX9toi833t/M9g+d7aZOffn8b7RF/DYgaOzHlcAI/9wDP2XLK5Kx/vkU+OYYTzX9nL5AtYP78WJfAEL0ymIwJiGWY960845OEZE1BwsQk1Uxu0WOtKsvZwA+O7lFERE7hKRgyJy8OTJk7GcoO1zd/vVy3w/j6NjOTzuCeQcxRmtKgc90JfFH9+6LpZzprklQGVj4XyhiNOTReMmw/Vs4FtvmXHbIBgHx4iI4sVgjqjMvXeXoLQ2qdXXd1B8ezmF0Yi9nGyfuwcH1vp+HrfvOeJb2CSXL1R14gf6sujlRnJtLWhPQSfgcmbWnKAvly9gy8g4VoQM7OqdWePgGBFRczDNksiF+d7tJ469nOaa7XPn93kM06l2z9Y4rzevJ4lJbiY3Z1JJwYJ5PThTKEauMpq1pOR6ncgXsHXnRNXMmvN+YVImM70pnJ4sGu8Po90r8xERtQsGc0TUyQL3cmpXtrV2JoXiNLbtmsDWnRPIF6o76NQ8m39+GR4cWAsgWpVRZ/PwLSPjgUGgLRBzCypGYit0bbvfViyFwRsRUWMx34aIOtkwgA+JyMsAPlS+DRHpF5EvOgeJyH8HsAPAB0TkuIhsnJOzjcCUxubn9GSRgVwL2P29Nyo/m/6GqaQg5S5PigvpiUGptc6xYXcc8pvdPWP5rJjuN6V0utfuERFR4zCYI6KOpao/VtUPqOpl5f+eKt9/sLwpr3PcL6nqElVNq+rFqrpn7s46HGet3aKQaW/UGtwzZqb1kttvuQLbP3KFca1kmNTam6/MWgMxL79iJFEKmNRbLIWIiGrHNEsiojY0OpablTYpUkqBy6RTOHt+invKtQm/9ZJeYVJr9710MtRxQcVIBjeuqtr0O5UUnD03hZVDu2elUnIbAiKiucOZOSKiNjM6lsPgjsOz0iZVS53trZvWYPPPL5vDsyM/mXTtM6lhUmtP5Avm9M2EYFFvKrBSr7OdwZaRcVyUSiCTLj1nUW8K0NJWCN7qmAmRqtcBuA0BEVEzcGaOiKjNbN05gaJh9+/itOKekXEkLZ1rmntbN62p/GwrGmLjrhBpm3lbmknXXEnSu1H46cki0qkkHt68Dtv3HKkqquJ8Ak2b23MbAiKi5mAwR20tameIqN2NjuUCC5mYOtfUGu4ZGcenv/4CVHXWFhHOTNfB109Vql2aOO2bqTKpO4Byp2867eSWkXHfdtJv7VuYlMmkCGZU2RYTETURgzlqW95R5DB7JxG1uziKSmQzaax4Zxr7f3AqhjOiqM6enzberwAeP3AU/ZcstrZh3nbPkZDZRUfcgVzYdtJv7VuYdXgzqvjh8PW+xxARUby4Zo7aFiuoUTeKo6hELl/AP/zwdAxnQ3FT+AfspnYPAJysW++2AFHaSb8KlmHW63mf76y/Wzm0G+uH93KrAiKiBmAwR22LFdSoG2Vi2orAtOaOWkMuX7AGP2HaN3ewZjs+ly9Uvb4pYHNSN93bKACAd1Wmd40c954jImoOpllS27Kl/bCCGnUS97rQTG8KZyar18slE4IEZgdo6VQSF6USVUUrqD3Y1tCFSXcELgRxfsd70y2DCqeY1uHZ1iv7zQgyDZ6IKD4M5qhtmfZBYgU16iSm6oImb5/fg62b1lR1rreMjDfzdClmpjV0pnbPxBnU8jveFFzZ9r3zCjqOmRNERM3BYI7aVq3lt4nahW19lNeZQtHYufYrYU/twVlD550927Zrwhrce6taAqUqmiaNCq5sM4KZ3hTWD+9lm01EFBMGc9TWwo4iE7WjsB1tW2px2FmcViO4sIdZJxApberulRQJtY2E6XPwlmtbA+DC7yxrCJAG+rLWwD7utHQn/TKXL1T9HVNJwZtvTVWCUFYgJiKqHwugEBG1qDAdbb/UYqdoRZBW22K8kwK5VEJwx9XLjYVFPn/rFfjC5nWBr+H9HJhmbJ1Abv/QtcbAyK+4SVzcRU+cc3I+W9lMGgvm9VQV3mEFYiKi+tQdzInIYhH5poi8XP7vIstxfyQiEyLyfRH5ExFptf4DEVFLMXXAU0lBJp2CoNRBfuimtb6zGgN92UoFQq9sJo3/n727D7OsLO98//3ZNARNTIM0BhuwcdIxwTfQHkLClcSICGpCMwkmGBPbhDk9RjITx8TYHHMOxsSTdpgZo5cec1CJMOMISBQ7gYgtyHiSI0oj7xJCi4w0EOjIS2Ig8nafP/aqdnf1rqpdtV9qr6rv57r2tdd61rPXuteuqlXrXutZz7Nm1f5LKnmaJAc8fSVnv+4l/NEpL9rdE+T0n9spR69h1f4z91Daq5fImZrOznYnt7s3yn5/d6a2N9fwAlN13nrh9bMmmQ/PMNi9z9FJWkrGPSzLMJpZbgauqKotSTY38+/orpDkJ4HjgBc3RX8N/Axw1RC2L0lL0rCeC52ts6DZOklZas0dx+kZ+67grJ9/Qc+eIKd718kv6Nkc9oCnr9xjHVN3vmZSwHFbrpzxd2S+zdL7GXB8pkHMu83Vs6Y9EEtaKvo5bg7bMJK5DcDLm+nz6CRo75hWp4DvA/alc36wErhvCNuWpCVtphPwubqGn74O6J0UzvQs1dSzV+/8zE3882OT9czdTM+gTZJ/fuzJvv+B95u099MhzjBPHPoZXqCfmGbrWdMeiCUtJYsxLMswkrlnV9W9AFV1b5KDp1eoqi8n+SJwL51k7oNVdWuvlSXZBGwCOPzww4cQniQtLQu58jdTUjjXCfakjS2+av+VM97JGqenr3waj0zrhGS6Rx9/kt+56Ab+44XX95Vwz/WPvt/miMM6cehneIG5Ylr5tPDIY09wxOZLec6q/fnFl63hi3+7y94sJS1JizEsS1/PzCX5QpKbe7w29Pn5HwZ+DDgUWAO8IslP96pbVedU1fqqWr969ep+90OSlo2Zrvz9wV/c0vfzTVN1gBmfpep3aIRh2HdFdscw2zNkDzXPXf3iy9awYhEevd5/5Qp+9djDqT67jXmyiuJ7Cfcgz07MpzniME4cZtped/lsMa3afyWkMz7i1Hfw59fezdtPfD7f3PLaGTtrkaS26ue4OWx9JXNV9cqqemGP12eB+5IcAtC8399jFf8GuLqqvlNV3wH+Cjh2WDshScvJTCfqDz7yOHc/9OiMyUN3b4PddQD+ZvMr9jrBHmfHFFXsPsm//qxXzVr37Z+6gQuvuauvbv2Hr/jvV39rQUnuoD039uoQZ6aUchgnDv30gDlTnT/55aN4xn778PiT9l4pafkYR8/B0w1jaIKtwMZmeiPw2R51vgX8TJJ9kqyk0/lJz2aWkqTZ9XuiPv3EeaY7er9z0Q097xjNNyEY5EbZ408V79p6y+75A54+8925x5+qvZKEYXraLPvx6BxNK+cySILcq0fKNxzbe9iDYZw49NMD5mx1FqO5kSQtpoX2HDyIYTwztwW4KMnpdJK21wEkWQ+8uar+LXAx8ArgJjqdoXyuqv5iCNuWpGVnPoOB9/N805NVPZ+567WdlU8LhD2Sqf1Xrtg9nt0gz7I99OjjXHLd3Zxy9BrO+vkX8PaLb5h30rZmhh4T52NFwlMLuOu3av+VPGO/fbjnoUd52gwDgg96x6zXs3Xrn3vgwD2ezmd7/dax90pJy9F8ew4e1MB35qrq21V1fFWta94faMq3N4kcVfVkVf27qvqxqjqyqt426HYlabnqdeVvpufM+n2+qVfzt17bOft1L+HsU1+ye+y6Fcken52qP7WM5nOz3WnrNrWeU45ew9mnvmRez8VNjWc207h6/Xr8qZr383j7r1zBu05+we7mqv/ll17Ss0nk3Q89OpRxh7qffTz78tsm8jm0xWhu1Bb9jtHb1H1mkruTfHCcMUpqh2HcmZOkiZTkQOBCYC1wJ/BLVfXgtDpHAR8Gngk8Cbynqi4cb6TzN/3KX6/xvno93zTbnbNed+5mu8I4vUfN/3jh9bzh2MP5m82v2KtuP+ORTa2ne9vTtwMz3x2c2td+m/Ed8PSVPPhI74Gsn6xi/5Ur+rrLuGbV/vzsj67m7Mtv26Pnyj/+hRftHvqhe8y+QYcPWIxxjBZiWOMkLlFzjtHb5Q+B/zm2yCS1ismcpKWsnxOmR4A3VtXtSZ4DXJvk8qp6aNzBDqKfE+ep6d+56IaBmwD2ev6ugE9c/S3WP/fAvU7Y59p2t7WbL92rbEXTbHFq/LupGKbv6yXX3T1jE8cpa7rqH7flylnH2fuDv7hlxoRvpualU8nVH//Ci/ibza/ouY1Bhg9YjHGMFmrczY1apJ8xeknyMuDZwOeA9WOKTVKLmMxJWsrmPGGqqr/rmr4nyf3AaqBVyRz0/3wT7H23a77N32a6+1UwY1Ix07b7MXWnrDtBnb6NqTtWMyVyK1eEs099yZzPBXZv5+zLb+uZzK1Idj/UftyWK2dNrobdEYgdiywJc47Rm+RpwH8Bfg04fraVOUavtHyZzElayuY8YeqW5BhgX+AbMyxfEidM823+dsl1d+9uLjh1h2zFLHe/Zksq5nOHbrq57j7NNi7eAU9fyVk//4K9mqb+wV/cssdnptebaV+eqpqzzlT5sDsCmcSORaZ+R2xO+T1JvgD8UI9F7+xzFW8BLququzLHM5xVdQ5wDsD69esXY8wOSYvEZE5Sqw3hhGlqPYcA/w3YWFU9+59fSidM/TZ/m/581lQCNlsiNldSMcgdutkSxdmW9UrkevWW+Z1/eWKP+X4Sp7nqzHb3byGGvb5BteUZvnGrqlfOtCzJfUkOaS4yzTRG708AP5XkLcD3A/sm+U5VbR5RyJJaaBjjzEnSoqmqV1bVC3u8Pgvc15woMcsJE0meCVwK/H5VXT2+6CffbHe7euk3qZjeU2a/vUfOlijOtmx6T51nX35bz2EPHn+q9qg7yMDZU3WGPe7QYoxjNJvZnuHTjOYco7eq3lBVh1fVWuB3gfNN5CRN5505LTs2B1pWpk6YtjDDCVOSfYHP0DlR+tR4w5t8/TyHtWbV/gv6e+q+O3jJdXfz9k/dwONPzXzHb65E8e0nPp+3Xnh9z2XT96PfO3zz6VhmrjrDPM5M32b3kA7j5jN8C9LPGL2SNCeTOS0rNgdadvo5Yfol4KeBZyV5U/O5N1VV76xgmZmpCeGUqbHdBjF1gWW2RG6m7v/7/budftdutv2aXneQgbNHZZKOZZP4DN+kq6pv06NTk6raDuyVyFXVx4GPjzwwSa1jM0stKzYHWl6q6ttVdXxVrWveH2jKt09d+a6q/15VK6vqqK6XiVyjVxPCKcN4TmsqKZktYQT42R9dzZ9fezd3P/QoRSd5eeuF13P0uz+/ewDumf6OA3vF+fYTn8/KFXs37Vz5tLRiUOtJOpY5OLgkLR7vzGlZsTmQND/dzfm6e7NcM6Qmyv0+k/eJq79Fr/t2Dz7y+O47UrMNlzDTuHfd48it2n8l7zr5Ba24Sz9JxzIHB5ekxWMyp2XF5kDS/I2yCWG/ycdsXYdO3ZGa6e97zQx/320e0HrSjmVt/i4lqc1sZqllxeZA0mQZVvJxz0OPjvzv+5Lr7ua4LVdyxOZLOW7Llbubdy4Gj2WSJDCZ0zIzaV16S8tdr6RkpkEKZhu84Dmr9h/p33f3s31Tz+yd+embFi2h81gmSQKbWWoJ6XfIAZsDSZOj1/NWa5+1P//fNx7Yo2nl/itX8IsvW8Nf3nAvDz36+B7rmD6m2/TBwY/bcuXAz3LN1OHIu7besmjHE49lkiSTOS0Jk9RNt6T5mT7e3JmfvmmPRC7AL75sDX90yov4o1Ne1PeFm4UeF3qtf6Zn+x569HEuue7uiTzOOKamJC19JnNaEmbrptuTF6k9ev0tF/DFv921e77fO1ILOS7MlAD+4P4r97oj2L2dSTvOtO0Cl4mnJC2Mz8xpSZikbrolLdww/5YXsq6ZEsDM8sDeJB5nJmkcurlM2vOIktQmJnNaEmbqEc8hB6R2Gebf8kLWNWNzykce54CnrxxabKM2037c/dCjE5cktSnxlKRJYzKnJcFuuqWlYZh/ywtZ12wJ4Fk//4LWHGdmSzAn7a6XLSskaeFM5rQk2E23tDQM8295IeuaLQFs03Gm135MWchdr1GOsWfLCklaODtA0ZJhN93S0jDMv+X5rqvXUAndnXG05TgzFeNbL7y+5/L53PUadWcqbz/x+XusHyb3jqckTRqTOUmSurQlYZvLKUev4ezLb+PuHonbfO56jborHo/cAAAgAElEQVS34LkSaEnSzEzmJElaooZx12scz7QtlQRaksbNZ+YkSVqihvGcn8+0SdLk8s6cJElL2KB3vXymTZIml8mcJEmakc+0SdLkMpmTJEmz8pk2SZpMPjMnSZIkSS00cDKX5HVJbknyVJL1s9Q7KcltSXYk2TzodiVJkiRpORvGnbmbgV8AvjRThSQrgA8BrwaOBF6f5MghbFuSZpTkwCTbktzevB/Qo85zk1yb5PrmwtSbFyNWaS6XXHc3x225kiM2X8pxW67kkuvuXuyQJEmLbOBkrqpurarb5qh2DLCjqu6oqseAC4ANg25bkuawGbiiqtYBVzTz090L/GRVHQX8OLA5yXPGGKM0p0uuu5szP30Tdz/0KAXc/dCjnPnpm0zoJGmZG9czc2uAu7rmdzZle0myKcn2JNt37do1luAkLVkbgPOa6fOAU6ZXqKrHquq7zex++Cyx5mkcd8zOvvy2PYYGAHj08Sc5+/K5rqVKkpayvk5aknwhyc09Xv3eXUuPsupVsarOqar1VbV+9erVfa5eknp6dlXdC9C8H9yrUpLDktxI56LTe6vqnhnqebFJe+h1x+ytF17P0e/+/FCTunseenRe5ZKk5aGvoQmq6pUDbmcncFjX/KFAz5MlSZqPJF8AfqjHonf2u46qugt4cdO88pIkF1fVfT3qnQOcA7B+/fqeF6S0vPS6Ywbw4COPc+anbwIYSpf+z1m1P3f3SNyes2r/gdctSWqvcY0zdw2wLskRwN3AacCvjGnbS9Il193tAK4Ss19sSnJfkkOq6t4khwD3z7Gue5LcAvwUcPGQQ9USNNudsalmkMM4Nr/9xOdz5qdv2iNx3H/lCt5+4vMHXrckLSbPaQczjKEJ/k2SncBPAJcmubwpf06SywCq6gngt4DLgVuBi6rqlkG3vVz5ILzUt63AxmZ6I/DZ6RWSHJpk/2b6AOA4wAeR1Je57owNqxnkKUev4Y9/4UWsWbU/Adas2p8//oUXecLTUv30tNvUOzzJ55PcmuTrSdaON1JptDynHdzAd+aq6jPAZ3qU3wO8pmv+MuCyQben2R+E9x+7tIctwEVJTge+BbwOoBkT881V9W+BHwP+S5Ki83zvf66qmxYrYLVLrztm3YbZDPKUo9d4jF86pnra3dKMvbsZeEePeucD76mqbUm+H3hqnEFKo+Y57eDG1cxSQ+SD8FJ/qurbwPE9yrcD/7aZ3ga8eMyhaYmYOtl419ZbeOjRx/dYZjNIzWID8PJm+jzgKqYlc814vPs0xyiq6jtjjE8aC89pB2cX3C0005VeH4SXpPE75eg1XH/Wq/iTXz7KZpDqVz897f4I8FCSTye5LsnZSVb0Wpk97aqtPKcdnHfmWsgH4SVp8tgMUt2G0NPuPnQ6YzqaTjPxC4E3AR+bXtGedtVWntMOzmSuhaZOFuz5R5KkyTSEnnZ3AtdV1R3NZy4BjqVHMie1lee0gzOZaymvAEuS1FpTPe1uYYaedukM63RAktVVtQt4BbB9fCFK4+E57WB8Zk6SJGm8tgAnJLkdOKGZJ8n6JB8FqKongd8FrkhyE53edj+ySPFKmlDemZMkSRqjfnrabebtbVfSrLwzJ0mSJEktZDInSZIkSS1kMidJkiRJLWQyJ0mSJEktZDInSZIkSS1kMidJkiRJLZSqWuwYZpRkF/C/5vGRg4B/GFE4k8T9XFqW434+t6pWL2Ywg1qixydjHJ42xGmMe1uOxyaYrN8FY+nNWHpbLrHMeGya6GRuvpJsr6r1ix3HqLmfS4v7uTy0Yf+NcXjaEKcxasokfc/G0pux9GYsNrOUJEmSpFYymZMkSZKkFlpqydw5ix3AmLifS4v7uTy0Yf+NcXjaEKcxasokfc/G0pux9LbsY1lSz8xJkiRJ0nKx1O7MSZIkSdKyYDInSZIkSS3UumQuyYFJtiW5vXk/oEed5ya5Nsn1SW5J8uauZVclua1Zdn2Sg8e7B/0Zwn6+LMlNSXYk+UCSjHcP+tPnfh6V5MvNPt6Y5Je7ln08yTe7fp5HjXcP+jOE/TwiyVeaz1+YZN/x7kF/+tnPpt7nkjyU5C+nlbfi5zmTNuz/EGIc+e/iPGLc2NS5PcnGrvKRHeeTnNSse0eSzT2W79d8Lzua72lt17Izm/Lbkpw4rJiGFWOStUke7fre/nRUMfYZ508n+VqSJ5KcOm1Zz5+99jZJ3/OAsTzZ9bu5dQyxvC3J19P5f3xFkud2LRva9zJgHEP9TvqM583pnF9en+SvkxzZtWxox7iFxjGK49hcsXTVOzVJJVnfVTb6435VteoF/CdgczO9GXhvjzr7Avs1098P3Ak8p5m/Cli/2Psxhv38KvATQIC/Al692Ps0wH7+CLCumX4OcC+wqpn/OHDqYu/HGPbzIuC0ZvpPgd9c7H1a6H42y44Hfh74y2nlrfh5tnn/hxDjyH8X+/x7ORC4o3k/oJk+oFk2kuM8sAL4BvC85vh7A3DktDpvAf60mT4NuLCZPrKpvx9wRLOeFRMW41rg5lH+/s0zzrXAi4Hzu/8uZvvZ+5rc73mQWJpl3xnz9/KzwNOb6d/s+jsZ2vcySBzD/k7mEc8zu6ZPBj7XTA/tGDdgHEM9jvUTS1PvB4AvAVfT/P8Z5ncy26t1d+aADcB5zfR5wCnTK1TVY1X13WZ2P1p4B5IB9jPJIXR+yb9cnd+m83t9fkL0s59/V1W3N9P3APcDq8cW4XAseD+TBHgFcPFsn58Qc+4nQFVdAfzTuIIaozbs/4JjHOPvYj8xnghsq6oHqupBYBtw0ghi6XYMsKOq7qiqx4ALmli7dcd+MXB8871tAC6oqu9W1TeBHc36JinGcZozzqq6s6puBJ6a9tnF+Nm31SR9z4PEMmz9xPLFqnqkmb0aOLSZHub3Mkgco9BPPP/YNfsMYKonxWEe4waJY9j6OaYC/CGdC5H/0lU2luN+G5OcZ1fVvQDNe8/mM0kOS3IjcBedq7r3dC3+s+bW6/+xCP/A+jXIfq4BdnZV29mUTaK+9nNKkmPoXBn5Rlfxe5rmB+9Lst/oQh3IIPv5LOChqnqiWbxkfp4zaMPPcyZt2P9BYhzX72I/Ma6hc9ybMj2WURzn59rmHnWa7+lhOt9bP59d7BgBjkhyXZL/meSnRhDffOIcxWeXm0n6ngdd3/cl2Z7k6iSDXkSabyyn02nltJDPjioOGO530nc8Sc5I8g06yct/mM9nxxAHDPc4NmcsSY4GDquqPR5L6Oezw7DPsFc4DEm+APxQj0Xv7HcdVXUX8OIkzwEuSXJxVd0HvKGq7k7yA8CfA79G587V2I1qP+k0rdyr6sKiHNww9rNZzyHAfwM2VtXUVbszgb+nk/icA7wDePfCo124Ue3nDCeirf95zmBifp4zacP+jzDGof0uDiHG2WIZ1XG+n/2fqc64/o4HifFe4PCq+naSl9H5n/KCaVfAh2WQ72OijokTbpK+50HXd3hV3ZPkecCVSW6qqm/M+akBY0nyq8B64Gfm+9kRxwHD/U76jqeqPgR8KMmvAL8PbOz3s2OIY9jHsVljSfI04H3Am+b72WGZyGSuql4507Ik9yU5pKrubU56759jXfckuQX4KeDiqrq7Kf+nJP+Dzu3ORUnmRriff8Oet+EPBe7p+cExGMZ+JnkmcCnw+1V1dde6720mv5vkz4DfHWLo8zLC/fwHYFWSfZqr6K3/ec6y7on5ec6kDfs/whiH9rs4hBh3Ai/vmj+UzrNyjPA4vxM4bNo2p+//VJ2dSfYBfhB4oM/PDsOCY6yqAr4LUFXXNle8fwTYvkhxzvbZl0/77FVDiWrpmaTveaC/gakWVlV1R5KrgKPZs5XO0GNJ8ko6F5h+pr73WMswv5dB4hj2d9J3PF0uAD68wM+OJI7m+xnmcWyuWH4AeCFwVXPt/YeArUlOXsB+LEgbm1lupZN507x/dnqFJIcm2b+ZPgA4DrgtyT5JDmrKVwI/B9w8lqjnb8H72ZwQ/lOSY5u7Om/s9fkJ0c9+7gt8Bji/qj41bdkhzXvoPFfT5p9nz/1sTrC+CJw62+cnxJz7OZsW/Txn0ob9X3CMY/xd7CfGy4FXJTmgOf69Crh8xMf5a4B16fTouS+dzkOm9yDXHfupwJXN97YVOC2dniSPANbR6ahq2BYcY5LVSVYANFf619Hp3GEU+olzJj1/9iOKs+0m6XtecCxNDPs10wfROd/5+ihjSafp3P8DnFxV3ReUhvm9LDiOEXwn/cazrmv2tcDtzfQwj3ELjmMEx7FZY6mqh6vqoKpaW1Vr6TzXeHJVbWdcx/0aco8qo37Radd/BZ0f2hXAgU35euCjzfQJwI10epC5EdjUlD8DuLYpuwV4PyPoVWax97Or3s10rtB8EMhi79MA+/mrwOPA9V2vo5plVwI3Nfv634HvX+x9GtF+Po/OAWAH8CmaXkwn7dXPfjbz/y+wC3iUzpWrE9v082zz/g8hxpH/Ls4jxt9o4tgB/HpTNtLjPPAa4O/oHFvf2ZS9m84/b4Dva76XHc339Lyuz76z+dxtjLCH4YXGCPxi853dAHwN+PkR/73MFee/bn73/hn4NnDLbD97X5P/PS80FuAn6Rwbb2jeTx9DLF8A7uN7/4+3juJ7WWgco/hO+ozn/c1x4no6F/de0PXZoR3jFhoHIziOzRXLtLpX0dWb8jC/k5leaTYkSZIkSWqRNjazlCRJkqRlz2ROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWshkTpIkSZJayGROkiRJklrIZE6SJEmSWmifxQ5gNgcddFCtXbt2scOQNGTXXnvtP1TV6sWOYxAen6Slx2OTpEk027FpopO5tWvXsn379sUOQ9KQJflfix3DoDw+SUuPxyZJk2i2Y5PNLCVJkoYoyblJ7k9y87Tyf5/ktiS3JPlPXeVnJtnRLDuxq/ykpmxHks3j3AdJ7TDRd+YkSZJa6OPAB4HzpwqS/CywAXhxVX03ycFN+ZHAacALgOcAX0jyI83HPgScAOwErkmytaq+Pra9kDTxTOYkSZKGqKq+lGTttOLfBLZU1XebOvc35RuAC5rybybZARzTLNtRVXcAJLmgqWsyJ2k3m1lKkiSN3o8AP5XkK0n+Z5J/3ZSvAe7qqrezKZupfC9JNiXZnmT7rl27RhC6pEllMidJkjR6+wAHAMcCbwcuShIgPerWLOV7F1adU1Xrq2r96tWt7oxT0jzZzFKSJGn0dgKfrqoCvprkKeCgpvywrnqHAvc00zOVSxLgnTlJkqRxuAR4BUDTwcm+wD8AW4HTkuyX5AhgHfBV4BpgXZIjkuxLp5OUrYsSuaSJ5Z05SZKkIUrySeDlwEFJdgJnAecC5zbDFTwGbGzu0t2S5CI6HZs8AZxRVU826/kt4HJgBXBuVd0y9p2RNNFM5iRJkoaoql4/w6JfnaH+e4D39Ci/DLhsiKFJWmKWVDK3dvOlI1nvnVteO5L1SloePDZJmkSjOjaBxydpXHxmTpIkSZJayGRO0pKT5PlJru96/WOStyY5MMm2JLc37wc09ZPkA0l2JLkxyUsXex8kSZLmYjInacmpqtuq6qiqOgp4GfAI8BlgM3BFVa0DrmjmAV5Npwe5dcAm4MPjj1qSJGl+TOYkLXXHA9+oqv8FbADOa8rPA05ppjcA51fH1cCqJIeMP1RJkqT+mcxJWupOAz7ZTD+7qu4FaN4PbsrXAHd1fWZnU7aHJJuSbE+yfdeuXSMMWZIkaW4mc5KWrGag3ZOBT81VtUdZ7VVQdU5Vra+q9atXrx5GiJIkSQtmMidpKXs18LWquq+Zv2+q+WTzfn9TvhM4rOtzhwL3jC1KSZKkBTCZk7SUvZ7vNbEE2ApsbKY3Ap/tKn9j06vlscDDU80xJUmSJtWSGjRckqYkeTpwAvDvuoq3ABclOR34FvC6pvwy4DXADjo9X/76GEOVJElakDnvzCU5N8n9SW7usex3k1SSg5r5GcdqSrKxGdvp9iQbp69Lkoapqh6pqmdV1cNdZd+uquOral3z/kBTXlV1RlX9q6p6UVVtX7zIJUmS+tNPM8uPAydNL0xyGJ2r3t/qKu45VlOSA4GzgB8HjgHOmhqsV5IkSZI0f3Mmc1X1JeCBHoveB/wee/b4NtNYTScC26rqgap6ENhGjwRRkiRJktSfBXWAkuRk4O6qumHaopnGauprDKdm3Y7jJEmSJElzmHcy13Qq8E7g/+y1uEdZzVK+d6HjOEmSJEnSnBZyZ+5fAUcANyS5k854TF9L8kPMPFaTYzhJkiRJ0hDNO5mrqpuq6uCqWltVa+kkai+tqr9n5rGaLgdeleSApuOTVzVlkiRJkqQF6Gdogk8CXwaen2RnMz7TTC4D7qAzVtNHgLcANN1//yFwTfN691SX4JIkSZKk+Ztz0PCqev0cy9d2TRdwxgz1zgXOnWd8kiRJkqQeFtSbpSRJkiRpcZnMSZIkSVILmcxJkiQNUZJzk9yf5OYey343SSU5qJlPkg8k2ZHkxiQv7aq7McntzWvjOPdBUjuYzEmSJA3Xx4GTphcmOQw4AfhWV/GrgXXNaxPw4abugcBZwI8DxwBnNT2CS9JuJnOSJElDVFVfAnr12v0+4PeA6irbAJxfHVcDq5IcApwIbKuqB6rqQWAbPRJEScubyZwkSdKIJTkZuLuqbpi2aA1wV9f8zqZspvJe696UZHuS7bt27Rpi1JImncmcJEnSCCV5OvBO4P/stbhHWc1Svndh1TlVtb6q1q9evXrhgUpqHZM5SZKk0fpXwBHADUnuBA4Fvpbkh+jccTusq+6hwD2zlEvSbiZzkiRJI1RVN1XVwVW1tqrW0knUXlpVfw9sBd7Y9Gp5LPBwVd0LXA68KskBTccnr2rKJGk3kzlJkqQhSvJJ4MvA85PsTHL6LNUvA+4AdgAfAd4CUFUPAH8IXNO83t2USdJu+yx2AJIkSUtJVb1+juVru6YLOGOGeucC5w41OElLinfmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iQtSUlWJbk4yd8muTXJTyQ5MMm2JLc37wc0dZPkA0l2JLkxyUsXO35JkqS5mMxJWqreD3yuqn4UeAlwK7AZuKKq1gFXNPMArwbWNa9NwIfHH64kSdL8mMxJWnKSPBP4aeBjAFX1WFU9BGwAzmuqnQec0kxvAM6vjquBVUkOGXPYkiRJ82IyJ2kpeh6wC/izJNcl+WiSZwDPrqp7AZr3g5v6a4C7uj6/synbQ5JNSbYn2b5r167R7oEkSdIcTOYkLUX7AC8FPlxVRwP/zPeaVPaSHmW1V0HVOVW1vqrWr169ejiRSpIkLZDJnKSlaCews6q+0sxfTCe5u2+q+WTzfn9X/cO6Pn8ocM+YYpUkSVoQkzlJS05V/T1wV5LnN0XHA18HtgIbm7KNwGeb6a3AG5teLY8FHp5qjilJkjSp5kzmkpyb5P4kN3eVnd10931jks8kWdW17Myme+/bkpzYVX5SU7YjyWzNnSRpGP498IkkNwJHAf8XsAU4IcntwAnNPMBlwB3ADuAjwFvGH64kSdL87NNHnY8DHwTO7yrbBpxZVU8keS9wJvCOJEcCpwEvAJ4DfCHJjzSf+RCdk6edwDVJtlbV14ezG5K0p6q6HljfY9HxPeoWcMbIg5IkSRqiOe/MVdWXgAemlX2+qp5oZq+m83wJdLr3vqCqvltV36RzlfuY5rWjqu6oqseAC5q6kiRJkqQFGMYzc78B/FUzPVP33n11+w12/S1JkiRJ/RgomUvyTuAJ4BNTRT2q1Szlexfa9bckSZIkzamfZ+Z6SrIR+Dng+OZ5E5i9e2+7/ZYkSZKkIVnQnbkkJwHvAE6uqke6Fm0FTkuyX5IjgHXAV4FrgHVJjkiyL51OUrYOFrokSZIkLV9z3plL8kng5cBBSXYCZ9HpvXI/YFsSgKur6s1VdUuSi+iM5/QEcEZVPdms57eAy4EVwLlVdcsI9keSJEmSloU5k7mqen2P4o/NUv89wHt6lF9GZywnSZIkSdKAhtGbpSRJkiRpzEzmJEmShijJuUnuT3JzV9nZSf42yY1JPpNkVdeyM5PsSHJbkhO7yk9qynYk2Tzu/ZA0+UzmJEmShuvjwEnTyrYBL6yqFwN/R6f/AZIcSadjuBc0n/m/k6xIsgL4EPBq4Ejg9U1dSdrNZE6SJGmIqupLwAPTyj5fVU80s1fTGaYJYANwQVV9t6q+CewAjmleO6rqjqp6DLigqStJu5nMSZIkjddvAH/VTK8B7upatrMpm6l8L0k2JdmeZPuuXbtGEK6kSbXgQcMlSYtr7eZLR7buO7e8dmTrlpazJO+kM3zTJ6aKelQrel9wr17rrKpzgHMA1q9f37OOpKXJZE6SJGkMkmwEfg44vqqmkq6dwGFd1Q4F7mmmZyqXJMBmlpIkSSOX5CTgHcDJVfVI16KtwGlJ9ktyBLAO+CpwDbAuyRFJ9qXTScrWccctabJ5Z06SJGmIknwSeDlwUJKdwFl0eq/cD9iWBODqqnpzVd2S5CLg63SaX55RVU826/kt4HJgBXBuVd0y9p2RNNFM5iRJkoaoql7fo/hjs9R/D/CeHuWXAZcNMTRJS4zNLCVJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJC1JSe5MclOS65Nsb8oOTLItye3N+wFNeZJ8IMmOJDcmeeniRi9JkjQ3kzlJS9nPVtVRVbW+md8MXFFV64ArmnmAVwPrmtcm4MNjj1SSJGmeTOYkLScbgPOa6fOAU7rKz6+Oq4FVSQ5ZjAAlSZL6ZTInaakq4PNJrk2yqSl7dlXdC9C8H9yUrwHu6vrszqZsD0k2JdmeZPuuXbtGGLokSdLc9lnsACRpRI6rqnuSHAxsS/K3s9RNj7Laq6DqHOAcgPXr1++1XJIkaZy8MydpSaqqe5r3+4HPAMcA9001n2ze72+q7wQO6/r4ocA944tWkiRp/uZM5pKcm+T+JDd3lc27R7gkG5v6tyfZOJrdkSRI8owkPzA1DbwKuBnYCkwdfzYCn22mtwJvbI5hxwIPTzXHlCRJmlT93Jn7OHDStLJ59QiX5EDgLODH6VwdP2sqAZSkEXg28NdJbgC+ClxaVZ8DtgAnJLkdOKGZB7gMuAPYAXwEeMv4Q5YkSZqfOZ+Zq6ovJVk7rXgD8PJm+jzgKuAddPUIB1ydZKpHuJcD26rqAYAk2+gkiJ8ceA8kaZqqugN4SY/ybwPH9ygv4IwxhCZJkjQ0C31mbr49wvXVUxzYW5wkSZIk9WPYHaDM1CNcXz3FQae3uKpaX1XrV69ePdTgJEmSJGmpWGgyN98e4ewpTpIkSZKGaKHJ3Hx7hLsceFWSA5qOT17VlEmSJEmSFmDODlCSfJJOByYHJdlJp1fKLcBFSU4HvgW8rql+GfAaOj3CPQL8OkBVPZDkD4FrmnrvnuoMRZIkSZI0f/30Zvn6GRbNq0e4qjoXOHde0UmSJLVMknOBnwPur6oXNmUHAhcCa4E7gV+qqgeTBHg/nYvhjwBvqqqvNZ/ZCPx+s9o/qqrzxrkfkibfsDtAkSRJWu4+jmP0ShoDkzlJkqQhqqovAdMfJ9lAZ2xemvdTusrPr46rgakxek+kGaO3qh4EpsbolaTdTOYkSZJGzzF6JQ2dyZwkSdLicYxeSQtmMidJkjR6jtEraehM5iRJkkbPMXolDd2cQxNIkiSpf47RK2lcTOYkSZKGyDF6JY2LzSwlSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iQtWUlWJLkuyV8280ck+UqS25NcmGTfpny/Zn5Hs3ztYsYtSZLUD5M5SUvZbwO3ds2/F3hfVa0DHgROb8pPBx6sqh8G3tfUkyRJmmgmc5KWpCSHAq8FPtrMB3gFcHFT5TzglGZ6QzNPs/z4pr4kSdLEMpmTtFT9CfB7wFPN/LOAh6rqiWZ+J7CmmV4D3AXQLH+4qb+HJJuSbE+yfdeuXaOMXZIkaU4mc5KWnCQ/B9xfVdd2F/eoWn0s+15B1TlVtb6q1q9evXoIkUqSJC3cQMlckv+Y5JYkNyf5ZJLvs4MBSRPgOODkJHcCF9BpXvknwKok+zR1DgXuaaZ3AocBNMt/EHhgnAFLkiTN14KTuSRrgP8ArK+qFwIrgNOwgwFJi6yqzqyqQ6tqLZ3j0pVV9Qbgi8CpTbWNwGeb6a3NPM3yK6tqrztzkiRJk2TQZpb7APs3V7KfDtyLHQxImlzvAN6WZAedZ+I+1pR/DHhWU/42YPMixSdJktS3feau0ltV3Z3kPwPfAh4FPg9cS58dDCSZ6mDgH7rXm2QTsAng8MMPX2h4kgRAVV0FXNVM3wEc06POvwCvG2tgkiRJAxqkmeUBdO62HQE8B3gG8OoeVe1gQJIkSZKGbJBmlq8EvllVu6rqceDTwE9iBwOSJEmSNHKDJHPfAo5N8vTm2bfjga9jBwOSJEk92RO4pGFacDJXVV+h05HJ14CbmnWdgx0MSJIk7cWewCUN24I7QAGoqrOAs6YV28GAJElSb1M9gZCtUc4AABRkSURBVD/Onj2B/0qz/DzgXcCH6fRN8K6m/GLgg0liyyZJUwYdmkCSJEl9qKq7gamewO8FHmYePYE39Z81fb1JNiXZnmT7rl27RrsTkiaKyZwkSdIY2BO4pGEzmZMkSRoPewKXNFQmc5IkSeNhT+CShspkTpIkaQzsCVzSsA3Um6UkSZL6Z0/gkobJZE6StJe1my8dyXrv3PLakaxXkqTlyGaWkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkpacJN+X5KtJbkhyS5I/aMqPSPKVJLcnuTDJvk35fs38jmb52sWMX5IkqR8mc5KWou8Cr6iqlwBHASclORZ4L/C+qloHPAic3tQ/HXiwqn4YeF9TT5IkaaKZzElacqrjO83syuZVwCuAi5vy84BTmukNzTzN8uOTZEzhSpIkLchAyVySVUkuTvK3SW5N8hNJDkyyrWnGtC3JAU3dJPlA04zpxiQvHc4uSNLekqxIcj1wP7AN+AbwUFU90VTZCaxpptcAdwE0yx8GntVjnZuSbE+yfdeuXaPeBUmSpFkNemfu/cDnqupHgZcAtwKbgSuaZkxXNPMArwbWNa9NwIcH3LYkzaiqnqyqo4BDgWOAH+tVrXnvdReu9iqoOqeq1lfV+tWrVw8vWEmSpAVYcDKX5JnATwMfA6iqx6rqIfZsrjS9GdP5TfOnq4FVSQ5ZcOSS1IfmuHQVcCyd484+zaJDgXua6Z3AYQDN8h8EHhhvpJIkSfMzyJ255wG7gD9Lcl2SjyZ5BvDsqroXoHk/uKm/uxlTo7uJ0242Y5I0qCSrk6xqpvcHXkmn5cAXgVObahuBzzbTW5t5muVXVtVed+YkSZImySDJ3D7AS4EPV9XRwD/zvSaVvdiMSdK4HAJ8McmNwDXAtqr6S+AdwNuS7KDzTNzHmvofA57VlL+N2Y9lkiRJE2GfuavMaCews6q+0sxfTOcE6L4kh1TVvU0zyvu76h/W9fnuJk6SNDRVdSNwdI/yO+g8Pze9/F+A140hNEnLXNNq4KPAC+lc1P4N4DbgQmAtcCfwS1X1YNOr7vuB1wCPAG+qqq8tQtiSJtSC78xV1d8DdyV5flN0PPB19myuNL0Z0xubXi2PBR6eao4pSZK0TNh5nKShGeTOHMC/Bz6RZF/gDuDX6SSIFyU5HfgW37vafRmdK0s76Fxd+vUBty1JktQaXZ3HvQk6nccBjyXZALy8qXYenU6b3kFX53HA1c2QUId4MVzSlIGSuaq6HljfY9HxPeoWcMYg25MkSWqx7s7jXgJcC/w20zqPSzJX53F7JHNJNtG5c8fhhx8+0h2QNFkGHWdOkiRJ/bHzOElDZTInSZI0Hr06j3spTedxAHYeJ2k+TOYkSZLGwM7jJA3boB2gSJIkqX92HidpaEzmJEmSxsTO4yQNk80sJUmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkSZIkqYVM5iRJkiSphUzmJEmSJKmFTOYkLTlJDkvyxSS3JrklyW835Qcm2Zbk9ub9gKY8ST6QZEeSG5O8dHH3QJIkaW4mc5KWoieA36mqHwOOBc5IciSwGbiiqtYBVzTzAK8G1jWvTcCHxx+yJEnS/AyczCVZkeS6JH/ZzB+R5CvNle8Lk+zblO/XzO9olq8ddNuS1EtV3VtVX2um/wm4FVgDbADOa6qdB5zSTG8Azq+Oq4FVSQ4Zc9iSJEnzss8Q1vHbdE6UntnMvxd4X1VdkORPgdPpXOU+HXiwqn44yWlNvV8ewvZHbu3mS0ey3ju3vHYk65X0Pc2Fo6OBrwDPrqp7oZPwJTm4qbYGuKvrYzubsnunrWsTnTt3HH744SONW5IkaS4DJXNJDgVeC7wHeFuSAK8AfqWpch7wLjrJ3IZmGuBi4INJUlU1SAySNJMk3w/8OfDWqvrHziGqd9UeZXsdm6rqHOAcgPXr13vsWoBRXRwDL5BJkpafQZtZ/gnwe8BTzfyzgIeq6olmfurqNnRd+W6WP9zU30OSTUm2J9m+a9euAcOTtFwlWUknkftEVX26Kb5vqvlk835/U74TOKzr44cC94wrVknLi4+oSBqWBSdzSX4OuL+qru0u7lG1+lj2vYKqc6pqfVWtX7169ULDk7SMNa0EPgbcWlX/tWvRVmBjM70R+GxX+RubXi2PBR6eao4pSSMw9YjKlKlHVNYBD9J5NAW6HlEB3tfUk6TdBrkzdxxwcpI7gQvoNK/8EzodB0w13+y+ur37ynez/AeBBwbYviTN5Djg14BXJLm+eb0G2AKckOR24IRmHuAy4A5gB/AR4C2LELOkZaDrEZWPNvNTj6hc3FSZ3jnTVKdNFwPHZ5b24pKWnwU/M1dVZwJnAiR5OfC7VfWGJJ8CTqWT4E2/8r0R+HKz/Eqfl5M0ClX11/RuDQBwfI/6BZwx0qAkqWPqEZUfaOb7fkQlydQjKv/QvUI7Z5KWr2H0ZjndO4ALkvwRcB2dpk407/8tyQ46d+ROG8G2JUmSJlL3IyrNhXAY0iMqTFjnTPYELo3HUJK5qroKuKqZvgM4pkedfwFeN4ztSZIktdDUIyqvAb6PzrBOux9Rae7O9XpEZaePqEjqZeBBwyVJkjS3qjqzqg6tqrV0WihdWVVvAL5I5xEU6P2ICviIiqQeTOYkSZIW1zvojNe7g84zcd2PqDyrKX8bsHmR4pM0oUbxzJwkSZJm4SMqkobBO3OSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRCJnOSJEmS1EImc5IkSZLUQiZzkiRJktRC+yx2AJIkDcPazZeOZL13bnntSNYrSdKgvDMnSZIkSS1kMidJkiRJLWQyJ0mSJEktZDInaclJcm6S+5Pc3FV2YJJtSW5v3g9oypPkA0l2JLkxyUsXL3JJkqT+LTiZS3JYki8muTXJLUl+uyn3hEnSYvs4cNK0ss3AFVW1DriimQd4NbCueW0CPjymGCVJkgYyyJ25J4DfqaofA44FzkhyJJ4wSVpkVfUl4IFpxRuA85rp84BTusrPr46rgVVJDhlPpJIkSQu34GSuqu6tqq810/8E3AqswRMmSZPp2VV1L3SOX8DBTfka4K6uejubsr0k2ZRke5Ltu3btGmmwkpYeWzVJGrahPDOXZC1wNPAVBjxh8mRJ0pilR1n1qlhV51TV+qpav3r16hGHJWkJslWTpKEaOJlL8v3AnwNvrap/nK1qj7K9Tpg8WZI0IvdNtQZo3u9vyncCh3XVOxS4Z8yxSVoGbNUkadgGSuaSrKSTyH2iqj7dFHvCJGkSbQU2NtMbgc92lb+xac50LPDwVOsCSRoVWzVJGoZBerMM8DHg1qr6r12LPGGStKiSfBL4MvD8JDuTnA5sAU5IcjtwQjMPcBlwB7AD+AjwlkUIWdIyYqsmScOyzwCfPQ74NeCmJNc3Zf87nROki5qTp28Br2uWXQa8hs4J0yPArw+wbUmaUVW9foZFx/eoW8AZo41Ikjpma9VUVffaqknSfCw4mauqv6b3FSPwhEmSJGkPfbRq2sLerZp+K8kFwI9jqyZJ0wxyZ06SJEn9s1WTpKEymZMkSRoDWzVJGrahjDMnSZIkSRovkzlJkiRJaiGTOUmSJElqIZM5SZIkSWohkzlJkiRJaiGTOUmSJElqIZM5SZIkSWohx5mTJGkWazdfOpL13rnltSNZryRp+fDOnCRJkiS1kMmcJEmSJLWQyZwkSZIktZDJnCRJkiS1kB2gLKJRPVQPPlgvSZIkLXXemZMkSZKkFjKZkyRJkqQWMpmTJEmSpBYymZMkSZKkFjKZkyRJkqQWMpmTJEmSpBYymZMkSZKkFnKcOUmSFoFjjUrz59+NtKexJ3NJTgLeD6wAPlpVW8Ydw3IwqoOdBzotVR6bJE0qj0+SZjLWZC7JCuBDwAnATuCaJFur6uvjjEML5xUxLUUem7TUjPJY3TZt/9/i8Wl8vBCuNhr3nbljgB1VdQdAkguADYAHJLXu5KONB2f/Uc3IY5OkSeXxqeX836tRGncytwa4q2t+J/Dj3RWSbAI2NbPfSXLbPNZ/EPAPA0XYPu7zIsl7x7q5idjnmSzgu3juCMIYxJzHJhjo+LTYP7/lvv1JiMHtL9L2u45P/cQwaccmGO2502L/XgxD2/dhwfGP+TxkNsv2ZzBGMx6bxp3MpUdZ7TFTdQ5wzoJWnmyvqvUL+Wxbuc/Lw3Lc5zGb89gECz8+LfbPb7lvfxJicPv+DgxgZOdOLf5Odmv7PrQ9fmj/PrQ9/nEPTbATOKxr/lDgnjHHIEnTeWySNKk8Pkma0biTuWuAdUmOSLIvcBqwdcwxSNJ0HpskTSqPT5JmNNZmllX1RJLfAi6n073uuVV1yxA3saDmmS3nPi8Py3Gfx2YZHJuW+/Zh8WNw+4tvEmKYtxEfn1r5nUzT9n1oe/zQ/n1odfyp2uuxEEmSJEnShBt3M0tJkiRJ0hCYzEmSJElSC7UumUtyUpLbkuxIsrnH8v2SXNgs/0qSteOPcrj62Oe3Jfl6khuTXJFkEsfJmZe59rmr3qlJKklru5Sd0s8+J/ml5md9S5L/Me4YtbdBjklJzmzKb0ty4ji3n2RtkkeTXN+8/nRE2//pJF9L8kSSU6ct25jk9ua1cRG2/2TX/i+oQ4lBjs/D2P8hxDCO7+DNSW5qtvHXSY7sWjaOv4Ge2x/W30AbDHKcmhSD/J5Pgrni76o3sec1/exDJvg8pY/focOTfDHJdc3v0WsWI855q6rWvOg8+PsN4HnAvsANwJHT6rwF+NNm+jTgwsWOewz7/LPA05vp31wO+9zU+wHgS8DVwPrFjnsMP+d1wHXAAc38wYsd93J/DXJMAo5s6u8HHNGsZ8UYt78WuHkM+78WeDFwPnBqV/mBwB3N+wHN9AHj2n6z7Dtj2P+ex+dh7P+gMYzxO3hm1/TJwOfG/Dcw0/YH/htow2uQ48SkvAb9PV/sVz/xN/Um9rymz5/BxJ6n9Bn/OcBvNtNHAncudtz9vNp2Z+4YYEdV3VFVjwEXABum1dkAnNdMXwwcn6TXgJttMec+V9UXq+qRZvZqOmPQtFk/P2eAPwT+E/Av4wxuRPrZ5/8N+FBVPQhQVfePOUbtbZBj0gbggqr6blV9E9jRrG9c2x+Gfo5Pd1bVjcBT0z57IrCtqh5ofqe3ASeNcfvDMMjxeRj7P2gMw9DP9v+xa/YZfG/A67H8Dcyy/eVisY8Tw7DYv+eDWgrnNW0/T+kn/gKe2Uz/IC0Zz7Ftydwa4K6u+Z1NWc86VfUE8DDwrLFENxr97HO304H/v73zeZGjiOL450nAXDwkbm4mYCALcXNZoqCC6EFEhYjgRSVgNMeg/0A8iHcVD7kneJF40SDmIjpBgkGirIQI/tjVQ4x42JCLB/HwPFRtmExm3Zrtququ8fuBYnqW6nnfV931+lV1V+/5oorKs6XPZrYM7HX3T2sKK0jKcV4EFs3sopldMrPtJH4iL11i0qx9O7d9gPvj4yQXzOyxGW2n2i+xb67f2Glml2N/en5G29uxPx6fc/jfVQNUagMzO2Fmq4RE9Y1tas9tH7r3gRaYh9yp9VxoHvKa1vOUFP1vAUfN7BrwGfB6HWndqPp/5jIwbZZocoYtpU5LJPtjZkeBB4HHiyoqz3/6bGZ3Ae8Bx2oJqkDKcd5BeIThCcKM41dmdsjdbxbWJjanS0zKEau62P8D2Ofu62Z2GPjYzJYm7mLksF9i31y/sc/dr5vZfuALM7vi7qsl7E+Jz7muVV2vEVXawN1PAafM7GXgTeCVWbQXsp+jD7TAPOROredC85DXtJ6npOh/CTjt7u+Y2SPAB1F/iSc7stHanblrwN6x7/dx5y3QW3XMbAfhNumNKurKkOIzZvYkcBJ4zt3/rqStFFv5fA9wCBiZ2W/Aw8C5IS4WnoHUc/sTd/8nPpL0IyFoiv7oEpOS+nYp+/HRtnUAd/+WsJZgsYD9Evtm+Q13vx4/14ARsFzC/ibxOYf/XTVUa4MxPgQ27gD2cQ7csp+pD7TAPOROredC85DXtJ6npOg/DpwFcPevgZ3AQhV1XaixMC9XIYz41wgLpTcWLy5N1DnB7Yt4z/atu4LPy4SL0IG+9dbyeaL+iIEtFC50nJ8GzsTtBcLjAvf2rf3/XLrEJGCJ21/+sMbsL3/oYn/Phj3CgvDfgd257Y/VPc2dL0D5lfDyj11xu6b9XcDdcXsB+JkpLyTI0P5T43MO/zNoqNUGB8a2jwCXK/eBzex37gMtlMQ2GnTu1OU8H0KZJVbF+iMGltckHoPB5imJ+s8Dx+L2QcJgz/rWvqVvfQvYxsF4FvgpdtiT8W9vE2ZhIIyiPyIspP4G2N+35go+fw78CazEcq5vzaV9nqg7uKBX6Dgb8C7wA3AFeLFvzSrdYhJhBnmVMHv5TE37wAvA1XhB+w44Usj+Q4QZ0b+AdeDq2L6vRV2/AK/WtA88GvvR9/HzeCH7m8bnHP530VCxDd6P59oK8CVjCVSlPjDVfq4+0EJJaKPB505d+toQylb6J+qOGGBek3AMBp2nJOh/ALgYY8IK8FTfmlOKRfFCCCGEEEIIIRqitTVzQgghhBBCCCHQYE4IIYQQQgghmkSDOSGEEEIIIYRoEA3mhBBCCCGEEKJBNJgTQgghhBBCiAbRYE4IIYQQQgghGkSDOSGEEEIIIYRokH8Bxot3aDabC+UAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots(2,3,figsize=(15,10))\n", "for i in range(3):\n", " ax[0,i].scatter(molecules_y_test[:,i],molecules_y_pred[:,i])\n", " ax[1,i].hist(abs(molecules_y_test[:,i]-molecules_y_pred[:,i]))\n", " ax[0,i].set_title(outputs[i])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Classification\n", "\n", "In classification, the target output is categorical instead of numerical. For example, we may predict a tumor as benign or malignant, or a piece of news in sports, economy, politics. In this part, we will use a neural network to predict the class of a flower named iris.\n", "\n", " " ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_iris\n", "\n", "X_iris,y_iris = datasets.load_iris(return_X_y=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we split the data directly selecting a contiguous subset, we will likely choose a bised sample. Some classes may not be present in the train or test sets. So we will use s different spliting schema. " ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "X_iris_train, X_iris_test, y_iris_train, y_iris_test = train_test_split(X_iris, y_iris, test_size = 0.20)\n" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "from sklearn.neural_network import MLPClassifier\n", "mlp = MLPClassifier(hidden_layer_sizes=(10, 10, 10), max_iter=1000)\n", "mlp.fit(X_iris_train, y_iris_train)\n", "\n", "y_iris_pred = mlp.predict(X_iris_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One way to evaluated the prediction, we could tabulate the correct and incorrect classification in a confusion matrix:\n", "\n", "\n", "\n", "where\n", "\n", "- True Positive:\n", "\n", " Interpretation: You predicted positive and it’s true.\n", " You predicted that a woman is pregnant and she actually is.\n", "\n", "- True Negative:\n", "\n", " Interpretation: You predicted negative and it’s true.\n", " You predicted that a man is not pregnant and he actually is not.\n", "\n", "- False Positive: (Type I Error)\n", "\n", " Interpretation: You predicted positive and it’s false.\n", " You predicted that a man is pregnant but he actually is not.\n", "\n", "- False Negative: (Type II Error)\n", "\n", " Interpretation: You predicted negative and it’s false.\n", " You predicted that a woman is not pregnant but she actually is." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[13 0 0]\n", " [ 0 5 1]\n", " [ 0 0 11]]\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVwAAAEGCAYAAAApAy29AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7wd873/8dc7CYJEgh2pxCVuVcQ1oYimiubQupUQraq0VYfetfRwqj/pDaU3dTmaapsqpRK0Gk4jtKhWVC4uCaGoIILkJ3UNuX3OH/Pd6crOTvZae689a/bk/Xw85pE1s2Z957PG9tnf/Z35fkYRgZmZdb5ujQ7AzGxt4YRrZpYTJ1wzs5w44ZqZ5cQJ18wsJz0aHUBXpB7rh9bt3egwCmvPnbZqdAjWxc2Z8wwLFixQR9rovtHWEUsXVbVvLJo/KSIO7cjxquGE2w5atzfr7Xh8o8MorL/ef1mjQ7Aubth7h3a4jVi6qOr/T99+8PKmDh+wCk64ZlZSAhVr1NQJ18zKSUC37o2OYiVOuGZWXurQMHDdOeGaWUl5SMHMLD/u4ZqZ5UC4h2tmlg+5h2tmlhvfpWBmlgdfNDMzy4fwkIKZWW7cwzUzy0PxhhSKFY2ZWb0I6N69uqWtpqRfSHpZ0syKbRdLmi3pYUk3S+rbVjtOuGZWXlJ1S9vGAS3LN04GBkfEbsATwDltNeKEa2YllYYUqlnaEBH3AK+02HZ7RCxNq1OALdpqx2O4ZlZe1d+l0CRpasX62IgYW8ORPgX8tq2dnHDNrLyqv2i2ICLaVfVc0teBpcC1be3rhGtm5VT9+GwHDqGTgcOBgyMi2trfCdfMyqsTp/ZKOhT4L+D9EfFWVeF0WjRmZg1Vv4tmkq4D7gN2lPS8pE8DlwG9gcmSHpR0ZVvtuIdrZuVVpyGFiPhoK5t/Xms7TrhmVk6uh2tmlpfiTe11wjWz8nI9XDOznLg8o5lZDuQhBTOz/LiHa2aWDznhmpl1vuwJO064ZmadT0LdnHCtAy79xon8xwGDWbDwdfY/4XwA/vu0D/Oh4buxPIL5r7zO5755DS8ueLXBkRbDHX97lHN+MIFly5dz0lH7c8boEY0OqXDKfI6K1sMt1iW8dpI0WtKARseRh+smTmHkFy9fadulv76TAz52AcNPvJBJ987ka6cc1qDoimXZsuWcddENjL/ks0y54VxuvH0as5+e1+iwCqXs50hSVUteSpFwgdHAWpFw/zbjKRa+tnJhotfffHvF6w3XX48qqsStFabNeoZtt2xi0BZNrLtOD4754F7cdvfDjQ6rUMp+jpxwqyRpQ0m3SnpI0kxJoyQNkXS3pGmSJknaXNJIYChwbarYs76kgyXNkPRIevjbeqnNCyU9mh769v207QhJ96f975DUv5Hfu73OPf0IZk78NscdOpTzf3pro8MphHnzX2Vg/41XrA/ovzHz5nuopVKpz5FqWHJS2IRL9sC2FyJi94gYDPwRuBQYGRFDgF8A342ICcBU4MSI2AMIsge+jYqIXcnGqU+XtAnwEWCX9NC376Tj3AvsGxF7AtcDX8vtG9bRd/7nDww+/BuM/+NUPnP88EaHUwit9fQLNqTXcGU+R6K63q17uJlHgEMkfU/S+4AtgcGk2pPAubT+0LYdgX9GxBNp/VfAcOA14G3gKknHAM1/l28BTJL0CHAWsEtrwUg6VdJUSVNj6aL6fMNOMOGPD3DkQXs0OoxCGLBZX+a+tHDF+gsvLeRdTX0aGFHxlP0cdevWraolt3hyO1KNUsIcQpZ4LwCOBWZFxB5p2TUiWruc2uqvq/R0zX2AG4GjyXrMkPWaL0u94f8Eeq7m82MjYmhEDFWP9Tvy1epu2y37rXh96PDdeOKZlxoYTXHstfPWPPXsfObMXcDiJUu5afJ0Dhu+W6PDKpSyn6Oi9XALe1tYuuvglYi4RtIbwKlAP0n7RcR9ktYB3h0Rs4DXySqvA8wGBknaPiKeBE4C7pbUC9ggIm6TNAV4Mu3fB5ibXp+c09drt6u+M5phQ3Zg0769mDnx21w49jY+OGwXdth6M5YvD5578RW+csH1jQ6zEHr06M5FXzueY794OcuWBSceuS87bbd5o8MqlFKfo5zHZ6tR2IQL7ApcLGk5sAQ4nezJmD+R1Ics9h8Ds8jGbK+UtAjYD/gkMF5SD+AB4EpgE+D3knqS/Wc4Ix1nTNp3Ltmz5bfJ5du10ynnjltl2zW33Jd/IF3EiGG7MGJYq6NElpT5HBXtPtzCJtyImARMauWtVa4IRcSNZEMFze4E9myx2zyyIYWWn/098Pv2R2pmRdR80axICptwzcw6ylN7zczyIA8pmJnlxgnXzCwnTrhmZjko4kWzwk58MDPrsDrVUkg1WV6WNLNi2yaSJkv6R/p34zW1AU64ZlZWquvU3nFk9V0qnQ3cGRE7kN2KenZbjTjhmllp1Wtqb0TcA7zSYvNRZLVaSP8e3VY7HsM1s/Lq3CHc/hExDyAi5knarK0POOGaWWnVcNGsSdLUivWxETG23vE44ZpZKdVYCWxBRAyt8RAvSdo89W43B15u6wMewzWz0urk8oy38O8KgydTRU0W93DNrLTqVUtB0nXAgWRDD88D5wEXAjdI+jTwLHBcW+044ZpZadVr4kNEfHQ1bx1cSztOuGZWTi5eY2aWD1G8B2I64ZpZSRWvloITrpmVVjcXIDczy4E8pGBmlgvhHq6ZWW7cwzUzy4kvmpmZ5cFjuGZm+RCqtrh4bpxwzay03MM1M8uJx3DNzPLgMVwzs3xktRSKlXGdcM2stAqWb51wzay8PNPMzCwProdbDnvutBV/vf+yRodRWL+eOqfRIRTekTsPaHQIhbZ0eXS4DdfDNTPLjevhmpnlpmD51gnXzEpKvmhmZpYL34drZpYjJ1wzs5wULN864ZpZeRWth1usYpFmZvWSitdUs1TVnHSGpFmSZkq6TlLPWkNywjWzUsoKkFe3tNmWNBD4IjA0IgYD3YETao3JQwpmVlrd6juk0ANYX9ISYAPghZrjqWc0ZmZFUsOQQpOkqRXLqZXtRMRc4PvAs8A84NWIuL3WeNzDNbNSUm3FaxZExNDVt6WNgaOAbYB/AeMlfTwirqklptUmXEkbremDEfFaLQcyM8tbHSeaHQL8MyLmA0i6CdgfqE/CBWYBQTZho1nzegBb1XIgM7O81XFq77PAvpI2ABYBBwNTa21ktQk3IrZsf2xmZo0lsjsV6iEi7pc0AZgOLAVmAGNrbaeqMVxJJwDbRsT5krYA+kfEtFoPZmaWp3rWromI84DzOtJGm3cpSLoM+ABwUtr0FnBlRw5qZtbplNXDrWbJSzU93P0jYi9JMwAi4hVJ63ZyXGZmHVawmb1VJdwlkrqRXShD0qbA8k6Nysysg0TdJz50WDUJ93LgRqCfpG8CxwPf7NSozMzqoMsVII+IqyVNI7sPDeC4iJjZuWGZmXVMLYVp8lLtTLPuwBKyYQVPBzazLqFoQwrV3KXwdeA6YACwBfAbSed0dmBmZh2lKpe8VNPD/TgwJCLeApD0XWAacEFnBmZm1lFFK0BeTcKd02K/HsDTnROOmVl9ZHcpNDqKla2peM2PyMZs3wJmSZqU1kcA9+YTnplZO6m64uJ5WlMPt/lOhFnArRXbp3ReOGZm9dNlhhQi4ud5BmJmVk9dakihmaTtgO8COwMrHpoWEe/uxLjMzDqsaD3cau6pHQf8kuwXxmHADcD1nRiTmVldFO22sGoS7gYRMQkgIp6KiHPJqoeZmRWWBN27qaolL9XcFvaOsn75U5JOA+YCm3VuWFatO/72KOf8YALLli/npKP254zRIxodUqGM+fpPWa/nuulx2N0465xPNDqkQjnrwuv4032PsunGvbh93H81Opy6K9qQQjUJ9wygF9kz2b8L9AE+1ZlBtUbSt4B7IuKOGj93IHBmRBzeKYE10LJlyznrohu4+bLPM6B/Xw46+WIOG74r79l280aHVihfOGMUvXpt0OgwCmnkYftw8jEH8JXzf9PoUDpFwfJtVcVr7k8vX+ffRcg7RepJKyJWKf8YEf+vM49dEUOPiFiax7E6atqsZ9h2yyYGbdEEwDEf3Ivb7n7YCdeq9t7dt+O5ea80OoxOIVS4WgprmvhwM6kGbmsi4pg1fPZ7wJyIuCKtjyFL2N3IyjuuB9wcEedJGgT8L/BnYD/g6FQGcmg6/i8i4keSxgETI2KCpL2BS4ANgXfIHui2BPif9LmlwFci4s8t4toE+AWwLdmEjlMj4uEU3wBgELAA+NjqvluRzJv/KgP7b7xifUD/jZk285nGBVREElf8ZDwghr1vd4a9b/dGR2R56WLVwi7rQLvXAz8GrkjrxwMXAgcA+5BdGLxF0nCyp2HuCHwyIj4raQgwMCIGA0jqW9lwetrEb4FREfFAepz7IuBLABGxq6T3ALdLannr2jeBGRFxtKSDgKuBPdJ7Q4ADImJRa19I0qnAqQBbblWMBxZHrPr7sGg/YI12xpkfo0/fXrz+2ptc/pPx9H/XJmy/g5+PurboMmO4EXFnexuNiBmSNpM0AOgHLAR2I5sWPCPt1gvYgSzhzomI5hlsTwPbSrqUbIbb7S2a3xGYFxEPpGO9BiDpAODStG22pDlAy4R7AHBs2udPkjaV1Ce9d8vqkm3afyzpKZ1Dhgxdbc8/TwM268vclxauWH/hpYW8q6nPGj6x9unTtxcAvTfakN322IE5z8xzwl1LCOhesITbmbVtJwAjgVFkPV4BF0TEHmnZvmI225vNH4qIhcDuwF3A54CrWrQrWh/qqObMtrZPc1tvtvJeoe2189Y89ex85sxdwOIlS7lp8nQOG75bo8MqjHfeWczbby9e8Xr2Y8+w+YB+DY7K8tRN1S15qbYAeXtcD/wMaALeD+wKfFvStRHxhqSBZOOuK5HUBCyOiBslPUU28aLSbGCApL3TkEJvsiGFe4ATgT+loYStgMfJxoWbNe/z7XT3woKIeK1of3ZUq0eP7lz0teM59ouXs2xZcOKR+7LTdr5g1uz1197iqp/+DoDly5czZO+d2HmXbRocVbF84ZtXM+XBJ1n46pvsO3IMZ3zyUEZ9eN9Gh1U3XW5qbzNJ60XEO9XuHxGzUjKcGxHzgHmSdgLuSwnuDbJau8tafHQg8Mv04EqAlYqdR8RiSaOASyWtT5ZsDyEbL75S0iNkF81GR8Q7LZLpmNT2w2QXzU6u9vsU1YhhuzBi2C6NDqOQmvr15exzRzc6jEK79Lzy3pecPWKnWBm3mloK+wA/J7v/ditJuwOnRMQX2vpsROzaYv0SsrsLWhpcsc9DwF6ttDW64vUDQGu/hke33BARd5ENTxARrwBHtbLPmNbiN7OurZ493HQB/yqyfBXApyLivpriqWKfnwCHA/8fViRET+01s8JrfpBkW0uVLgH+GBHvIbvO9Fit8VQzpNAtIua06Jq3HAYwMysUAT3qNKSQbj8dTvorOiIWA4trbaeaHu5zaVghJHWX9GXgiVoPZGaWtxp6uE2SplYsp7ZoaltgPtk1oBmSrpK0Ya3xVNPDPZ1sWGEr4CXgjrTNzKywpJqm9i6IiKFreL8H2bWlL0TE/ZIuAc4GvlFLTNXUUngZOKGWRs3MiqCONyk8DzxfUVtmAlnCrUk1dyn8jFYmGkREyy63mVmh1OsuhYh4UdJzknaMiMfJ6rc8Wms71QwpVJZD7Al8BHiu1gOZmeVJUO/i4l8Ark31XJ4GPllrA9UMKfy2cl3Sr4HJtR7IzCxXdZ62GxEPklUjbLf2TO3dBti6Iwc1M8uDcn1iWduqGcNdyL/HcLsBr9COwWIzszx1ucekpycw7E72HDOA5dFaEVYzswIqWsJd48SHlFxvjohlaXGyNbMuQ1JVS16qmWn2d0mrFJMxMyuy7DHp1S15WdMzzZofpngA8JlUm/ZNUgHwiHASNrNC6zIPkQT+TjaV7eicYjEzq5uudtFMABHxVE6xmJnVVcE6uGtMuP0kfWV1b0bEDzshHjOzOhHdutB9uN3JnqxbrIjNzKogulYPd15EfCu3SMzM6knQo2CDuG2O4ZqZdUVdrYd7cG5RmJl1gi5zW1h6wq2ZWZdVsHzbrmphZmaFJ6qbSpsnJ1wzKyd1oSEFM7OuLJtp5oRrZpaLYqVbJ1wzK7GCdXCdcM2srPKtdVsNJ1wzKyXfpWBmliNfNLPSO2moH+rcls1OurrRIRTam3PqMO9KeEjBzCwPHlIwM8tR0Xq4RfsFYGZWN6pyqaotqbukGZImtjce93DNrJQEdK9vD/dLwGPARu1twD1cMystqbql7Xa0BfBh4KqOxOMerpmVlFD1k3ubJE2tWB8bEWMr1n8MfA3o3ZGInHDNrLRqGFFYEBFDW29DhwMvR8Q0SQd2JB4nXDMrpey2sLqM4Q4DjpT0IaAnsJGkayLi47U25DFcMyunKsdv2+oFR8Q5EbFFRAwCTgD+1J5kC+7hmlmJeWqvmVkOsgLk9W0zIu4C7mrv551wzay0arhLIRdOuGZWWgUbUXDCNbPycg/XzCwHnTGG21FOuGZWTpLvUjAzy0ux0q0TrpmVVDakUKyU64RrZqVVrHTrhGtmZVawjOuEa2al5SEFM7OcFCvdOuGaWZkVLOM64ZpZKWUPiCxWxnXCNbNyqvJ5ZXlywjWz0ipYvnXCNbOyEipYF9cJ18xKq2D51gnXzMpJeEjBzCw/Bcu4TrhmVlpFuy3Mj0nv4u7426Psfey32OsjY/jRuNsbHU7h+Pys6sen7M+sy4/j7guOWLHtiH225p4LjuTFX53E7tts2sDo6qsej0mvp4YnXEkDJE1ox+eukrRzG/ucJukT7Y+u2JYtW85ZF93A+Es+y5QbzuXG26cx++l5jQ6rMHx+Wnf9X57khIvuXGnb7Of/xScvuYv7Hn+pQVF1giqTbZ4Jt+FDChHxAjCy5XZJPSJi6Ro+d0oVbV/ZwfAKbdqsZ9h2yyYGbdEEwDEf3Ivb7n6Y92y7eYMjKwafn9ZNefxltmzacKVt/3jh1QZF07nW6iEFSd+T9NmK9TGSvippZlofLWm8pD8At0vqJukKSbMkTZR0m6SRad+7JA1Nr9+Q9F1JD0maIql/RftnptfbS7oj7TNd0naSekm6M60/IumoPM9HR82b/yoD+2+8Yn1A/42ZN7+c/+O0h8/P2k0Ur4eb95DC9cCoivXjgQda7LMfcHJEHAQcAwwCdgVOSe+1ZkNgSkTsDtwDfKaVfa4FLk/77A/MA94GPhIRewEfAH6got0pvQYRscq2rhN95/P5MVW5tNmOtKWkP0t6LHUAv9SeeHIdUoiIGZI2kzQA6AcsBJ5tsdvkiHglvT4AGB8Ry4EXJf15NU0vBiam19OAD1a+Kak3MDAibk5xvJ22rwOcL2k4sBwYCPQHXmx5AEmnAqcCbLnVVtV/6U40YLO+zH1p4Yr1F15ayLua+jQwomLx+bE6jigsBb4aEdNTPpkmaXJEPFpLI424aDaBbMx2FFmPt6U3K15Xe7qWxL+7M8tY9RfJ6to5kSzxD4mIPYCXgJ6t7RgRYyNiaEQM7dfUr8qwOtdeO2/NU8/OZ87cBSxespSbJk/nsOG7NTqswvD5sW7pyb1tLW2JiHkRMT29fh14jKyDVpNGXDS7HvgZ0AS8H1hvDfveC5ws6VdkifFA4De1HjAiXpP0vKSjI+J3ktYDugN9gJcjYomkDwBb19p2I/Xo0Z2LvnY8x37xcpYtC048cl922m7tviBUyeendVd+9n0M26k/m/TqyYOXHMtFNz3Ev954h/M/sQ+b9u7Jb756EDPnLGTUxXc0OtQOq6GD2yRpasX62IgY22qb0iBgT+D+WuPJPeFGxKzUJZ8bEfNS8KtzI3AwMBN4guwLtveqx0nATyV9C1gCHEc2rvuHdKIfBGa3s+2GGTFsF0YM26XRYRSWz8+qTrviL61uv23aczlHkoPqM+6CiBjaZnNSL7K89OWIeK3WcBpyW1hE7Frx+hlgcHo9DhhX8d5ySWdGxBuSNgX+DjyS3juwYr9eFa8nkA1bEBFjKrb/AziolXBWdyHOzLqwehcgT9d8bgSujYib2tNGw+/DrcJESX2BdYFvR8QqF7TMzFZRx1u+0t1LPwcei4gftredwifcyp6smVkt6ngX4DCyYclHJD2Ytv13RNxWSyOFT7hmZu1TvwLkEXEvdcjfTrhmVlpFm+jihGtmpeQC5GZmeSpYxnXCNbPSKlq1MCdcMystj+GameVB0M0J18wsL8XKuE64ZlZKzQXIi8QJ18xKq2D51gnXzMrLPVwzs5wU7YlZTrhmVlrFSrdOuGZWUnk/kbcaTrhmVlqeaWZmlpdi5VsnXDMrr4LlWydcMyur6h6BnicnXDMrpSLONOvW6ADMzNYW7uGaWWkVrYfrhGtmpeXbwszM8uCJD2Zm+SjiRTMnXDMrLQ8pmJnlpGg9XN8WZmalpSqXqtqSDpX0uKQnJZ3dnniccM2svOqUcSV1By4HDgN2Bj4qaedaw3HCNbNSEtBNqmqpwj7AkxHxdEQsBq4Hjqo1Jo/htsP06dMWrL+O5jQ6jgpNwIJGB1FwPkdrVrTzs3VHG5g+fdqk9ddRU5W795Q0tWJ9bESMrVgfCDxXsf488N5aY3LCbYeI6NfoGCpJmhoRQxsdR5H5HK1ZGc9PRBxax+Za6wZHrY14SMHMrG3PA1tWrG8BvFBrI064ZmZtewDYQdI2ktYFTgBuqbURDymUw9i2d1nr+Rytmc/PGkTEUkmfByYB3YFfRMSsWttRRM3DEGZm1g4eUjAzy4kTrplZTpxwuxhJoyUNaHQcXYGkb0k6pB2fO1DSxM6IqbNIGiBpQjs+d1VbM6YknSbpE+2Pzpp5DLeLkXQXcGZETG1r37WBJJH9HC+vY5sHkp3jw6vcv0dELK3X8eupyLGtjdzDLQBJG0q6VdJDkmZKGiVpiKS7JU2TNEnS5pJGAkOBayU9KGl9SQdLmiHpEUm/kLReavNCSY9KeljS99O2IyTdn/a/Q1L/Rn7vSpK+J+mzFetjJH1V0lmSHkjf45vpvUGSHpN0BTAd2FLSuHTuHpF0RtpvXDpnSNpb0t/SOf67pN6Sekr6ZfrMDEkfaCWuTST9Lh1/iqTdKuIbK+l24OocTlFlTKs7VzPT+mhJ4yX9AbhdUjdJV0iaJWmipNsqzstdkoam129I+m46R1Oafz5S+2em19unn52HJE2XtJ2kXpLuTOuPSKp5yutaIyK8NHgBjgV+VrHeB/gb0C+tjyK7DQXgLmBoet2TbLrhu9P61cCXgU2Ax/n3XzB9078bV2w7BfhBo797xXfeE7i7Yv1R4BNktyuJrHMwERgODAKWA/umfYcAkys+2/x9xwEjgXWBp4G90/aNyG6J/Crwy7TtPcCz6ZweCExM2y8FzkuvDwIeTK/HANOA9QtyroYDM9P6aLIb9TdJ6yOB29I5fBewEBjZys9TAEek1xcB51Z81zPT6/uBj1T8/G2QzuVGaVsT8GTzz5mXlRffh1sMjwDfl/Q9sqSyEBgMTM7+YqY7MK+Vz+0I/DMinkjrvwI+B1wGvA1cJenW1CZks2N+K2lzsiT0z875OrWLiBmSNkvj0/3IzsFuwAhgRtqtF7ADWWKcExFT0vangW0lXQrcCtzeovkdgXkR8UA61msAkg4gS6hExGxJc4B3t/jsAWS/EImIP0naVFKf9N4tEbGo49++Nqs5V8+22G1yRLySXh8AjI9s2OVFSX9eTdOL+ffPyjTgg5VvSuoNDIyIm1Mcb6ft6wDnSxpO9otwINAfeLEDX7OUnHALICKekDQE+BBwATAZmBUR+7Xx0VbLHEV2k/Y+wMFkM2I+T9Y7uxT4YUTcksYpx9TnG9TNBLLe2LvIqjENAi6IiJ9W7iRpEPBm83pELJS0O/AfZL9wjgc+VfkRWp/3Xk2ZqDXNoX+zlffy0vJctVQZW7UlX5dE6qYCy1g1P6yunRPJEv+QiFgi6Rmy3q+14DHcAkg9lbci4hrg+2RViPpJ2i+9v46kXdLurwO90+vZwCBJ26f1k4C7JfUC+kTEbWRDDHuk9/sAc9PrkzvzO7XT9WS/IEaSJZRJwKfS90HSQEmbtfyQpCagW0TcCHwD2KvFLrOBAZL2Tvv3ltQDuIcsWSDp3cBWZEMxlSr3ORBY0NxDbrCW52pN7gWOTWO5/cmGTGqWvvfzko4GkLSepA3Ifq5eTsn2A9Sh0ldZuYdbDLsCF0taDiwBTgeWAj9Jf772AH4MzCIbl7xS0iJgP+CTwPiUQB4AriQbw/29pJ5kvZIz0nHGpH3nAlOAbXL5dlWKiFnpz9a5ETEPmCdpJ+C+NLTyBvBxst5XpYHALyU1dyDOadHuYkmjgEslrQ8sAg4BriA7l4+Qne/REfGOVq6POia1/TDwFgX5RdXyXKVe/+rcSPbXzkzgCbJx2FfbeeiTgJ9K+hbZz+pxwLXAH5SVN3yQ7BectcK3hZmtBST1iog3JG0K/B0YFhEeY82Ze7hma4eJkvqSXSz9tpNtY7iHa2aWE180MzPLiROumVlOnHDNzHLihGt1J2mZsloPM9Oc/g060NaKyl2SjpR09hr27VtZY6CGY6yoFVDN9hb7rKjXUOWxBjXXPLC1jxOudYZFEbFHRAwmmy56WuWbytT8sxcRt0TEhWvYpS9Qc8I1y4sTrnW2vwDbq/UKXyMk3ZeqTI2vmFF2qKTZku4FjmluSFkVrMvS6/6Sbk5Vqx6StD9wIbBd6l1fnPZbpdpY2v51SY9LuoOs1sIaSfpMauchSTe26LUfIukvkp6QdHjav7ukiyuO/Z8dPZHW9TnhWqdJs98OIyvOA1liuzoi9iSb638ucEhE7AVMBb6SZsf9DDgCeB9ZrYDW/ISsYtbuZFN5ZwFnA0+l3vVZkkaQFbvZh2x68xBJw1PdihPIqm4dA+xdxde5KSL2Tsd7DPh0xXuDgPcDHyabudYzvf9qROyd2v+MpELN7LP8eeKDdYb1JT2YXv8F+DkwgJUrfO0L7Az8NU2lXRe4j6xM4j8j4h8Akq4BTm3lGAeRlW8kIpYBr0rauMU+I2i92lhv4JS32O0AAAFnSURBVOaIeCsdo5rHXQ+W9B2yYYteZHUemt2QKnH9Q9LT6TuMAHarGN/tk479BLbWcsK1zrAoIvao3JCSassKVpMj4qMt9tuD1it7tYdovdrYl9txjHHA0RHxkKTRrFwApmVbkY79hYioTMzNlc5sLeUhBWuUKcCw5kpnkjZIFbtmA9tI2i7t99HVfP5OsiI/zeOlG7FyJTVYfbWxe4CPKHtiRm+y4Yu29CYrprMOqXpYheNSJa7tgG3JKo5NAk5P+yPp3ZI2rOI4VmLu4VpDRMT81FO8TumxQGRPGHhC0qnArZIWkJUWHNxKE18Cxkr6NFn1sNMj4j5Jf023Xf1vGsddpdpYREyX9FuyylZzyIY92vINsipbc8jGpCsT++PA3WRFt0+LiLclXUU2tjtd2cHnA0dXd3asrFxLwcwsJx5SMDPLiROumVlOnHDNzHLihGtmlhMnXDOznDjhmpnlxAnXzCwn/wdRm1z0TLDFdQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.metrics import confusion_matrix, plot_confusion_matrix\n", "cm = confusion_matrix(y_iris_test,y_iris_pred)\n", "print(cm)\n", "plot_confusion_matrix(mlp, X_iris_test, y_iris_test,\n", " display_labels=['setosa','versicolor','virginica'],\n", " cmap=plt.cm.Blues)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the matrix, we can compute several performance measures, such as:\n", "\n", "- Precision: Out of all the positive classes we have predicted correctly, how many are actually positive.\n", "- Recall: Out of all the positive classes, how much we predicted correctly. It should be high as possible\n", "- Accuracy: Out of all the classes, how much we predicted correctly\n", "- F1: Harmonic mean of the Precision and Recall" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " 0 1.00 1.00 1.00 13\n", " 1 1.00 0.83 0.91 6\n", " 2 0.92 1.00 0.96 11\n", "\n", " accuracy 0.97 30\n", " macro avg 0.97 0.94 0.96 30\n", "weighted avg 0.97 0.97 0.97 30\n", "\n" ] } ], "source": [ "from sklearn.metrics import classification_report\n", "\n", "print(classification_report(y_iris_test,y_iris_pred))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cross validation\n", "\n", "Using a single train/test split may have some problems dangers — what if the split we make isn’t random? What if one subset of our data has only people from a certain state, employees with a certain income level but not other income levels, only women or only people at a certain age? (imagine a file ordered by one of these). This will result in overfitting, even though we’re trying to avoid it! This is where cross validation comes in.\n", "\n", "The idea is, instead of using a single train/test split, create a series of splits in the following way: in K-Folds Cross Validation we split our data into k different subsets (or folds). We use k-1 subsets to train our data and leave the last subset (or the last fold) as test data. We then average the model against each of the folds and then finalize our model.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import cross_validate\n", "\n", "results = cross_validate(mlp,X_iris,y_iris,cv=5,scoring=['f1_macro','accuracy'])" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean accuracy: 0.967, Std accuracy: 0.030\n", "Mean F1: 0.967, Std F1: 0.030\n" ] } ], "source": [ "print(\"Mean accuracy: {0:.3f}, Std accuracy: {1:.3f}\".format(np.mean(results['test_accuracy']),\n", " np.std(results['test_accuracy'])))\n", "\n", "print(\"Mean F1: {0:.3f}, Std F1: {1:.3f}\".format(np.mean(results['test_f1_macro']),\n", " np.std(results['test_f1_macro'])))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learn more\n", "\n", "- [Machine learning for phisicists (review)](https://arxiv.org/abs/1803.08823)\n", "- [Machine learning for phisicists (review)](https://machine-learning-for-physicists.org/)\n", "- [Sklearn](https://scikit-learn.org/)\n", "- [TensorFlow Playground](https://playground.tensorflow.org/)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "5. Supervised.ipynb", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 4 }