SIT720 Machine Learning Assessment Task 4: Individual ML project © Deakin University 1 FutureLearn ...

Question

SIT720 Machine Learning
Assessment Task 4: Individual ML project
© Deakin University 1 FutureLearn

Assessment 4: Machine Learning Project

This document supplies detailed information on assessment tasks for this unit.

Key information
• Due: Wednesday 25 September 2019 by 11.30pm (AEST)
• Weighting: 30%

Learning Outcomes
This assessment assesses the following Unit Learning Outcomes (ULO) and related Graduate Learning Outcomes (GLO):
Unit Learning Outcome (ULO) Graduate Learning Outcome (GLO)
ULO 2: Perform linear regression, classification using
logistic regression and linear Support Vector
Machines.
GLO 1: Discipline knowledge and capabilities
GLO 5: Problem solving
ULO 3: Perform non-linear classification using
Support Vector Machines with kernels, Decision
trees and Random forests.
GLO 1: Discipline knowledge and capabilities
GLO 5: Problem solving
ULO 4: Understand the concept of maximum
likelihood and Bayesian estimation.
GLO 1: Discipline knowledge and capabilities
GLO 5: Problem solving
ULO 5: Construct a multi-layer neural network using
ackpropagation training algorithm.
GLO 1: Discipline knowledge and capabilities

ULO 6: Perform model selection and compute
elevant evaluation measure for a given problem.
GLO 2: Communication

Purpose
This assessment is an extensive machine learning project. You will be given a specific data set for analysis and will be required to
develop and compare various classification techniques. You must demonstrate skills acquired in data representation,
classification and evaluation. You will use a lot of concepts learnt in this unit to come up with a good solution for a given human
activity recognition problem.

Instructions
The dataset consists of training and testing data in "train" and "test" folders. Use training data: X_train.txt labels: y_train.txt and
testing data: X_test.txt labels: y_test.txt. There are other files that also come with the dataset and may be useful in
understanding the dataset better.

Please read the pdf file "dataset-paper.pdf" to answer Part 1.
Part 1: Understanding the data (2 Marks)
Answer the following questions
iefly, after reading the paper
• What is the objective of the data collection process? (0.5 Marks)
• What human activity types does this dataset have? How many subjects/people have performed these activities? (0.5
Marks)
• How many instances are available in the training and test sets? How many features are used to represent each
instance? Summarize the type of features extracted in 2-3 sentences. (0.5 Marks)
• Describe
iefly what machine learning model is used in this paper for activity recognition and how is it trained. How
much is the maximum accuracy achieved? (0.5 Marks)
https:
cloudstor.aarnet.edu.au/plus/s/6Uxker9axWrCJAx
SIT720 Machine Learning
Assessment Task 4: Individual ML project
© Deakin University 2 FutureLearn

Part 2: K-Nearest Neighbour Classification (5 Marks)
Build a K-Nearest Neighbour classifier for this data.
• Let K take values from 1 to 50. Show a plot of cross-validation accuracy with respect to K. (1 Mark)
• Choose the best value of K based on model performance P. (2 Marks)
• Using the best K value, evaluate the model performance on the supplied test set. Report the confusion matrix, multi-
class averaged F1-score and accuracy. (2 Marks)
[Hints: To choose the best K value, you have to do the following:
• For each value of K, use 10 fold cross-validation to computer the performance P.
• The best hyperparameter will be the one that gives maximum validation performance.
• Performance is defined as: P='f1-score' if fID=0, P='accuracy' if fID=1. Calculate fID using modulus operation fID=SID
% 2, where SID is your student ID. For example, if your student ID is XXXXXXXXXXthen fID=(356289 % 2)=1 then use
'accuracy' for selecting the best value of K.]
Part 3: Multiclass Logistic Regression with Elastic Net (5 Marks)
Build an elastic-net regularized logistic regression classifier for this data.
• Elastic-net regularizer takes in 2 parameters: alpha and l1-ratio. Use the following values for alpha: 1e-4,3e-4,1e-
3,3e-3, 1e-2,3e-2. Use the following values for l1-ratio: 0,0.15,0.5,0.7,1. Choose the best values of alpha and l1-ratio
ased on model performance P. (2 Marks)
• Draw a surface plot of F1-score with respect to alpha and l1-ratio values. (1 Mark)
• Use the best value of alpha and l1-ratio to re-train the model on the training set and use it to predict the labels of
the test set. Report the confusion matrix, multi-class averaged F1-score and accuracy. (1+1=2 Marks)
[Hints: To choose the best alpha/l1-ratio value, you have to do the following:
• For each value of hyperparameter, use 10 fold cross-validation to computer the performance P.
• The best hyperparameter will be the one that gives maximum validation performance.
• Performance is defined as: P='accuracy' if fID=0, P='f1-score' if fID=1. Calculate fID using modulus operation fID=SID
% 2, where SID is your student ID. For example, if your student ID is XXXXXXXXXXthen fID=(356289 % 2)=1 then use 'f1-
score' for selecting the best value of alpha/l1-ratio. ]
Part 4: Support Vector Machine (RBF Kernel) (6 Marks)
Build a SVM (with RBF Kernel) classifier for this data.
• SVM with RBF takes 2 parameters: gamma (length scale of the RBF kernel) and C (the cost parameter). Use the
following values for gamma: 1e-3, 1e-4. Use the following values for C: 1, 10, 100, 1000. Choose the best values of
gamma and C based on model performance P. (2 Marks)
• Draw a surface plot of F1-score with respect to gamma and C. Describe the graph. (1+1=2 Mark)
• Use the best value of gamma and C to re-train the model on the training set and use it to predict the labels of the
test set. Report the confusion matrix, multi-class averaged F1-score and accuracy. (1+1=2 Marks)
[Hints: To choose the best gamma/C value, you have to do the following:
• For each value of hyperparameter, use 10 fold cross-validation to computer the performance P.
• The best hyperparameter will be the one that gives maximum validation performance.
• Performance is defined as: P='f1-score' if fID=0, P='precision' if fID=1, P='accuracy' if fID=2. Calculate fID using
modulus operation fID=SID % 3, where SID is your student ID. For example, if your student ID is XXXXXXXXXXthen
fID=(356289 % 3)=0 then use 'f1-score' for selecting the best value of gamma/C.]
SIT720 Machine Learning
Assessment Task 4: Individual ML project
© Deakin University 3 FutureLearn
Part 5: Random Forest (6 Marks)
Build a Random forest classifier for this data.
• Random forest uses two parameters: the tree-depth for each decision tree and the number of trees. Use the
following values for the tree-depth: 300,500,600 and the number of trees: 200,500,700. Choose the best values of
tree-depth and number of treesbased on model performance P. (2 Marks)
• Draw a surface plot of F1-score with respect to tree-depth and number of trees. Describe the graph. (1+1=2 Marks)
• Use the best value of tree-depth and number of trees to re-train the model on the training set and use it to predict
the labels of the test set. Report the confusion matrix, multi-class averaged F1-score and accuracy. (1+1=2 Marks)
[Hints: To choose the 'tree-depth'/'number of trees' value, you have to do the following:
• For each value of hyperparameter, use 10 fold cross-validation to computer the performance P.
• The best hyperparameter will be the one that gives maximum validation performance.
• Performance is defined as: P='f1-score' if fID=0, P='precision' if fID=1, P='accuracy' if fID=2, P='recall' if fID=3 .
Calculate fID using modulus operation fID=SID % 4, where SID is your student ID. For example, if your student ID is
356289 then fID=(356289 % 4)=1 then use 'precision' for selecting the best value of 'tree-depth'/'number of trees'.]
Part 6: Discussion (6 Marks)
• Write a
ief discussion about which classification method achieved the best performance and your thoughts on the
eason behind this. (2 Marks)
• Which method performed the worst and why? (2 Marks)
• Do you have any suggestions to further improve model performances? (2 Marks)

SIT720 Machine Learning
Assessment Task 4: Individual ML project
© Deakin University 4 FutureLearn

Submission details
Deakin University has a strict standard on plagiarism as a part of Academic Integrity. To avoid any issues with plagiarism,
students are strongly encouraged to run the similarity check with the Turnitin system, which is available
through Unistart. A Similarity score MUST NOT exceed 39% in any case.
Late submission penalty is 5% per each 24 hours from 11.30pm, 25th of September. No marking on any submission after 5 days
(24 hours X 5 days from 11.30pm 25th of September)
Be sure to downsize the photos in your report before your submission in order to have your file uploaded in time.

Extension requests
Requests for extensions should be made to Unit/Campus Chairs well in advance of the assessment due date. If you wish to seek
an extension for an assignment, you will need to apply by email directly to Chandan Karmakar ( XXXXXXXXXX), as
soon as you become aware that you will have difficulty in meeting the scheduled deadline, but at least 3 days before the due
date. When you make your request, you must include appropriate documentation (medical certificate, death notice) and a copy
of your draft assignment.
Conditions under which an extension will normally be approved include:

Medical To cover medical conditions of a serious nature, e.g. hospitalisation, serious injury or chronic illness. Note: Temporary
minor ailments such as headaches, colds and minor gastric upsets are not serious medical conditions and are unlikely to be
accepted. However, serious cases of these may be considered.

Compassionate e.g. death of close family member, significant family and relationship problems.

Hardship/Trauma e.g. sudden loss or gain of employment, severe disruption to domestic a
angements, victim of crime. Note:
Misreading the timetable, exam anxiety or returning home will not be accepted as grounds for consideration.

Special consideration
You may be eligible for special consideration if circumstances beyond your control prevent you from undertaking or completing
an assessment task at the scheduled time.
See the following link for

assessmenttask-4-533yxhd5.zip document-3p1mnujf.pdf

Ximi · Accepted Answer

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SIT 720 - Machine Learning
",
    "
",
    "---
",
    "Lecturer:   Chandan Karmakar     | karmakar@deakin.edu.au
",
    "
",
    "
",
    "School of Information Technology, 
",
    "Deakin University, VIC 3125, Australia.
",
    "
",
    "---
",
    "
",
    "## Assignment 4
",
    "
",
    "
",
    "In this assignment, you will use a lot of concepts learnt in this unit to come up with a good solution for a given human activity recognition problem.
",
    "
",
    "**Instructions**
",
    "1. The dataset consists of training and testing data in "train" and "test" folders. Use training data: X_train.txt labels: y_train.txt and testing data: X_test.txt   labels: y_test.txt. There are other files that also come with the dataset and may be useful in understanding the dataset better.
",
    "
",
    "2. Please read the pdf file "dataset-paper.pdf" to answer Part 1.
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1: Understanding the data **(2 Marks)**
",
    "
",
    "Answer the following questions briefly, after reading the paper "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* What is the objective of the data collection process? **(0.5 Marks)**
",
    "
",
    "
",
    "
",
    "* What human activity types does this dataset have? How many subjects/people have performed these activities? **(0.5 Marks)** 
",
    "
",
    "
",
    "
",
    "* How many instances are available in the training and test sets? How many features are used to represent each instance? Summarize the type of features extracted in 2-3 sentences. **(0.5 Marks)**
",
    "
",
    "
",
    "
",
    "* Describe briefly what machine learning model is used in this paper for activity recognition and how is it trained. How much is the maximum accuracy achieved? **(0.5 Marks)**
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading training data 
",
    "with open('train/X_train.txt') as f:
",
    "    train_x = f.read().split('\n')
",
    "with open('train/y_train.txt') as f:
",
    "    train_y = f.read().split('\n')
",
    "# Reading testing data 
",
    "with open('test/X_test.txt') as f:
",
    "    test_x = f.read().split('\n')
",
    "with open('test/y_test.txt') as f:
",
    "    test_y = f.read().split('\n')
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np
",
    "# Converting each data point into float for training and test data as they are in string
",
    "train_x = map(lambda x: x.strip().split(), train_x)
",
    "train_x = [np.array(map(float, row)) for row in train_x]
",
    "train_x = np.array(filter(lambda x: len(x) == 561, train_x))
",
    "
",
    "test_x = map(lambda x: x.strip().split(), test_x)
",
    "test_x = [np.array(map(float, row)) for row in test_x]
",
    "test_x = np.array(filter(lambda x: len(x) == 561, test_x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Converting each data point into float
",
    "train_y = map(float, filter(lambda x: len(x) == 1, train_y))
",
    "train_y = np.array(train_y)
",
    "
",
    "test_y = map(float, filter(lambda x: len(x) == 1, test_y))
",
    "test_y = np.array(test_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data size: 7352, and features: 561
"
     ]
    }
   ],
   "source": [
    "print ("Data size: %d, and features: %d"%train_x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sampling training and test data for quick running of models
",
    "# Please comment out this code for actual model training
",
    "SAMPLE_SIZE = 100
",
    "train_x = train_x[:SAMPLE_SIZE]
",
    "train_y = train_y[:SAMPLE_SIZE]
",
    "
",
    "test_x = test_x[:SAMPLE_SIZE]
",
    "test_y = test_y[:SAMPLE_SIZE]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2: K-Nearest Neighbour Classification **(5 Marks)**
",
    "
",
    "Build a K-Nearest Neighbor classifier for this data.  
",
    "
",
    "- Let K take values from 1 to 50. Show a plot of cross-validation accuracy with respect to K. **(1 Mark) ** 
",
    "- Choose the best value of K based on model performance P. **(2 Marks) **
",
    "- Using the best K value, evaluate the model performance on the supplied test set. Report the confusion matrix, multi-class averaged F1-score and accuracy. **(2 Marks)**
",
    "
",
    "*[Hints: To choose the best K value, you have to do the following:*
",
    "- *For each value of K, use 10 fold cross-validation to computer the performance P. *
",
    "- *The best hyperparameter will be the one that gives maximum validation performance.*
",
    "- *Performance is defined as: P='f1-score' if fID=0, P='accuracy' if fID=1. Calculate fID using modulus operation fID=SID % 2, where SID is your student ID. For example, if your student ID is 356289 then fID=(356289 % 2)=1 then use 'accuracy' for selecting the best value of K.]* 
",
    "
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier
",
    "from sklearn.model_selection import cross_val_score
",
    "
",
    "# Values of K from 1-50
",
    "K = list(range(1, 51))
",
    "accuracies = []
",
    "def model(k, X, y):
",
    "    """
",
    "    Defining Model and performing 10-fold CV
",
    "    """
",
    "    clf = KNeighborsClassifier(n_neighbors=k)
",
    "    scores = cross_val_score(clf, X, y, cv=10)
",
    "    return np.average(np.array(scores))
",
    "
",
    "for k_value in K:
",
    "    accuracies.append(model(k_value, train_x, train_y))
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Structuring accuracy data for plotting
",
    "import pandas as pd
",
    "df = pd.DataFrame(list(zip(K, accuracies)), columns = ['K', 'Accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       ""
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEPCAYAAABcA4N7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4VGXePvB7SjLpfWbSC0mAVBKBEEITpJkCrOCCuGZBwPKqvPLqu6CiuP5WVFbF8qq7qKAICLhKCQJGQCwklERqSGghpJHe22SSOb8/IlnZkEkCU5KZ+3NdXBczc86c78OEuXOe55znEQmCIICIiKgbYmMXQERE/RuDgoiItGJQEBGRVgwKIiLSikFBRERaMSiIiEgrBgUREWnFoCAiIq0YFEREpBWDgoiItGJQEBGRVgwKIiLSikFBRERaSY1dwJ2orm6ERtP95LeurnaorGwwYEX9A9ttfsy17Wx334jFIjg72/Z5vwEdFBqNoDUobmxjjthu82OubWe79Y9dT0REpBWDgoiItGJQEBGRVgwKIiLSikFBRERaMSiIiEgrBkUPBEHA3zZmYPsPl41dChGRUTAoepBXUo/c4jrsP5aPC/nVxi6HiMjgGBQ9OJ5dColYBDdHK6zfmw1Va7uxSyIiMigGhRYaQcCJnDKEB7hgUUIIymta8PWPV4xdFhGRQZldUGg0AgrLezdHSm5RHarqVBgZosAQX2fcc5c3DmQW4mJBjdb9mlVtKCwzv/lniMg0mV1Q5BbX4aVPjyPralWP2x7PLoVUIkZ0sBwAMPvuQf/uglLfugvq6vU6rFp/HKvWH8flolqd1k5EZAxmFxR+7vZwtLPEvmPXtG6nEQScuFCGiEEusJZ1zJ1oZSnFwvgQlFU3Y8dPuTdtLwgCvs8owOovMiEIApzsZVj/bTZauwkUIqKBwuyCwkIqxuTh3jifV4380vput7tUUIPahlbEhChvej7EzxkTo73w/YkCXCrs6IJqamnDhzvP4csDlxAe4IJVC2OwMH4oSqqasOuXq3ptDxGRvpldUADAxGgvyCwl2H88v9ttjueUwVIqxrAg1y6vzbk7EC4OVli/NweXC2vxymcncPJiBe6fGIin5kTCztoC4QGuGD/MA/uP5+NKMbugiGjgMsugsLGywPhITxw/X4aqupYur7drNMjMKUNkkBusLLsu2WEtk2JB/FCUVjVh9aZMqNs1WP5gNO4d5QexSNS53R8nBsPJToYNe3Ogbuu+C6qpRY0dP+WirKZZNw0kItIhswwKAJgy0hsAkHqioMtrF/JrUNekRsxQRbf7h/m7ICnOHzEhCry8cCSCvZ26bGNjJcWfpw9FcUUjdh/Ju+X7XL1eh5c3nEBKWh72dLMNEZExmW1QuDlaIyZEgR9PF6OpRX3TaydyyiCzkCAisGu30+/9YfwgPDYzHPY2lt1uExnoirERHth3NB9Xr9d1Pi8IAg5mFuK1TZnQCAKG+Dgh82KZ1jMPIiJjMNugAIBpMb5Qtbbjx1PFnc+1tWuQeaEcUcFukFlIdHKcefcEwcHWAuv3ZkPdpkFTSxs+2pWFzd9fRKi/C15eGIPEOH80q9px+nKlTo5JRKQrZh0Ufu72CPFzxvcZBWhr1wAAcq5Vo6FZe7dTX9lYWeDP04eiqLwRG7/LwSufn8CvF8px/92BWPrb4PdQPyc42Fri6PlSnR2XiEgXzDooAGD6KF/UNLTi2G9f0Mezy2AtkyB8kItOjzMsyA1x4e44crYE6jYN/jI/GvfG/nvwWyIWIyZEgTNXKtD4H11hRETGZPZBER7gAm+5LfYfz4e6TYNfL5YjKkgOC6luup1+b/7kwbh/YiBWLRyJwT5dB79Hh7mjrV1A5oVynR+biOh2mX1QiEQiTIvxRVF5I7YfuowmVRtiQnTX7fR7NlZS3DvKDw7dDH77u9tD4WyNo1klejk+EdHtMPugAIBRoUo428tw8NdC2FpJERag226n3hKJRIgNVeJCfg2q61VGqYGI6D8xKABIJWJMHtFxX0X0YDmkEuP9s8SGuUMAOsdMiIiMjUHxmwnDvDAs0BWTh3sbtQ53Fxv4u9vj6Hl2PxFR/8Cg+I2NlRT/ff8w+CrtjV0KYsPckV/agOKKRmOXQkTEoOiPYkIUEIlwW/dUqNTtqKxthkYj6KEyIjJHXWe8I6NzspMhxM8Zx86X4A/jAiD63USDN1TUNuNsbhUqaptRUdOCitoWVNY2o66p4x4MiVgEFwcZ3Byt4epoBTdHK3i62uKuwXKIxV3fj4ioOwyKfio21B3r92bjSnEdgrwcO58XBAGHTxVj+6HLUKnbIRGLOoMgKtgNbo7WUMrtkF9ci4ralt8CpRK1Da0AgD+MC0DSmABjNYuIBiAGRT81fIgcG7+7gGNZpZ1BUV2vwoZ92TiXW4Uwf2fMnzIYSmebLmcIcrk9ystvXpSpVd2OT77NRkpaHkYMVcDD1dZgbSGigY1jFP2UtUyKqCBXHM8pRVu7BkfPl+ClT4/hYn4N/jR1MP5nbhQ8XG173Y1kaSHBg5ODYSmV4PP9F6AROIZBRL3DoOjHYsPcUd+kxuubf8W63efh7mKDvz4cg0l3ed9y3KInjnYy/HFSEC4W1ODn08U970BEBHY99WsRg1xhayXFtZJ6zJ4wCNNH+UIivrNsHxfpgaNZJdj+wxUMC3KDk51MR9USkaniGUU/ZiEV49l50XhlUQwSRvvfcUgAHdOEJE8fCnWbBlsOXNJBlURk6hgU/Zyfu73OB57dXWyQNMYfGTllOHWpQqfvTUSmh0Fhpu4d5QsvuS2+SL2AZlWbscshon5Mr0GRkpKC+Ph4TJkyBZs3b+7y+o8//oikpCQkJSXhmWeeQWMjp6wwFKlEjAX3DkVNvQrf/JgLoOMejcraFpy8WI5dv1zF+1+fwYc7ziIlLQ9nrlSgul4FgVdLEZkdvQ1ml5aWYu3atfjmm29gaWmJefPmYdSoUQgKCgIA1NXVYcWKFfjiiy8QFBSEjz/+GGvXrsXKlSv1VRL9h0BPR0wa7o1DmYUoLG9AYXkDGls6zi5EABQuNhA0AjJ+t5CSvY0FfBV2CAtwxZSR3r0aN2loVuPwySKMj/Lsdi0OIuq/9BYUaWlpiI2NhZNTx0pu06ZNw/79+/Hkk08CAPLy8uDp6dkZHBMnTsTixYsZFAZ23/hByC2uQ2tbO0YMVcBXYQcfpT185HaQWXas8tesakNBWQPyS+uRX9aAayX12P7DZZy6XIFHZ4TB2b77K6euFNXiH7vOobJOhZKqJixODDVU04hIR/QWFGVlZZDL5Z2PFQoFzpw50/nY398fJSUlyMnJwdChQ7Fv3z5UVHBg1dCsZVK8+OcRPW4z2MfppuVb08+V4PPvcvDyhuN4ZEYYwvxvXuxJEAR8f6IAXx2+Amd7GUYOVSD9XAmmjvTp1Qy9py5VQICA6GB5j9sSkX7pLShu1Zf9+5vEHBwc8MYbb+DFF1+ERqPBH//4R1hYWPTpGK6udj1uI5cbf9pwY9B3u2dMtEd0qDte33gCb287hXlThmDulCGQiEVoaGrFu9tO4ui5EsSGu+O/590FAHhk9ffYdSQPrzwap/W9r12vw4c7z8JCKsFnL/nAxqr3Pxfm+nkD5tt2tlv/9BYUSqUSGRkZnY/LysqgUPx7Ler29na4u7vjq6++AgBkZWXBx8enT8eorGzQOp32reY8MgeGareVGHhu/l3Y9P0FfJl6AaculGFajC82pV5Adb0K8+4JxpQR3mhuaAEAJIz2x9aDl/DD8TyEB7je8j3bNRq8uSkTUokYzao27Dh4EVNjfHtVj7l+3oD5tp3t7huxWNSrX7C77NfnPXopLi4O6enpqKqqQnNzM1JTUzF+/PjO10UiER5++GGUlpZCEASsX78e8fHx+iqH9ERmKcGihFAsjB+KK0W1eOer0xAEASsevAtTR/rcdBY5MdoLbo5W+OqHK90G/HfHC5BXUo8F9w7FYG9HHMgs5NoaREamt6BQKpVYtmwZkpOTMWvWLCQmJiIyMhJLlizB2bNnIRaL8corr2Dx4sWYPn067O3tsWjRIn2VQ3o2LtITK/88Aolx/li1MAaBv5sa/QYLqRizJwSioKwB6Vldl3otrmjEzp9zMXywHCOHKjBlpC8qalvw68XyLtsSkeGIhAF8YTy7nm6tP7dbIwj42+cZqGtqxeolsbC06LiySqMRsHpTJkqrmvC3JbFwtLWERiPguXXpcLST4fk/De/xvftzu/XNXNvOdvdNv+t6IroVsUiEP04MQlWdCgcyCzufTz1RgNziOjw4dTAcbTvutRCLRZg83AeXC2tx9XqdsUomMnsMCjK4oX7OGBboim/T81Df1IrrlY3Y8XMuooPdMCpEedO2YyM9YC2TIPVEgXGKJSIGBRnHnIlBaGltx+5f8rBhbw4spWIkTxvSZZ0Na5kU4yI9kZFThqq6FiNVS2TeGBRkFF5uthgX6YmDvxbiclEt5k8eDMdu1saYPNwbGkHAwd91VRGR4TAoyGhmjQuAlaUEUUFuiA1Tdrudm5M1hg+W48dTxWhp5Uy3RIbGoCCjcbKT4bVHYvHEfeE9Lu06daQvmlRtOHK262W1RKRfDAoyKkc7Wa9moA30ckCAhwMOZBRAM3Cv6CYakBgUNCCIRCJMHemD0upmnLlcaexyiMyK3uZ6ItK14UPkcLaXYdsPl3H0fAkaW9rQ2KxGY4saTS1tEIvFeHnhSK3TnhNR3/GMggYMqUSMmWMD0Kxqw7XSBjSr2mBvY4lAT0dEDHJFfVMrrhTVGrtMIpPDMwoaUMYP88T4YZ5dnle3teN4dikKyhowYqjiFnsS0e3iGQWZBAupBJ5yOxSWNxi7FCKTw6Agk+Hv4YCCMgYFka4xKMhk+Hs6oKK2Bc0q3pRHpEsMCjIZAR4da2AUVTQauRIi08KgIJPh5+EAAChk9xORTjEoyGQonK1hZSlBAQe0iXSKQUEmQyQSwVthhyKeURDpFIOCTIqP3A4F5Y0YwCv8EvU7DAoyKd5yWzSr2lBVpzJ2KUQmg0FBJsVb0bFwPMcpiHSHQUEmxcutIyiKGBREOsOgIJNiYyWFq4MV79Am0iEGBZkcH4UdCst50x2RrjAoyOR4K2xRUtkEdZvG2KUQmQQGBZkcb7kdNIKA65U8qyDSBQYFmRxv+W9XPnGcgkgnGBRkcpQu1pBKxFybgkhHGBRkciRiMbzcbDmgTaQjDAoySd4KW84iS6QjDAoySd5yO9Q2tqKusdXYpRANeAwKMkk3pvLgOAXRnWNQkEnykd8ICo5TEN0pBgWZJAdbSzjYWnKcgkgHGBRksrzltpxFlkgHGBRksrzldiiuaIRGw0WMiO5Ej0FRVVVliDqIdM5HYQd1mwal1U3GLoVoQOsxKBITE/HMM88gIyPDEPUQ6Yw3B7SJdKLHoDh06BDi4uKwZs0aJCUlYfPmzWho6F2/b0pKCuLj4zFlyhRs3ry5y+tZWVmYPXs2ZsyYgUcffRR1dXV9bwFRNzzdbCAWiTjnE9Ed6jEorKysMHv2bGzfvh0rV67E+vXrMW7cOPz1r39FZWVlt/uVlpZi7dq12LJlC3bt2oVt27bh8uXLN23z6quvYunSpdi9ezcCAgLw6aef3nmLiH5jIZVA6WLNK5+I7lCvBrN/+uknPPXUU1i2bBkmT56MrVu3wsPDA48//ni3+6SlpSE2NhZOTk6wsbHBtGnTsH///pu20Wg0aGzs6BZobm6GlZXVHTSFqKuORYwYFER3QtrTBnfffTecnZ0xf/58/P3vf+/8Mh8yZAi2bdvW7X5lZWWQy+WdjxUKBc6cOXPTNitWrMDChQuxevVqWFtbY/v27bfbDqJb8pbb4Xh2GZpVbbCW9fjjTkS30OP/nLfffhtDhgyBra0tWltbUVlZCVdXVwDAwYMHu91PELpekigSiTr/3tLSghdeeAGff/45IiMjsWHDBixfvhzr1q3rdfGurnY9biOX2/f6/UwJ290hLEiOb37KRaNagK+3af+b8DM3L4Zsd49BUVJSghUrViA1NRVFRUV44IEHsHr1akyaNEnrfkql8qYrpcrKyqBQKDofX7x4ETKZDJGRkQCAuXPn4t133+1T8ZWVDVqvkZfL7VFeXt+n9zQFbPe/OVpJAAD7juTCzc7CGGUZBD9z83K77RaLRb36BbvLfj1t8I9//AMbN24EAAQEBGDHjh14//33e3zjuLg4pKeno6qqCs3NzUhNTcX48eM7X/fz80NJSQlyc3MBdJydRERE9LkBRNq4OFhhyggf/HCyCCdyyoxdDtGA1OMZhUajgbu7e+djDw8PaDQ9L1qvVCqxbNkyJCcnQ61WY86cOYiMjMSSJUuwdOlSRERE4LXXXsPTTz8NQRDg6uqK1atX31lriG7h/omByC2uxYa92fBR2MHdxcbYJRENKCLhVoMJv5OcnIz4+HjMmTMHIpEIO3bswL59+/rFpazsero1trurqroWvLzhBJzsZFiZPByWFhIDV6df/MzNS7/renrllVewfft2REZGIjIyEtu3b8eqVav6fCAiY3JxsMLixFAUljdg8/cXjV0O0YDSY9eTv78/vvnmG9TW1kIikcDOru9pRNQfRAa6IjHOD3vSrmGwjxPGRHh02UYQBFwrrYdUIu6cAoTI3PUYFFVVVdi9ezcaGxshCAI0Gg2uXbuGt956yxD1EenUrLGDcLmwFl98dwF+SvvOlfCq61U4mlWCI+dKUFzRCJmlBC/9eQQ8XG2NXDGR8fXY9fT0008jLS0NX3/9NUpKSrBz506IxZydnAYmsViER2eEwVomxYc7zyE9qwRvbz+FZz88gq8OX4G1TIIH7gmGhUSMD3eeg0rdbuySiYyux2/84uJirFu3DuPHj8ef/vQnfPnll8jPzzdEbUR64Wgnw6MzwlBa3YSPU87jekUjEkb7YfUjsXjhoRGYMtIHj8wIRXF5IzalXjB2uURG12PXk5ubG4COsYqLFy9ixowZaGtr03thRPo01M8Zz8yNggjAED9niH83awAAhAe4IjHOHylpeRjs44RxkZ7GKZSoH+gxKFxdXfHJJ58gKioK77//Puzs7Ho9zThRfxbq76L19ZljA3C5qBabUi/C390BPgoObpN56tXlsZaWlhgxYgTCw8Px3nvv4dlnnzVEbURGJRaL8MiMMNhYSfHhjrNoVvFMmsxTj0HxxhtvIDk5GQDwv//7v9i5cyemTJmi98KI+gNHW0s8NiMM5TUt2LAv55aTXRKZuh6DIieH/znIvA3xdcZ9EwYhI6cMh34tMnY5RAbX4xiFXC5HQkIChg0bBlvbf19TvnLlSr0WRtSfTB/li0sFNdh68BJkFhKMjex6sx6RqeoxKKKjoxEdHW2IWoj6LbGoY7ziwx1nsX5vNqrqWpA0xv+mNVaITFWPQfHkk08aog6ifs9aJsV/3z8Mn+/Lwc5frqKyrgUPTRsCqYQ3oJJp6zEokpKSbvl8SkqKzosh6u+kEjEeTgiBq6MVdh/JQ3WDCo/PDOcyq2TSevzpfvHFFzv/rlarceDAgZtWqiMyNyKRCLPGDYKLgxU27r+AN7b8iqfvHwYnO5mxSyPSix6DIiYm5qbHcXFxmDdvHh5//HG9FUU0EIwf5gknOxk+2nkOr27MxJ/vHYLwAFdjl0Wkc33uXK2urkZZGZeUJAI6pi5f/mA0xGLg7W2nsXb7aRSVc+YCMi19HqMoLi7G3Llz9VYQ0UDj7+6Avy2OxcHMQqSk5eGl9ccxIcoLs8YGwMHW0tjlEd2xPo1RiEQiuLi4IDAwUK9FEQ00FlIxpo/yxdhID+z+5Sp+OFmEo1klSBjth2kxvrwyiga0Hn96fX19sXfvXsTExMDV1RVvvfUWKioqDFEb0YBjZ22B+VMG4/8tHoUQP2d8/WMudh+5auyyiO5Ij0GxYsUKDBo0CADg5eWFmJgYPPfcc3ovjGggc3exwVOzIzFiiByHMos4oSANaD0GRXV1deekgDKZDAsWLEB5ebneCyMyBdNG+aJJ1Yafz1w3dilEt63HoGhvb0dpaWnn44qKCk4SSNRLgZ6OGOztiO9P5KNdozF2OUS3pcfB7AULFmDWrFkYN24cRCIR0tLS8Je//MUQtRGZhOmj/PDe12dwIqcMsaHuxi6HqM96DIo5c+YgPDwcR48ehUQiweLFixEcHGyI2ohMQmSQKzxcbfDdsQKMClFyIkEacHrseiotLcXWrVuxYMECjBkzBmvXruUYBVEfiEUiTIvxxbXSeuRcqzZ2OUR91mNQLF++vMtVT88//7zeCyMyJaPDlHCwtcS+4/nGLoWoz3jVE5EBWEgluGe4N87lVqGQU3zQAMOrnogMZGK0FywtxPjuGM8qaGDp01VPAJCens6rnohug521BcZFeuLwySLcNyEQzvaclpwGhh7PKObMmYMNGzYgNDQUERERmDt3LjZu3GiI2ohMztSRPtAIAg5kFBi7FKJe69WyXB4eHlCpVNiyZQuamprw0EMP6bsuIpMkd7LGiCEKHD5VhMQ4/y4r42kEARAAsZiX0FL/oTUocnNz8dlnnyElJQVeXl5oaWnBoUOHYG9vb6j6iEzO9FG+OJFThrVfnYaVhQSNLWo0trShsVmNJlUbxCIRnO1lcHO0gpujNdwcreDqaAUfhR18lfy/R4bXbVAsWbIEWVlZiI+Px8aNGxEREYFJkyYxJIjuUICHA8aEu+NyUS3arSxga20BhbMNbK2ksLWygEYQUFnbgoraFpy7WomahlYAgAjAq4/Ewt3FxrgNILPTbVBkZ2cjNDQUwcHB8Pf3BwDeUUqkI4sSQ3u9rbqtHUUVjfjb55k4cvY6Zk/gejBkWN0OZh8+fBizZ8/Gnj17MHbsWCxduhQqlcqQtREROu7B8Hd3QPggFxw5ex0aDS9PJ8PqNiikUinuvfdefPHFF/j666+hUCjQ0tKCqVOn4ssvvzRkjUQEYGyEB2oaWnHuapWxSyEz06v1GYOCgrBy5Ur8/PPPWLRoEbZv367vuojoP0QFu8HO2gK/nOXaFmRYvbo89gZra2vMnTsXc+fO7dX2KSkp+Oijj6BWq7FgwQI8+OCDna9lZ2djxYoVnY+rqqrg6OiIPXv29KUkIrMhlYgRG6bE4ZNFaGhWw87awtglkZnoU1D0RWlpKdauXYtvvvkGlpaWmDdvHkaNGoWgoCAAQEhICHbt2gUAaG5uxv3334+XX35ZX+UQmYSxER44kFGIo1klmDzCx9jlkJnoVdfT7UhLS0NsbCycnJxgY2ODadOmYf/+/bfc9p///CdGjhyJESNG6KscIpPgq7SHn9Ke3U9kUHoLirKyMsjl8s7HCoXipskFb6irq8P27dvx5JNP6qsUIpMyNtID+aUNyC+tN3YpZCb01vV0qxlmb3UfRkpKCiZPngxXV9c+H8PV1a7HbeRy87xBkO02XQnjA7Ht0GVkXq7E8HDPzufNoe23wnbrn96CQqlUIiMjo/NxWVkZFApFl+0OHDiARx999LaOUVnZoPWacrncHuXl5vdbF9tt+qKD3fBDRgGSYn0hlYjNqu2/x3b3jVgs6tUv2F326/MevRQXF4f09HRUVVWhubkZqampGD9+/E3bCIKArKwsREdH66sMIpM0NtIDDc1qnLpUYexSyAzoLSiUSiWWLVuG5ORkzJo1C4mJiYiMjMSSJUtw9uxZAB2XxFpYWEAm47z8RH0R5u8CZ3sZB7XJIPTW9QQASUlJSEpKuum5jz/+uPPvrq6uOHLkiD5LIDJJYrEIceHu2Hv0GqrrVWbbT0+GobczCiLSr7ERHhAEID2rxNilkIljUBANUEoXGwR7O+LnM9e5jj3pFYOCaAAbG+mB0qom/HSyyNilkAljUBANYKNClAj0csDbWzrWqiDSBwYF0QBmaSHBM3OjEBHkhk+/zcbBzEJjl0QmiEFBNMBZWUrx0qJYRAe7YfP3F/Ftep6xSyITw6AgMgGWFhI8PiscsWFKfP1jLr46fLnLAHdTSxtOXarA1oOXcJRXSlEf6PU+CiIyHKlEjMWJobCylGLf0Xy0qNoRPdgN2deqkXOtGnkl9biRHRKxCB6utvBz5/0X1DMGBZEJEYtEeGjqYFhbSrDvWD5+OFkEiViEQZ4OSIrzx1BfZyhdbPDK5yfw8Z7zWLVgBCykEmOXTf0cg4LIxIhEItw/MQhDfJ0hFgFB3o6wsrz5v/qi+BC8vf00vv4xF/PuCTZSpTRQMCiITFRkYPdT94cPcsWku7yQeqIAwwJdEeLvYsDKaKDhYDaRmbp/YhCULjb4dG82mlrUxi6H+jEGBZGZkllI8EhSKGrqW7H5+4vGLof6MQYFkRkL8HBA0hh/pGeV4kROmbHLoX6KQUFk5hJG+yHAwwEb9+egul5l7HKoH+JgNpGZk0rEWJIUipfXH8eaL09C6WzdZRsnOxnmTgqCtYxfGeaIZxREBHcXGyxKDIWNTILaxtab/zS04qfTxdiTlmfsMslI+OsBEQEARg5VYORQxS1f+/Tb80g9UYCxkR7wcLU1cGVkbDyjIKIezbk7CJYWYnx58BIXSTJDDAoi6pGjrSVmjgnAudwqnL5caexyyMAYFETUK5OGe8PTzRZfHrwIdVu7Tt87+1o1thy4iB9OFuFKcS1Uat2+P90ZjlEQUa9IJWI8MDkYb209hf3HC5AU53/L7SprW/BF6gV4uNpg5tiALvNM/Z5GI2D3katIOZIHsViEdk1Ht5ZI1DHA7qu0x2BvR0yI8oJYLNJHs6gXGBRE1Gth/i4YPliOb9PzMCbcHS4OVje9fvpyBT7Zcx7qNg3OXKlERk4ZHpw6BFFBbl3eq7ZBhXUp55F9rRpjItzxpylDUN/UivyyBuSX1iO/tAGXCmtw7HwprCylGB3ubqBW0n9iUBBRn8ydFIQzn1Ri+w+X8djMcABAW7sGO37Oxb6j+fBV2OHxWeGob1Lj8/05eO9fZzBiiBzzpwyGk50MQEdX07rdWWhWtWFh/FCMi/QEAMgsreHmZI27BssBAIIgYOUnx3Ags5BBYUQMCiLqEzcna8TH+mHXL1dxd1Q1lC42+MeGLb3fAAAOTklEQVSuc7hUWIu7ozzxwORgWEglULoAqxaOxP5j+dh9JA9ZeUcxZ0IgGprV2PnLVSidbfDMvCh4y+26PZZIJMI9w72xKfUirhTXItDT0YAtpRsYFETUZ/eO8sWRs9fx2b4cNKnaoG7T4JGkUMSG3fxbv1QiRmKcP0aGKLBx/wV8kdox+WBsqBLJ04doHb+4IS7cHV//eAUHMwoROINBYQwMCiLqM0sLCeZOCsYHO87CS26L/5oVrvVGPKWzDZ6dF4UTOWXQaASMClVCJOrd4LSVpRTjIj1xMLMQ908MgrO9TFfNoF5iUBDRbRk+RI4XHhoOb4UdZBY9L6cqEokQE6K8rWNNussL358owI+nijBr3KDbeg+6fbyPgohuW6CXY69C4k4pnG0wLMgNh08WQd2m0fvx6GYMCiIaEO4Z4Y26JjVO5JQauxSzw6AgogEh1M8ZHq42+D6jkPNNGRiDgogGBJFIhMkjfHCtpB5XiuuMXY5ZYVAQ0YARF+YOa5kUBzIKjF2KWWFQENGAIbOUYPwwD2ReKOeyrQbEoCCiAWXSXd7QaAT8cLLI2KWYDQYFEQ0ocidrRAW74cdTRWjldOQGwaAgogFn8nBv1Dep8RPPKgyCQUFEA85QP2d4utni27Srxi7FLOg1KFJSUhAfH48pU6Zg8+bNXV7Pzc3FQw89hBkzZmDRokWora3VZzlEZCJEIhEm3eWFywU1yOWlsnqnt6AoLS3F2rVrsWXLFuzatQvbtm3D5cuXO18XBAGPP/44lixZgt27dyMkJATr1q3TVzlEZGJG/3ap7MHMQmOXYvL0FhRpaWmIjY2Fk5MTbGxsMG3aNOzfv7/z9aysLNjY2GD8+PEAgMceewwPPvigvsohIhNjLZNi0ggfnMgpRV1Tq7HLMWl6C4qysjLI5fLOxwqFAqWl/56jJT8/H25ubli+fDmSkpKwatUq2NjY6KscIjJBCWMC0NYu4OfTxcYuxaTpbZrxW83F8vv559va2nD8+HFs2rQJEREReOedd/D666/j9ddf7/UxXF27XxnrBrncvtfvZ0rYbvNjrm2PDHLDT2eu46HEcEjEvVvjwhQY8vPWW1AolUpkZGR0Pi4rK4NCoeh8LJfL4efnh4iICABAYmIili5d2qdjVFY2QKPpfnIwudwe5eX1fax84GO7zY+5tl0ut8e4CHd8sOMcDqZfRfRgec87mYDb/bzFYlGvfsHusl+f9+iluLg4pKeno6qqCs3NzUhNTe0cjwCA6OhoVFVVIScnBwBw6NAhhIWF6ascIjJRUcFucLaX4eCvHNTWF72eUSxbtgzJyclQq9WYM2cOIiMjsWTJEixduhQRERH44IMPsHLlSjQ3N8Pd3R1r1qzRVzlEZKIkYjHujvbCjp9ycb2yUeuSrHR7RMIAntidXU+3xnabH3Nt+4121zW24tkPj2BClBcenDLY2GXpncl0PRERGYqDrSVGDFUg7dx1tLS2Gbsck8OgICKTMOkubzSr2pGexaVSdY1BQUQmIdDTAX5KexzK5FKpusagICKTcGP+p6KKRlwsqDF2OSaFQUFEJmNUqBK2VlKs35uNayXmN7ivLwwKIjIZlhYS/PecYWhrF/DqF5n44WQRu6F0gEFBRCYlyNsRqxaOxFBfJ3zx3QWsSzmPZpVpXAlV06DC2u2nkZlj2AF7vd1wR0RkLA42lnj6j8Pwbfo17Pw5F9dK6vFfs8Lhrej7PQS3kl9aj5qGW89YG+TlABsrC50c5/cqaprx5tZTqG1shZOdTOfvrw2DgohMklgkQlKcP4K9HPHP3Vn428YMzJ8yGOMiPW6aoLQv2to1+NfhK0g9UdDtNh6uNli1YCQsLSS3W3oX1ysb8ebWU2hVt+PZB6IQ6O1k0BssGRREZNKG+jnj5Ydj8HFKFj7bl4NTlyrw53uHwtHWsk/vU1nbgo92nUNucR3uucsbseHKLtuUVDbh02+z8a8fr2D+ZN3cIX6tpB5vbTsFsViEv8y/Cz46OivqCwYFEZk8R1tL/M/cKBzIKMS/Dl/Bi58cQ/K0IRgxVNHzzgBOX67AJ3vOo10j4PFZ4RjZzX6Bno64VlKPAxmFGBbkhjB/lzuq+1JhDd756gxsZBI8Oy8aShfjrNnDwWwiMgtikQhTR/pg1cKRcHW0woc7z+HjlCw0tai73aetXYOvfriMd/91Bq4OVli1cGS3IXHDnLsD4eFqg/XfZqNRy3v//hi3+nPuaiXe2nYKDjYWWPHgcKOFBMAzCiIyM15utnjhoeH4Nv0aUo7kISe/BnPuDoTMQoJmVRuaVG1oUbWhWdWOCwU1uHq9DndHeeKBycGwkPY87mBpIcHixFCs/iITm1Iv4tEZt14+Qd3WEUIHfy1Ed1fwesvt8My8qD53k+kag4KIzI5UIsbMsQGIDHTFJ3vO4+OU8122sZSKYW9jiUeSQhEb5t6n9w/wcMCMMf7Y8fNVRAW5YVTozeMZ5TXN+Meuc7h6vR5jIzygcLa+5fHHRHrAVg9XUPUVg4KIzFaAhwNeXjgSeSX1sJRKYG0lhbWlBNYyKaSSO+uZjx/thzNXKvHFdxcw2McJzvYdl7SevFiOT7/NhgDgiT9EYPiQ/r8qH8coiMisWUglCPZ2gp+7PRRO1rC3sbzjkAA6FlRanBiKNo0G6789D3WbBlsPXsL735yF3NkaqxaOHBAhAfCMgohIb5QuNpg3KRgbv7uA59elo7JOhUl3eWHupGBYSAfO7+kMCiIiPZoQ5YkzVyqRk1+Nx2aGISak6/0X/R2DgohIj0QiEZ64Lxytag2sZQPzK3dgVk1ENIBIxGJYywZOV9N/GriVExGRQTAoiIhIKwYFERFpxaAgIiKtGBRERKQVg4KIiLQa0JfHisU9r1LVm21MEdttfsy17Wy3fvcBAJEgdDfBLREREbueiIioBwwKIiLSikFBRERaMSiIiEgrBgUREWnFoCAiIq0YFEREpBWDgoiItGJQEBGRViYbFCkpKYiPj8eUKVOwefNmY5ejVw0NDUhMTERhYSEAIC0tDUlJSZg6dSrWrl1r5Or05//+7/+QkJCAhIQErFmzBoB5tP3dd99FfHw8EhISsGHDBgDm0e4b3njjDaxYsQIAkJ2djdmzZ2PatGl44YUX0NbWZuTqdC85ORkJCQmYOXMmZs6cidOnTxv++00wQSUlJcLEiROF6upqobGxUUhKShIuXbpk7LL04tSpU0JiYqIQFhYmFBQUCM3NzcKECROE/Px8Qa1WCw8//LBw+PBhY5epc0eOHBHmzp0rqFQqobW1VUhOThZSUlJMvu3Hjh0T5s2bJ6jVaqG5uVmYOHGikJ2dbfLtviEtLU0YNWqUsHz5ckEQBCEhIUE4efKkIAiC8NxzzwmbN282Znk6p9FohDFjxghqtbrzOWN8v5nkGUVaWhpiY2Ph5OQEGxsbTJs2Dfv37zd2WXqxfft2rFq1CgqFAgBw5swZ+Pn5wcfHB1KpFElJSSbZdrlcjhUrVsDS0hIWFhYIDAxEXl6eybc9JiYGGzduhFQqRWVlJdrb21FXV2fy7QaAmpoarF27Fo899hgAoKioCC0tLYiKigIA3HfffSbX7tzcXIhEIixZsgQzZszApk2bjPL9ZpJBUVZWBrlc3vlYoVCgtLTUiBXpz6uvvooRI0Z0PjaXtgcHB3d+QeTl5WHv3r0QiURm0XYLCwu89957SEhIwOjRo83mM3/ppZewbNkyODg4AOj6sy6Xy02u3XV1dRg9ejQ++OADfPbZZ9i6dSuKi4sN/nmbZFAIt5gQVyQyj6mIza3tly5dwsMPP4zly5fD19e3y+um2valS5ciPT0d169fR15eXpfXTa3dX331FTw8PDB69OjO58zhZz06Ohpr1qyBjY0NXFxcMGfOHLz33ntdttN3uwf0ehTdUSqVyMjI6HxcVlbW2TVj6pRKJSoqKjofm3LbMzMzsXTpUjz//PNISEjA8ePHTb7tV65cQWtrK0JCQmBtbY2pU6di//79kEgknduYYrv37t2L8vJyzJw5E7W1tWhqaoJIJLrp8y4vLze5dmdkZECtVncGpCAI8PLyMvjPuUmeUcTFxSE9PR1VVVVobm5Gamoqxo8fb+yyDGLYsGG4evUqrl27hvb2duzZs8ck2379+nU88cQTePPNN5GQkADAPNpeWFiIlStXorW1Fa2trTh48CDmzZtn8u3esGED9uzZg127dmHp0qWYNGkSXnvtNchkMmRmZgIAdu7caXLtrq+vx5o1a6BSqdDQ0IAdO3bg73//u8G/30z2jGLZsmVITk6GWq3GnDlzEBkZaeyyDEImk+H111/HU089BZVKhQkTJmD69OnGLkvnPv30U6hUKrz++uudz82bN8/k2z5hwgScPn0as2bNgkQiwdSpU5GQkAAXFxeTbnd33nzzTaxcuRKNjY0IDQ1FcnKysUvSqYkTJ3Z+3hqNBvPnz8fw4cMN/v3GFe6IiEgrk+x6IiIi3WFQEBGRVgwKIiLSikFBRERaMSiIiEgrBgWRDhQWFiI6Ovqm5/bu3YtRo0YhPT3dSFUR6YZJ3kdBZGxbt27Fhx9+iM8++wwhISHGLofojjAoiHRs3bp1+Oabb7BlyxZ4e3sbuxyiO8auJyIdWrNmDd566y089NBDDAkyGQwKIh1pamrCxYsXsW7dOrz11lvIzs42dklEOsGgINIRKysrfPTRR5gwYQIeffRRPPHEE6ipqTF2WUR3jEFBpCNisRgWFhYAgEceeQRBQUF45plnoNFojFwZ0Z1hUBDpgUgkwhtvvIErV67gnXfeMXY5RHeEs8cSEZFWPKMgIiKtGBRERKQVg4KIiLRiUBARkVYMCiIi0opBQUREWjEoiIhIKwYFERFp9f8BtwUUR+xqlkkAAAAASUVORK5CYII=
",
      "text/plain": [
       ""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns; sns.set()
",
    "sns.lineplot(x='K', y='Accuracy', data=df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Finding best K in terms of accuracy
",
    "best_k = df.sort_values('Accuracy', ascending=False)['K'].iloc[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9482517482517483"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Model accuracy on best K
",
    "model(best_k, test_x, test_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
",
       "           metric_params=None, n_jobs=None, n_neighbors=3, p=2,
",
       "           weights='uniform')"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Model evalutation on test set
",
    "clf = KNeighborsClassifier(n_neighbors=best_k)
",
    "clf.fit(train_x, train_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predictions on test set
",
    "test_y_predictions = clf.predict(test_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support
",
      "
",
      "         1.0       1.00      1.00      1.00        21
",
      "         4.0       0.60      0.12      0.21        24
",
      "         5.0       0.58      0.94      0.72        31
",
      "         6.0       1.00      1.00      1.00        24
",
      "
",
      "   micro avg       0.77      0.77      0.77       100
",
      "   macro avg       0.80      0.77      0.73       100
",
      "weighted avg       0.77      0.77      0.72       100
",
      "
"
     ]
    }
   ],
   "source": [
    "# Classification report on test set predictions
",
    "from sklearn.metrics import classification_report
",
    "print(classification_report(test_y, test_y_predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 3: Multiclass Logistic Regression with Elastic Net **(5 Marks)**
",
    "
",
    "Build an elastic-net regularized logistic regression classifier for this data. 
",
    "
",
    "- Elastic-net regularizer takes in 2 parameters: alpha and l1-ratio. Use the following values for alpha: 1e-4,3e-4,1e-3,3e-3, 1e-2,3e-2. Use the following values for l1-ratio: 0,0.15,0.5,0.7,1. Choose the best values of alpha and l1-ratio based on model performance P. ** (2 Marks)**
",
    "
",
    "- Draw a surface plot of F1-score with respect to alpha and l1-ratio values. ** (1 Mark)**
",
    "
",
    "- Use the best value of alpha and l1-ratio to re-train the model on the training set and use it to predict the labels of the test set. Report the confusion matrix, multi-class averaged F1-score and accuracy. ** (1+1=2 Marks)**
",
    "
",
    "*[Hints: To choose the best alpha/l1-ratio value, you have to do the following:*
",
    "- *For each value of hyperparameter, use 10 fold cross-validation to computer the performance P.* 
",
    "- *The best hyperparameter will be the one that gives maximum validation performance.*
",
    "- *Performance is defined as: P='accuracy' if fID=0, P='f1-score' if fID=1. Calculate fID using modulus operation fID=SID % 2, where SID is your student ID. For example, if your student ID is 356289 then fID=(356289 % 2)=1 then use 'f1-score' for selecting the best value of alpha/l1-ratio. ]*
",
    "
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Python/2.7/site-packages/sklearn/linear_model/stochastic_gradient.py:166: FutureWarning: max_iter and tol parameters have been added in SGDClassifier in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.
",
      "  FutureWarning)
"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import SGDClassifier
",
    "
",
    "# Defining hyperparameters for grid searching
",
    "ALPHA = [1e-4,3e-4,1e-3,3e-3, 1e-2,3e-2]
",
    "L1_RATIO = [0,0.15,0.5,0.7,1]
",
    "
",
    "accuracies = []
",
    "def model(alpha, l1_ratio, X, y):
",
    "    """
",
    "     Defining Model and performing 10-fold CV
",
    "     """
",
    "    clf = SGDClassifier(penalty='elasticnet', alpha=alpha, l1_ratio=l1_ratio)
",
    "    scores = cross_val_score(clf, X, y, cv=10)
",
    "    return np.average(np.array(scores))
",
    "
",
    "for alpha in ALPHA:
",
    "    for l1_ratio in L1_RATIO:
",
    "        accuracy = model(alpha, l1_ratio, train_x, train_y)
",
    "        accuracies.append({'alpha': alpha, 'l1_ratio': l1_ratio, 'accuracy': accuracy})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame().from_records(accuracies)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       ""
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png":

SIT720 Machine Learning Assessment Task 4: Individual ML project © Deakin University 1 FutureLearn Assessment 4: Machine Learning Project This document supplies detailed information on assessment...

Solution

Answer To This Question Is Available To Download

Related Questions & Answers

Submit New Assignment