Assignment-1-CS522-overview-slides
Assignment #1
Autism Detection
Overview
● Research has shown that gaze and fixation vision patterns differ for
people with autism when compared to those without
● In this assignment, you will use data gathered by recording people’s gaze
patterns as they watch videos
Sample Video
https:
docs.google.com/file/d/1RfZo7LL5ldb7vr55ZW_A2m7ViKNgOBoR/preview
Data
● The data comes from a study involving 60 individuals
○ 25 non-autistic (aka control group), 35 autistic
○ Gaze was monitored using an off-the-shelf eye tracker
● The data for the participants has already been processed into .npy files
for you
○ Check out this resource if you are unfamiliar:
https:
towardsdatascience.com/what-is-npy-files-and-why-you-should-use-them-603373c
78883
https:
towardsdatascience.com/what-is-npy-files-and-why-you-should-use-them-603373c78883
https:
towardsdatascience.com/what-is-npy-files-and-why-you-should-use-them-603373c78883
Your Jo
● You are going to perform a simple data analysis pipeline
● Your job is to build a classification model that accurately predicts whether
or not someone is autistic based on their gaze patterns
● The goal of this assignment is to better understand feature
engineering and how that affects the final classification model
● Skeleton.ipynb is available on Canvas
○ Contains 7 tasks for you to complete
○ Most tasks already have code
○ Some contain questions or require explanations; do not forget to leave responses!
Task #1 - Data Loading
● Read in the .npy video files and process them into X and Y vectors
● This is done for you
● For +5 extra credit, you may read in additional videos
Task #2 - Feature Engineering
● Calculate features to enhance your dataset
○ You may choose which features you would like to calculate
○ These can be statistical features (min, max, mean, etc.) or something more complex
○ Be creative!
● You should modify the function featurize_input(X,Y) to do the feature
engineering
○ You can change the parameters of the function (or add additional parameters) if
necessary (just note you may also need to modify the load functions)
● The functions load_data_autistic_fv() and load_data_non_autistic_fv() are
written for you and call featurize_input(X,Y)
Task #3 - Balancing Classes
● Determine whether you need to re-balance your class labels
● Choose a method for re-balancing your data
○ Make sure to explain why you chose your method!
● If you are unfamiliar with class re-balancing the method I prefer is called
SMOTE.
○ You can read up on it here:
https:
towardsdatascience.com/applying-smote-for-class-imbalance-with-just-a-few-lines
-of-code-python-cdf603e58688
https:
towardsdatascience.com/applying-smote-for-class-imbalance-with-just-a-few-lines-of-code-python-cdf603e58688
https:
towardsdatascience.com/applying-smote-for-class-imbalance-with-just-a-few-lines-of-code-python-cdf603e58688
Task #4 - Creating Classification Labels
● Use your autistic and control groups to assign the actual (read: real,
ground truth) labels to the dataset
○ Remember classification is a supervised learning task
● Form your data and labels vectors to be used in the next steps
○ This is also done for you!
○ Use the “sanity check” to confirm your data is co
ect shape
Task #5 - Normalizing
● Determine if the data needs to be normalized
○ Run the code and interpret the plots
○ Does the distribution look normal?
● Perform normalization of the data (regardless of whether you think it
should be normalized or not)
○ Code for RobustScalar as studied in class is already written
○ You may modify this if you like
Task #6 - SVM Classification
● We have written a basic SVC model for you
○ Your job is to understand the effects of the feature engineering from task 2
○ You should re-run your SVC with multiple different combinations of feature engineering!
● Briefly explain your model performance and how the feature engineering
played a role
○ You may use any statistics/graphs to support your answe
Task #7 - SVM Parameter Tuning
● SVM has many hyperparameters that could be tuned to produce a better
model
● In this task, we expect you to reuse the code from task 6 but vary the
hyperparameters to build two new models
○ You MUST try two different kernels
○ You should use the BEST set of features that you discovered in task 6
○ CS Majors are encouraged to be creative and also try tuning other parameters!
● The default configs for scikit’s version of SVC are in the docs:
https:
scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
https:
scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
Grading Scale
Task # XXXXXXXXXX% (written for you)
Task # XXXXXXXXXX%
Task # XXXXXXXXXX%
Task # XXXXXXXXXX% (written for you)
Task # XXXXXXXXXX%
Task # XXXXXXXXXX% (explanation of models)
Task # XXXXXXXXXX% (5% per kernel, 5% explanation)
Note: -10 penalty if none of the classifiers (from task 6) reach an accuracy of ~95%
What you should submit on Canvas:
1. Your assignment.ipynb file
2. assignment.pdf of your completed (and run) assignment.ipynb file
To get assignment.pdf (assuming you are in Jupyter):
● Select “run all” under the “Runtime” tab at the top of your environment
● When it completed running, select “download as HTML” under the “File”
ta
● Right click on the HTML document and save to PDF
A Note on Environment:
You should be using the version of Python XXXXXXXXXXand packages as
outlined in Environment.pdf under the Assignment #1 Canvas page.
Failure to follow these environment constraints may result in a 0 on the
assignment.
Please consult an instructor if you cannot meet the environment constraints
or have questions in getting set-up.
Another Note on Discord:
● If you have doubts regarding code, instructions, ideas, mistakes,
solutions, etc., please use Discord!
● We are fine answering questions using email, but it’s better to discuss
anything using Discord since everybody can see that “you are not alone”.
● We are not experts on all the topics, but we all can contribute to share our
knowledge. We all are learning!
Environment.docx
Environment Setup – COSC 522 Machine Learning
Prof. Sai Swaminathan
To make things easy to help groups and provide useful feedback, we are providing the only commands
that are necessary for assignment 1. On the one hand you may feel restricted in some sense (keep in
mind that for the final project you will have more freedom in terms of software), but on the other hand,
you have to wo
y about just accomplishing the tasks of the assignment and nothing else.
Install Anaconda
● https:
docs.anaconda.com/anaconda/install
Anaconda Prompt
1. Open the Anaconda Prompt
2. Create a new environment by typing:
conda create --name ml python=3.8
a. ml is just the name of the environment.
. If at some point of the semester, the environment is messed up, just create another one
(with a different name).
3. Once you have an environment, activate it by using:
conda activate ml
4. Install Jupyter and some dependencies in this environment such as
pip install jupyterla
pip install -U scikit-learn
pip install matplotli
pip install seaborn
pip install sklearn-evaluation
https:
docs.anaconda.com/anaconda/install