Great Deal! Get Instant $25 FREE in Account on First Order + 10% Cashback on Every Order Order Now

Apache Spark: Movie Recommendation Engine</o:p> Dataset: MovieLens </o:p> Output:Jupyter Notebook (please display the output results)</o:p> I think I got the majority of the...

1 answer below »

Apache Spark:Movie Recommendation Engine

Dataset:MovieLens

Output:Jupyter Notebook (please display the output results)

I think I got the majority of the programming completed but my spark crash and Iam trying to fix it.

Requirements for exercise:

In this exercise, you will create a movie recommendation engine from the MovieLens data. You can download that data by clicking here:MovieLens.

1. Movie Recommendation Engine

a. Prepare Data

Load the data from the ratings.csv and movies.csv files and combine them on movieId. The resultant data set should contain all of the user ratings and include movie titles. The schema should look something like this.

b. Train Recommender

Using the data you prepared in the last step, create a movie recommendation model using collaborative filtering.Spark’s collaborative filtering documentationprovides a template for building and testing this model.

Before you train the recommendation model, split the data into a training dataset and a testing dataset using the randomSplit dataframe method. Use 80% of your data for training and 20% for testing.

After fitting your model using the training dataset, calculate the predictions on the test dataset and use the RegressionEvaluator to calculate the root-mean-square error of the model.

As a reminder,Spark’s collaborative filtering documentationwill be helpful in completing this task.

c. Generate top 10 movie recommendations

Using the recommendation model, generate the top ten recommendations for each user. Using the show method, print the recommendations for the user IDs, 127, 151, and 300. You should not truncate the results and so should call the show method like this recommendations_127.show(truncate=False).

Answered Same DayNov 08, 2021

Solution

Neha answered on Nov 11 2021
53 Votes
{
"cells": [
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"