Problem Statement To perform Topic Modelling using RBM on a set of documents and print the distribution of words in each topic.In this assignment, we shall create a function to perform Gibbs sampling...

RBM_problem_statement.ipyn
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Restricted Boltzman Machine \n",
"\n",
"A restricted Boltzmann Machine is an \"Energy Based\" generative stochastic model. They were initially invented by Paul Smolensky in 1986 and were called \"Harmonium\". After the evolution of training algorithms in the mid 2000's by Geoffrey Hinton, the boltzman machine became more prominent. They gained big popularity in recent years in the context of the Netflix Prize where RBMs achieved state of the art performance in collaborative filtering and have beaten most of the competition.\n",
"\n",
"RBM's are useful for dimensionality reduction, classification, regression, collaborative filtering, feature learning and topic modeling. \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Architecture\n",
"RBMs are shallow, two-layer neural nets that constitute the building blocks of deep-belief networks. The first layer of the RBM is called the visible, or input, layer, and the second is the hidden layer. The absense of an output layer is apparent. As we move forward we would learn why the output layer won't be needed.\n",
"\n",
"Figure1: Layers in RBM\n",
"Each circle in the figure above represents a neuron-like unit called a node, and nodes are simply where calculations take place. \n",
"\n",
"Figure2: Structure of RBM\n",
"The nodes are connected to each other across layers, but no two nodes of the same layer are linked. That is, there is no intra-layer communication – this is the restriction in a restricted Boltzmann machine. Each node is a locus of computation that processes input, and begins by making stochastic decisions about whether to transmit that input or not. Each visible node takes a low-level feature from an item in the dataset to be learned."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's start by importing the packages that will be required in this project."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from collections import Counter, defaultdict\n",
"import pandas as pd\n",
"from scipy.sparse import coo_matrix, hstack\n",
"from sklearn.feature_extraction.text import CountVectorizer\n",
"np_rng = np.random.RandomState(1234) #setting the random state"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# graded\n",
"# import data\n",
"\n",
"df = pd.read_excel('amazon.xlsx')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"