Theory (40 points/10 points each)1. We have three vectors, v1=(1, 2, 5), v2=(4, 2, 7), and v3=(6, 6,...

Question

Theory (40 points/10 points each)1. We have three vectors, v1=(1, 2, 5), v2=(4, 2, 7), and v3=(6, 6, 17). Please check that these vectors are linearly independent or not. If these three vectors are linearly dependent, present v3 as a linear combination of v1 and v2. (you can use python code for checking linear dependency).2. We have two vectors, v1=(2, 4, 1) and v2=(3, -1, -3). Please check that these vectors are orthogonal or not. If they are not orthogonal, please find the projected vector of v1 onto v2 (Pv2(v1)). (Do not use a python code).3. We have a matrix, . Using rank, please check that this matrix is a singular matrix or not. If this is a singular matrix, find coefficients a and b, which satisfies this equation with 1st row (v1) and 2nd row (v2) of the matrix: av1 + bv2 = 0. (you can use a python code)4. We have a matrix,  . Calculate the inverse matrix and check this is a singular matrix or not. (you can use a python code)Practice (60 points)1. Least-squares (30 points) (U: uniform distribution, N: normal distribution)a. Make 1,000 noise data by x ~ U[0, 1], y = 2+ 3x + 0.1e (e ~N(0, XXXXXXXXXXpoints). Do least-squares method by solving linear equation (refer to our lecture slide) and find an approximated linear function. (10 points)c. Draw scatter plot for (x,y) and the approximated linear function on the same graph. (10 points)2. Singular Value Decomposition (30 points) (U: uniform distribution, N: normal distributiona. Using same data (x,y) in practice problem #1.. Do SVD (refer to our lecture slide) and reduce a noise. (10 points)c. Do least-squares method for denoised data from b and find an approximated linear function (10 points)d. Draw scatter plot for denoised (x,y) and the approximated linear function on the same graph. (10 points)

Abr Writing · Accepted Answer

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np
",
    "import sys
",
    "import matplotlib.pyplot as plt
",
    "import scipy.optimize as optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Theory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Linear Independence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2.13798760e+01 1.70320365e+00 1.17478406e-15]
",
      "3
"
     ]
    }
   ],
   "source": [
    "v1 = [1,2,5]
",
    "v2 = [4,2,7]
",
    "v3 = [6,6,17]
",
    "
",
    "A = np.row_stack([v1, v2, v3])
",
    "
",
    "# MATLAB definition of tolerance
",
    "eps = np.finfo(np.linalg.norm(A).dtype).eps
",
    "TOLERANCE = max(eps * np.array(A.shape))
",
    "
",
    "U, s, V = np.linalg.svd(A)
",
    "print(s)
",
    "print(np.sum(s > TOLERANCE))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2
"
     ]
    }
   ],
   "source": [
    "TOLERANCE = 1e-14
",
    "print(np.sum(s > TOLERANCE))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see if you choose too small a TOLERANCE, nothing looks like zero. T he finding with TOLERANCE=1e-14 implies the rows are not linearly independent. Let's see that one row can be represented as the other rows' linear combination."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number of rows is greater than the rank, so these vectors are not independent. Let's demonstrate that one vector can be defined as a linear combination of the other two vectors. Mathematically we represent this as:
",
    "
",
    "$x_1 \mathit{v_1} + x_2 \mathit{v_2} = v_3$
",
    "
",
    "or
",
    "
",
    "$[x_1 x_2][v_1; v_2] = v_3$
",
    "
",
    "This is not the usual linear algebra form of Ax = b. To get there, we transpose each side of the equation to get:
",
    "
",
    "$[v_1^T v_2^T][x_1; x_2] = v_3^T$
",
    "
",
    "which is the form Ax = b. We solve it in a least-squares sense."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2. 1.]
"
     ]
    }
   ],
   "source": [
    "A = np.column_stack([v1, v2])
",
    "x = np.linalg.lstsq(A, v3, rcond=1)
",
    "print(x[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows that $v_3=2v_1+v_2$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We say that 2 vectors are orthogonal if they are perpendicular
",
    "to each other. i.e. the dot product of the two vectors is zero
",
    "
",
    "$(2,4,1).(3,-1,-3)=6-4-3=-1$
",
    "
",
    "Therefore, the two vectors are not orthogonal to each other. Now finding the project of $v_1$ on $v_2$.
",
    "
",
    "$Proj_{v_2}(v_1)=\frac{v_1.u_1}{|v_2|^2}v_2=\frac{-1}{\sqrt{19}}(3,-1,-3)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix is Singular
"
     ]
    }
   ],
   "source": [
    "A = [[1,3],[2,6]]
",
    "if np.linalg.cond(A) "
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-5,5,100)
",
    "y = -1/3*x
",
    "plt.plot(x, y, '-r', label='x+3y=0')
",
    "plt.plot(x, y, 'og', label='2x+6y=0')
",
    "plt.xlabel('x', color='#1C2833')
",
    "plt.ylabel('y', color='#1C2833')
",
    "plt.legend(loc='upper left')
",
    "plt.grid()
",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are infinte many solutions, since we know that Matrix is Singular. The coefficients (a, b), which satisfies this equation with 1st row ($v_1$) and 2nd row ($v_2$) of the matrix: $av_1 + bv_2 = 0$ are (2, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix is Singular
"
     ]
    }
   ],
   "source": [
    "A=[[1,4,-1],[3,2,7],[2,5,1]]
",
    "if np.linalg.cond(A) "
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y, '-r', label='Actual')
",
    "plt.plot(x, y2, 'og', label='LS Approximations')
",
    "plt.title('y = 2+3*x+e')
",
    "plt.xlabel('x')
",
    "plt.ylabel('y')
",
    "plt.legend(loc='upper left')
",
    "plt.grid()
",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.transpose([x,y])
",
    "
",
    "# Singular-value decomposition
",
    "U, s, VT = np.linalg.svd(A)
",
    "# create m x n Sigma matrix
",
    "Sigma = np.zeros((A.shape[0], A.shape[1]))
",
    "# populate Sigma with n x n diagonal matrix
",
    "Sigma[:A.shape[1], :A.shape[1]] = np.diag(s)
",
    "B = U.dot(Sigma.dot(VT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = B[:,0]
",
    "y = B[:,1]
",
    "
",
    "sol = optimization.curve_fit(opt, x, y, x0, [1.0]*1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "y2 = sol[0][0] + sol[0][1]*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png":

Theory (40 points/10 points each) 1. We have three vectors, v1=(1, 2, 5), v2=(4, 2, 7), and v3=(6, 6, 17). Please check that these vectors are linearly independent or not. If these three vectors are...

Solution

Answer To This Question Is Available To Download

Related Questions & Answers

Submit New Assignment