We used the `mtcars` data in the lecture notes, and also introduced the $k$-fold cross-validation....

Question

We used the `mtcars` data in the lecture notes, and also introduced the $k$-fold cross-validation. For this question you need to complete the following:
* Write a $5$-fold cross-validation code by yourself, using the `lm.ridge()` function to fit the model and predict on the testing data. Choose an appropriate range of lambda values based on how this function specifies the penalty. Obtain the cross-validation error corresponding to each $\lambda$ and produce an intuitive plot to how it changes over different $\lambda$. What is the best penalty level you obtained from this procedure? Compare that with the GCV result. Please note that you should clearly state the intention of each step of your code and state your result. For details regrading writing a report, please watch the `Comment Video on HW` from week 1 webpage, or the discussion broad. * Use the `cv.glmnet()` function from the `glmnet` package to perform a $5$-fold cross-validation using their built-in feature. Produce the cross-validation error plot against $\lambda$ values. Report the `lambda.min` and `lambda.1se` selected $\lambda$ value.

ridge-regression-and-the-bias-variance-trade-off-wcucmcto-wcz2zzxa.pdf also-to-do-wgkhc4q1-e3hygmjg.docx

Robert · Accepted Answer

> library(MASS)
> set.seed(2)
> nsim=1000
> n=100
> lambda=0.9
> allridgebeta=matrix(NA,nsim,2)
> for(i in lambda){
+   for(i  in 1:nsim)
+   {
+     x=mvrnorm(n,c(0,0),matrix(c(1,0.99,0.99,1),2,2))
+     y=rnorm(n,mean = x[,1]+x[,2])
+     allridgebeta[i,]=solve(t(x)%*%x+lambda*n*diag(2))%*%t(x)%*%y  }  
+ }
> allridge
Error: object 'allridge' not found
> allridgebeta
             [,1]      [,2]
   [1,] 0.7389524 0.7294337
   [2,] 0.7077690 0.7233129
   [3,] 0.6920325 0.6835985
   [4,] 0.6486345 0.6530556
   [5,] 0.6794410 0.6695640
   [6,] 0.7051073 0.7004656
   [7,] 0.6940947 0.6474362
   [8,] 0.7269877 0.7021716
   [9,] 0.7116283 0.7273537
  [10,] 0.6897933 0.6716877
  [11,] 0.6642712 0.6723500
  [12,] 0.6862648 0.6720921
  [13,] 0.7197306 0.7300141
  [14,] 0.7126882 0.6773983
  [15,] 0.6848574 0.6933491
  [16,] 0.6854437 0.6712318
  [17,] 0.6267624 0.6454580
  [18,] 0.6877776 0.6793959
  [19,] 0.7157528 0.7414439
  [20,] 0.6671051 0.6796411
  [21,] 0.6557646 0.6652206
  [22,] 0.7185531 0.7357587
  [23,] 0.7082842 0.7153055
  [24,] 0.6580081 0.6378027
  [25,] 0.6286690 0.6388322
  [26,] 0.6763657 0.6951301
  [27,] 0.6732662 0.6609507
  [28,] 0.7442778 0.7269747
  [29,] 0.6870474 0.6822823
  [30,] 0.7414386 0.7347276
  [31,] 0.6947309 0.6867296
  [32,] 0.7104227 0.6828393
  [33,] 0.6826539 0.6954103
  [34,] 0.5895557 0.5910012
  [35,] 0.7161227 0.6988593
  [36,] 0.6784354 0.6672047
  [37,] 0.7262887 0.7170657
  [38,] 0.6542274 0.6504329
  [39,] 0.7941030 0.7847422
  [40,] 0.6808163 0.6755186
  [41,] 0.6930936 0.6990251
  [42,] 0.6712715 0.6501668
  [43,] 0.7232006 0.7275698
  [44,] 0.6615936 0.6561905
  [45,] 0.7662802 0.7457218
  [46,] 0.7237270 0.7206266
  [47,] 0.6390294 0.6383227
  [48,] 0.6723270 0.6843780
  [49,] 0.6918943 0.6733999
  [50,] 0.6883160 0.6872397
  [51,] 0.6871003 0.7055573
  [52,] 0.6650362 0.6381830
  [53,] 0.6466315 0.6375580
  [54,] 0.7226079 0.7464210
  [55,] 0.7489416 0.7291950
  [56,] 0.6923512 0.7177470
  [57,] 0.6736165 0.7285528
  [58,] 0.6758865 0.7152195
  [59,] 0.7259267 0.7142723
  [60,] 0.7230352 0.6754965
  [61,] 0.6501280 0.6725631
  [62,] 0.6490608 0.6506161
  [63,] 0.6818176 0.6847639
  [64,] 0.6206999 0.6598768
  [65,] 0.7193833 0.7461775
  [66,] 0.6707291 0.6449064
  [67,] 0.7008421 0.6935624
  [68,] 0.6683640 0.6861007
  [69,] 0.6659492 0.6845213
  [70,] 0.6688520 0.6794082
  [71,] 0.6491853 0.6463675
  [72,] 0.7159586 0.7042456
  [73,] 0.7348803 0.7479636
  [74,] 0.6562752 0.6874757
  [75,] 0.7176799 0.7120179
  [76,] 0.6612925 0.6806900
  [77,] 0.7063993 0.7124529
  [78,] 0.6962405 0.6870503
  [79,] 0.7490764 0.7161758
  [80,] 0.6636010 0.6763998
  [81,] 0.6946656 0.6685110
  [82,] 0.6838478 0.6845294
  [83,] 0.6795334 0.6790051
  [84,] 0.7330894 0.7293710
  [85,] 0.6766380 0.6919856
  [86,] 0.7642594 0.7324315
  [87,] 0.7592950 0.7538307
  [88,] 0.7060697 0.7220847
  [89,] 0.6811913 0.7231565
  [90,] 0.6624037 0.6587553
  [91,] 0.6537901 0.6610349
  [92,] 0.7147342 0.7257448
  [93,] 0.7226490 0.7437074
  [94,] 0.7043543 0.7068494
  [95,] 0.6567092 0.6718592
  [96,] 0.6620378 0.6642713
  [97,] 0.6658794 0.6674918
  [98,] 0.6144146 0.6305772
  [99,] 0.7134501 0.7020546
 [100,] 0.6445555 0.6619028
 [101,] 0.6784511 0.6671402
 [102,] 0.6832418 0.6891065
 [103,] 0.6997761 0.7039386
 [104,] 0.6930114 0.6972309
 [105,] 0.6710235 0.6676713
 [106,] 0.6860930 0.6902883
 [107,] 0.6770385 0.6497607
 [108,] 0.6914006 0.6900819
 [109,] 0.8145281 0.8169119
 [110,] 0.7114872 0.7163694
 [111,] 0.6614077 0.6437184
 [112,] 0.6489671 0.6632859
 [113,] 0.6519089 0.6666262
 [114,] 0.6788132 0.6838106
 [115,] 0.7299828 0.7393066
 [116,] 0.6842492 0.7025505
 [117,] 0.7074228 0.7195030
 [118,] 0.6872104 0.6720514
 [119,] 0.7477430 0.7317494
 [120,] 0.6279157 0.6314735
 [121,] 0.7204059 0.6779059
 [122,] 0.7694764 0.7518159
 [123,] 0.7124437 0.7277123
 [124,] 0.6898084 0.6887544
 [125,] 0.6351810 0.6531063
 [126,] 0.6856369 0.6847211
 [127,] 0.7075331 0.6832022
 [128,] 0.6567508 0.6660169
 [129,] 0.5919217 0.5959365
 [130,] 0.7364259 0.6997260
 [131,] 0.6658973 0.6534935
 [132,] 0.6123640 0.6149497
 [133,] 0.6946630 0.7047923
 [134,] 0.6869406 0.6481854
 [135,] 0.6444343 0.6322343
 [136,] 0.6668394 0.6849088
 [137,] 0.7519278 0.7257369
 [138,] 0.6743340 0.6666009
 [139,] 0.7933577 0.8009690
 [140,] 0.7058471 0.7394899
 [141,] 0.7462868 0.7253455
 [142,] 0.6464666 0.6290356
 [143,] 0.7212719 0.7042940
 [144,] 0.7292913 0.7335335
 [145,] 0.6180142 0.6465725
 [146,] 0.6303513 0.6397933
 [147,] 0.6197922 0.5860371
 [148,] 0.6397047 0.6553883
 [149,] 0.6827134 0.6830068
 [150,] 0.6417920 0.6401584
 [151,] 0.6342336 0.6155749
 [152,] 0.6692087 0.6646812
 [153,] 0.6198626 0.6231652
 [154,] 0.7125273 0.7025529
 [155,] 0.6858071 0.6834789
 [156,] 0.6181276 0.6278842
 [157,] 0.6509586 0.6472210
 [158,] 0.6385507 0.6586909
 [159,] 0.7087213 0.7095249
 [160,] 0.6836741 0.6845208
 [161,] 0.7516262 0.7658949
 [162,] 0.6949414 0.7321509
 [163,] 0.6563652 0.6357689
 [164,] 0.6457708 0.6677397
 [165,] 0.7304504 0.7024972
 [166,] 0.6980177 0.7031083
 [167,] 0.6563077 0.6674934
 [168,] 0.6833228 0.6702808
 [169,] 0.6774816 0.6768391
 [170,] 0.6081640 0.6227935
 [171,] 0.6583614 0.6682111
 [172,] 0.7108753 0.7286042
 [173,] 0.6851565 0.6852504
 [174,] 0.7181315 0.7277144
 [175,] 0.6301477 0.6365511
 [176,] 0.6478784 0.6421675
 [177,] 0.6814620 0.6793399
 [178,] 0.7017001 0.7004576
 [179,] 0.7199283 0.7192826
 [180,] 0.7312182 0.7300593
 [181,] 0.7112028 0.7386449
 [182,] 0.6532713 0.6353015
 [183,] 0.6526045 0.6638799
 [184,] 0.6537434 0.6335062
 [185,] 0.6524831 0.6622835
 [186,] 0.7471322 0.7316105
 [187,] 0.6473737 0.6419769
 [188,] 0.6533374 0.6593799
 [189,] 0.7280034 0.7106427
 [190,] 0.7120298 0.7066254
 [191,] 0.6606219 0.6087123
 [192,] 0.6873505 0.6928968
 [193,] 0.6071935 0.6046323
 [194,] 0.6512521 0.6574918
 [195,] 0.7463697 0.7457380
 [196,] 0.6949842 0.7091577
 [197,] 0.6886831 0.6710035
 [198,] 0.6385254 0.6252832
 [199,] 0.7001493 0.7060116
 [200,] 0.7419171 0.7261673
 [201,] 0.6528970 0.6813738
 [202,] 0.6564823 0.6632605
 [203,] 0.6278699 0.6614065
 [204,] 0.6237987 0.6024441
 [205,] 0.6336605 0.6055392
 [206,] 0.6876407 0.6808091
 [207,] 0.6300581 0.6572785
 [208,] 0.7312763 0.7150187
 [209,] 0.6775512 0.6739138
 [210,] 0.6988106 0.7075290
 [211,] 0.6250965 0.6098240
 [212,] 0.6979451 0.7299821
 [213,] 0.6461386 0.6377946
 [214,] 0.5865553 0.5851633
 [215,] 0.7409691 0.7425174
 [216,] 0.6799459 0.6773195
 [217,] 0.5389710 0.5697125
 [218,] 0.7307889 0.7172438
 [219,] 0.7018655 0.6785799
 [220,] 0.6279856 0.6362603
 [221,] 0.7053955 0.6811361
 [222,] 0.6919720 0.7283655
 [223,] 0.6477783 0.6779867
 [224,] 0.6617265 0.6862306
 [225,] 0.7206971 0.7093053
 [226,] 0.7219330 0.7237765
 [227,] 0.6375829 0.6553047
 [228,] 0.7062510 0.6889642
 [229,] 0.6660188 0.6753649
 [230,] 0.7174401 0.7167230
 [231,] 0.7187021 0.7132055
 [232,] 0.7593531 0.7314633
 [233,] 0.7199457 0.6879721
 [234,] 0.6788201 0.6625146
 [235,] 0.6029288 0.5856765
 [236,] 0.6868495 0.6991774
 [237,] 0.6936096 0.6910464
 [238,] 0.7713016 0.7932740
 [239,] 0.7447803 0.7347986
 [240,] 0.6840325 0.6909074
 [241,] 0.7387234 0.7424752
 [242,] 0.7261938 0.7285364
 [243,] 0.6612881 0.6803125
 [244,] 0.6188530 0.6169640
 [245,] 0.6171705 0.6467993
 [246,] 0.7199133 0.7381158
 [247,] 0.7077606 0.7072651
 [248,] 0.6694994 0.6799761
 [249,] 0.6812093 0.6726553
 [250,] 0.7283383 0.7298808
 [251,] 0.6669434 0.6689352
 [252,] 0.7188229 0.7112304
 [253,] 0.6495821 0.6662271
 [254,] 0.6887372 0.6962643
 [255,] 0.6919647 0.7060185
 [256,] 0.7650472 0.7499832
 [257,] 0.6821010 0.7054979
 [258,] 0.6933143 0.7023347
 [259,] 0.7056755 0.7337125
 [260,] 0.7311842 0.7293319
 [261,] 0.6438946 0.6536721
 [262,] 0.7592535 0.7544752
 [263,] 0.6610529 0.6443233
 [264,] 0.5631444 0.5800831
 [265,] 0.7349365 0.7402307
 [266,] 0.6453862 0.6503205
 [267,] 0.5695209 0.5779737
 [268,] 0.6642748 0.6468120
 [269,] 0.6256396 0.6264312
 [270,] 0.7143611 0.7006545
 [271,] 0.6340024 0.6319773
 [272,] 0.6520180 0.6755251
 [273,] 0.7278351 0.6945972
 [274,] 0.6890067 0.6765668
 [275,] 0.6410469 0.6391050
 [276,] 0.7286407 0.7102486
 [277,] 0.6637165 0.6149366
 [278,] 0.7312348 0.7239802
 [279,] 0.6090046 0.6016778
 [280,] 0.7369604 0.7593658
 [281,] 0.7042370 0.6957684
 [282,] 0.6890935 0.7060871
 [283,] 0.6762016 0.7078844
 [284,] 0.7111281 0.6977876
 [285,] 0.7009408 0.6804474
 [286,] 0.6656447 0.6952991
 [287,] 0.6851282 0.6991748
 [288,] 0.6468834 0.6440928
 [289,] 0.7008427 0.7022022
 [290,] 0.6325470 0.6444935
 [291,] 0.7837527 0.7806910
 [292,] 0.7032622 0.7075240
 [293,] 0.6942236 0.7272430
 [294,] 0.6624292 0.6768260
 [295,] 0.6511714 0.6496781
 [296,] 0.6631962 0.

We used the `mtcars` data in the lecture notes, and also introduced the $k$-fold cross-validation. For this question you need to complete the following: * Write a $5$-fold cross-validation code by...

Solution

Answer To This Question Is Available To Download

Related Questions & Answers

Submit New Assignment