-- Nov 8 In-Class Exercise

let h=[0 , 0 , b]T W'h= [0 , -b , 0] Minimising ||x-W'h||2 Is the same as minimising sqr(|-1|) + sqr(3+b) + sqr(|-3|) h=[0, 0, -3] if h = [0, b, 0] h=[0, -1, 0] if h = [b, 0, 0] h=[-0.5, 0, 0]

-- Nov 8 In-Class Exercise

Since the number of non-zero entries in h must be at most 1, there are 3 possibilities for the solution to OMP-2, i.e. the vectors [-1/2, 0, 0], [0, -3, 0], [0, 0, 1]. I found that the last vector gave the smallest value for the OMP equation and thus is the solution for h.

-- Nov 8 In-Class Exercise

((resource:img1.jpeg|Resource Description for img1.jpeg)) ((resource:img2.jpeg|Resource Description for img2.jpeg))

-- Nov 8 In-Class Exercise

@BT@W’ = [[2,0,0], [0,0,-1], [0,3,0]]@BT@ @BT@x = [−1,3,−3]@BT@ k=2 , h has one non-zero entry Taking h which minimizes ||x- W'h||^2 gives us the solution Let h = [c, 0, 0] = [-0.5, 0, 0]. (1) @BT@||x-W'h|| = (1+2c)^2 + 9 + 9@BT@ gives min. value = 18 when c = -1/2 Let h = [0, c, 0] = [0, -1, 0]. (2) @BT@||x-W'h|| = (3 + 3c)^2 + 1 + 9@BT@ gives min. value = 10 when c = -1 Let h = [0, 0, c] = [0, 0, -3] (3) @BT@||x-W'h|| = (3+c)^2 + 1 + 9@BT@ gives min. value = 10 when c = -3 Both (2) and (3) gives the answer

-- Nov 8 In-Class Exercise

<nowiki> W’ = [[2,0,0],[0,0,-1],[0,3,0]] X = [-1,3,-3] We need to minimise for h, the value of ||x - W’h||^2, such that h0<k. Given that we are using OMP-2, we have k=2. Thus, h0=1. Case 1 : Let h = [a 0 0]: Put this in ||x - W’h||^2, we get, (-1-2a)^2 + 9 + 9 Min val of this expression is when first term is 0. => -1-2a = 0, => a = 1/2, and the min value is 18. Case 2: h = [0 b 0]: Put this in ||x - W’h||^2, we get, 1 + 9 + (-3-3b)^2 Min val of this expression is when last term is 0. => -3-3b = 0, => b = -1, and the min value is 10. Case 3: h = [0 0 c] Put this in ||x - W’h||^2, we get, 1 + (3+c)^2 + 9 Min val of this expression is when second term is 0. => -3+c = 0, => c = -3, and the min value is 10. Both Case 2 & Case 3, give same min val, so either could be used. Thus, h = [0 -1 0] or h = [0 0 -3]</nowiki>

-- Nov 8 In-Class Exercise

((resource:cs256-inclass-11-8.jpeg|Resource Description for cs256-inclass-11-8.jpeg))

-- Nov 8 In-Class Exercise

((resource:WhatsApp Image 2017-11-10 at 3.19.00 PM.jpeg|Resource Description for WhatsApp Image 2017-11-10 at 3.19.00 PM.jpeg))

-- Nov 8 In-Class Exercise

W' = [[2,0,0],[0,0,-1],[0,3,0]] x = [-1,3,-3] let h = [a, b, c] W'h = [2a, -b, 3c]

-- Nov 8 In-Class Exercise

W' = [[2 0 0], [0 0 -1], [0 3 0]] x = [-1 3 -3] Using OMP-2, k = 2 Minimizing ||x - W'h||^2, Case 1: h = [c 0 0] h = [-1/2 0 0] (1 + 2c)^2 + 9 + 9 --> min value = 18 Case 2: h = [0 c 0] h = [0 -1 0] (3 + 3c)^2 + 1 + 9 --> min value = 10 Case 3: h = [0 0 c] h = [0 0 -3] (3 + c)^2 + 1 + 9 --> min value = 10 So, below h-vectors are valid solutions: [0 -1 0] [0 0 -3]

-- Nov 8 In-Class Exercise

((resource:In class 2.jpeg|Resource Description for In class 2.jpeg)) ((resource:In class 1.jpeg|Resource Description for In class 1.jpeg))