Post to this thread your solutions to the Nov 8 In-class Exercise.

Best,

Chris

W' = [[2,0,0], [0,0,-1], [0,3,0]] x = [-1,3,-3]

For OMP -2 number of non zero entries in h is 1

Case 1 : Let h = [c,0,0] || x - W'h|| = (1+2c)^2 + 9 + 9 --> min value . = 18 when c = -1/2

Case 2: || x - W'h|| = (3+3c)^2 + 1 + 9 --> min value . = 10 when c = -1 and h=[0,-1,0]

Case 3: || x - W'h|| = (3+c)^2 + 1 + 9 --> min value . = 10 when c = -3 and h=[0,0,-3]

Both case 2 and case 3 provide the answer

• assuming h = [0, 0, C]
• W' * h = [ 0, -C, 0]
• X - W'*h = [-1, 3+C, -3]
• argmin = 1+(3+c)^2 + (-3)^2 ==> C = -3
• h = [0, 0, -3]
• we still need to do the two other cases. h=[0,C,0], h=[C,0,0]

we have w as a 3x3 matrix, x = 3x1 matrix, h = 3x1 matrix, since k = 2, h can only contain 1 non zero entry.

write the magnitude as a system of equations, distance = ||x - w*h|| solving for h1, h2, h3 we get h1 = -1/2, h2=3, h3=1

evaluate the arrays for each of these h values with the other entries as 0.

import numpy as np

x = np.matrix([[-1], [3], [-3]])
w = np.matrix([[2, 0, 0], [0, 0, -1], [0, 3, 0]])
print(x)
print(w)

h = np.matrix([[-1 / 2], [0], [0]])
distance = np.linalg.norm(x - w * h)
print(distance)

h = np.matrix([[0], [3], [0]])
distance = np.linalg.norm(x - w * h)

print(distance)

h = np.matrix([[0], [0], [1]])
distance = np.linalg.norm(x - w * h)
print(distance)

4.24264068712
12.409673646
5.09901951359

since h1 = -1/2 had the lowest distance, we can use h = [-1/2,0,0]

Still Working

W' = [[2,0,0],[0,0,-1],[0,3,0]] x=[[-1],[3],[-3]]

k=2, h has 1 non-0 entry

x - W' * h

1. h = [c,0,0]
2. h = [0,c,0]
3. h = [0,0,c]

W' = [[2,0,0],[0,0,-1],[0,3,0]] x = [-1,3,-3]

let h = [a, b, c] W'h = [2a, -b, 3c]

If we can have X = W'h, then we will have norm to be zero. But, in that case we might not have two out of those 3 entries to be zero.

So finding [h1, h2, h3] which makes X = W'h, i.e. making h = h = [-1/2, -1, -3] and then finding x-W'h for
[h1,0,0] as (1), [0,h2,0] as (2) [0,0,h3] as (3)

and comparing (1), (2) and (3) will 3 different values. taking the h which minimizes ||x- W'h||^2 gives the solution. In our case, (2) and (3) give the result with ||x- W'h||^2 being minimum.

h can be [0, -1, 0] or [0, 0, -3].

W’ = [[2,0,0], [0,0,-1], [0,3,0]] x = [-1,3,-3]

For OMP -2 number of non zero entries in h is 1

Case 1 : Let h = [c,0,0] || x - W’h|| = (1+2c)^2 + 9 + 9 --> min value . = 18 when c = -1/2

Case 2: || x - W’h|| = (3+3c)^2 + 1 + 9 --> min value . = 10 when c = -1 and h=[0,-1,0]

Case 3: || x - W’h|| = (3+c)^2 + 1 + 9 --> min value . = 10 when c = -3 and h=[0,0,-3]

Both case 2 and case 3 provide the answer