Yioop - PHP Search Engine

2017-10-28

Print confusion matrix.

In your slide on Oct 11 give the formula to calculate the confusion matrix. For example:

True Positive Rate (TPR, aka sensitivity)= number of times model output true when test example was true /number of times model output true.

Let m = number of times model output true , and e = test example was true

1) So if calculating TPR manually, I have to record m when training and e when testing. For mini batch training, the input data will be a batch (e.g 100 snippets) how can I record m, because in 100 snippets may be 10 snippets are true while 90 are false.

2) Similarly, when testing, the whole dataset will be feed to the graph, how can I record e ?

3) Tensorflow has the function to calculate confusion matrix which I tried as below

     confusion_matrix_tf = tf.confusion_matrix(tf.argmax(self.nn_output, 1), tf.argmax(self.y_holder, 1))
     print 'Confusion matrix:', session.run(confusion_matrix_tf, feed_dict={self.input_holder: x, self.y_holder: y})
     
      The output will look something like this

       Confusion matrix: [[   0    0    0]
                                       [   0    0    0]
                                       [5001    0    0]]

       The confusion matrix dimensions is different with what you mention in the slide because we have 6 labels (classes) for this homework 
       
      So, which approach is fit with your assignment requirements? 
        -  If "manually" is your requirement, can you explain how to calculate m and e
        -  If using tensorflow is your prefer, can you give the example and explain the confusion matrix of 3 or 4 classes.

In your slide on Oct 11 give the formula to calculate the confusion matrix. For example: True Positive Rate (TPR, aka sensitivity)= '''number of times model output true''' when '''test example was true'''/number of times model output true. Let m = '''number of times model output true''', and e = '''test example was true''' 1) So if calculating TPR manually, I have to record m when training and e when testing. For mini batch training, the input data will be a batch (e.g 100 snippets) how can I record m, because in 100 snippets may be 10 snippets are true while 90 are false. 2) Similarly, when testing, the whole dataset will be feed to the graph, how can I record e ? 3) Tensorflow has the function to calculate confusion matrix which I tried as below confusion_matrix_tf = tf.confusion_matrix(tf.argmax(self.nn_output, 1), tf.argmax(self.y_holder, 1)) print 'Confusion matrix:', session.run(confusion_matrix_tf, feed_dict={self.input_holder: x, self.y_holder: y}) The output will look something like this Confusion matrix: [[ 0 0 0] [ 0 0 0] [5001 0 0]] The confusion matrix dimensions is different with what you mention in the slide because we have 6 labels (classes) for this homework So, which approach is fit with your assignment requirements? - If "manually" is your requirement, can you explain how to calculate m and e - If using tensorflow is your prefer, can you give the example and explain the confusion matrix of 3 or 4 classes.

2017-10-29

-- Print confusion matrix

Hey Thinh,

For (1) and (2), you should only be computing TPR for the test data.

I tacked on to my confusion matrix slide a little more on how the confusion matrix works when you are in a multi-label setting.

Best,

Chris

Hey Thinh, For (1) and (2), you should only be computing TPR for the test data. I tacked on to my confusion matrix slide a little more on how the confusion matrix works when you are in a multi-label setting. Best, Chris