Please post your solutions to the May 11 In-Class Exercise to this thread.

Best,

Chris

P@100 = x N = 5000 = corpus K = 100 = batch N/X = # batches = 5000/100 = 50 batches N-X = threshold = 5600 - 100 = 4900 K=pN > 100 = p*5000 >p = row = 0.02

procedure would follow as: compute threshold (4900), go through all 50 batches, get document that scores over 4900, determine the relevance ( human test), then calculate precision and the average of all the batches, Compute aggregate P@100.

N=5000 X = 100 documents N / X = 5000/ 100 = 50 batches N - X = 5000 - 100 = 4900

Check for every doc if the score is greater than 900 and if the doc is relevant. Then the list is made: successful_docs The precision for a batch = length / succesful_docs

P@100 = Average of the precision for each batch.

N=5000 X = 100 documents 
N / X = 5000/ 100 = 50 batches
N - X = 5000 - 100 = 4900
go through all 50 batches
get document that scores over 4900
determine the relevance 
Compute precision 
Compute the average of all the batches
Calculate aggregate P@100 

N=5000 documents
Batch size is 100
X documents score > N−X documents

Number of batches= 5000/100= 50

To compute aggregate P@k,
1) Scan all the batches to find relevant documents that score over the threshold.
2) Check whether the document is relevant to query.
3) Calculate precision
4) Calculate average precision  
5) Caluclaute aggregate precision P@100 where
if 100 documents score over threshold here N_X. ρ =k/N= 100/5000= 0.02 where X= 100

N=5000 X = 100 documents, N / X = 5000/ 100 = 50, N - X = 5000 - 100 = 4900 check whichever documents score is greater than 4900 check the relevancy of the document calculate precision find out the average of all the documents Calculate aggregate P@100

Given, N = 5000 X = 100 Number of batches = N/X = 50 Threshold score = N - X = 4900

For every batch, 1. Find documents above the score of 4900 and determine if it is relevant or not 2. Calculate precision 3. Calculate the average precision of all batches

Calculate aggregate P@100