2017-09-09

Experiments PAC learnable without δ ? .

Hi,
I understand that with the num_test number of examples our program should ideally work with error rate <= epsilon.
However, PAC learning model tells us that it should arrive at this value with a probability of 1-δ. It would be great if you can let us know what should be proved by our experiments without knowledge of δ
Thanks.
(Edited: 2017-09-09)
Hi, I understand that with the num_test number of examples our program should ideally work with error rate <= epsilon. However, PAC learning model tells us that it should arrive at this value with a probability of 1-δ. It would be great if you can let us know what should be proved by our experiments without knowledge of δ Thanks.

-- Experiments PAC learnable without δ ?
Hey Vasudha,
Run multiple experiments. Does the fraction of times for which the value is less than `epsilon` agree with the `1-delta` value? Remember the number of trials needed for PAC learning is a polynomial function of both `1/delta` and `1/epsilon`. If you log plot things the exact degree of the polynomial won't matter.
Best, Chris
(Edited: 2017-09-09)
Hey Vasudha, Run multiple experiments. Does the fraction of times for which the value is less than @BT@epsilon@BT@ agree with the @BT@1-delta@BT@ value? Remember the number of trials needed for PAC learning is a polynomial function of both @BT@1/delta@BT@ and @BT@1/epsilon@BT@. If you log plot things the exact degree of the polynomial won't matter. Best, Chris

-- Experiments PAC learnable without δ ?
Hi Dr. Pollett,
.
Thanks for responding to Vasudha's question as I had similar thoughts to her. How can estimate the `1-\delta` value to determine if the `\epsilon` "agrees" with it? My understanding from class was that the estimate of PAC learnability was more of a squishy/non-rigorous determination which considered whether if the trend for `n` seemed reasonable in particular on the logarithmic scale. Is this incorrect?
.
Also, one thing you mention here is to plot the trend. In my reports, I have maybe a dozen or so graphs with "num_train" on the x-axis and error rate on the y-axis. These graphs show the performance for the different generating distributions (e.g., bool and sphere), activation functions, and ground formulas. Does this align with your expectation? In the homework description, it mentioned to summarize our results in a text file. However, I did not see how a text formatted file could communicate these trends well so I wrote in LaTeX with pgfplots. Maybe I am overthinking, but your comment about plots made it seem like a text file may not be the right medium.
(Edited: 2017-09-10)
Hi Dr. Pollett, . Thanks for responding to Vasudha's question as I had similar thoughts to her. How can estimate the @BT@1-\delta@BT@ value to determine if the @BT@\epsilon@BT@ "agrees" with it? My understanding from class was that the estimate of PAC learnability was more of a squishy/non-rigorous determination which considered whether if the trend for @BT@n@BT@ seemed reasonable in particular on the logarithmic scale. Is this incorrect? . Also, one thing you mention here is to plot the trend. In my reports, I have maybe a dozen or so graphs with "num_train" on the x-axis and error rate on the y-axis. These graphs show the performance for the different generating distributions (e.g., bool and sphere), activation functions, and ground formulas. Does this align with your expectation? In the homework description, it mentioned to summarize our results in a text file. However, I did not see how a text formatted file could communicate these trends well so I wrote in LaTeX with pgfplots. Maybe I am overthinking, but your comment about plots made it seem like a text file may not be the right medium.
X