Testing for Information with Brain Decoding

"Is there information about the stimulus given to the subject within brain data?" The brain decoding approach tries to answer this question by means of machine learning algorithms. A classifier is learned from a small sample of brain data that is class-labeled according to the stimuli provided to the subject during the experiment. The classifier is tested on a different small sample, the test set, in order to observe the number of misclassifications. The idea is that accurate prediction provides evidence of the presence of information about stimuli within brain data. In this work we show the connection between information theory and learning theory in order to bridge the gap between the initial information question and the observed number of classification errors on a small test set. We propose a hierarchical model about this connection and a related statistical test about the presence of information. This test lies within the Bayesian hypothesis testing framework and is compared against the classical binomial test of the null hypothesis testing framework. We show the empirical similarity between the two tests and present an application on a real neuroimaging dataset about a covert spatial attention task.