to be the vector space of all possible inputs, and to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space , i.e. there exists some unknown . The training set is made up of samples from this probability distribution, and is notated

Every is an input vector from the training data, and is the output that corresponds to it.