More Related Content
Similar to Inductive analytical approaches to learning (20)
Inductive analytical approaches to learning
- 2. 2.1 The Learning Problem • Given: • A set of training examples D, possibly containing errors. • A domain theory B, possibly containing errors. • A space of candidate hypotheses H. • Determine: • A hypothesis that best fits the training examples and domain theory.
- 3. • There are 2 approaches: • 1. To find best fit find errorD(h),errorB(h) errorD(h) is defined to be the proportion of examples from D that are misclassified by h. errorB(h) of h with respect to a domain theory B to be the probability that h will disagree with B on the classification of a randomly drawn instance. we could require the hypothesis that minimizes some combined measure of these errors, such as
- 4. • It is not clear what values to assign to kB and kD to specify the relative importance of fitting the data versus fitting the theory. If we have a very poor theory and a great deal of reliable data, it will be best to weight errorD(h) more heavily. • Given a strong theory and a small sample of very noisy data, the best results would be obtained by weighting errorB(h) more heavily. Of course if the learner does not know in advance the quality of the domain theory or training data, it will be unclear how it should weight these two error components.
- 5. • 2.Bayes theorem perspective. • Bayes theorem computes this posterior probability based on the observed data D, together with prior knowledge in the form of P(h), P(D), and P(Dlh). • The Bayesian view is that one should simply choose the hypothesis whose posterior probability is greatest, and that Bayes theorem provides the proper method for weighting the contribution of this prior knowledge and observed data.
- 6. • Unfortunately, Bayes theorem implicitly assumes pefect knowledge about the probability distributions P(h), P(D), and P(Dlh). When these quantities are only imperfectly known, Bayes theorem alone does not prescribe how to combine them with the observed data. • (One possible approach in such cases is to assume prior probability distributions over P(h), P(D), and P(D1h) themselves, then calculate the expected value of the posterior P (h / D) . • we will simply say that the learning problem is to minimize some combined measure of the error of the hypothesis over the data and the domain theory.
- 7. 2.2 Hypothesis Space Search • We can characterize most learning methods as search algorithms by describing the hypothesis space H they search, the initial hypothesis ho at which they begin their search, the set of search operators O that define individual search steps, and the goal criterion G that specifies the search objective.
- 8. Three different methods for using prior knowledge to alter the search performed by purely inductive methods. • Use prior knowledge to derive an initial hypothesis from which to begin the search.: • In this approach the domain theory B is used to construct an initial hypothesis ho that is consistent with B. A standard inductive method is then applied, starting with the i ho. • For example, the KBANN system learns artificial neural networks in It uses prior knowledge to design the interconnections and weights for an initial network, so that this initial network is perfectly consistent with the given domain theory. This initial network hypothesis is then refined inductively using the BACKPROPAGATaIlOgNor ithm and available data. Beginning the search at a hypothesis consistent with the domain theory makes it more likely that the final output hypothesis will better fit this theory.
- 9. • Use prior knowledge to alter the objective of the hypothesis space search: • The goal criterion G is modified to require that the output hypothesis fits the domain theory as well as the training examples. • For example, the EBNN system learns neural networks ,Whereas inductive learning of neural networks performs gradient descent search to minimize the squared error of the network over the training data, EBNN performs gradient descent to optimize a different criterion. • This modified criterion includes an additional term that measures the error of the learned network relative to the domain theory.
- 10. • Use prior knowledge to alter the available search steps. • In this approach, the set of search operators 0 is altered by the domain theory. • For example, the FOCL system learns sets of Horn clauses. It is based on the inductive system FOIL, which conducts a greedy search through the space of possible Horn clauses, at each step revising its current hypothesis by adding a single new literal. • FOCL uses the domain theory to expand the set of alternatives available when revising the hypothesis, allowing the addition of multiple literals in a single search step when warranted by the domain theory. • FOCL allows single-step moves through the hypothesis space that would correspond to many steps using the original inductive algorithm. These "macro-moves" can dramatically alter the course of the search, so that the final hypothesis found consistent with the data is different from the one that would be found using only the inductive search steps.