ECS 271, Machine Learning: Homework #5
Due: 6 May 2004

Instructor: Prof. Rao Vemuri, rvemuri@ucdavis.edu

In the problems below, whenever necessary, use "epsilon" = 0.05 and "delta" = 0.05

1. (10 points) Let the hypothesis space be all conjunctive formulae over 10 Boolean variables x1, ..., x10. (2 points) What is the size of the hypothesis space H? (8 points) What is the estimate of m?

2. (10 + 10 points) How many training examples are needed to train a 2-input, 1-output Perceptron. Compare this estimated value with the actual value from your experience.You can look back at your own training experience with Perceptrons while doing an earlier homework problem. Or, you can run the program again.

3. (20 points) Suppose you want to train a neural net with W weights. Roughly, how many training examples are needed? Assume that we want 90% accuracy on test examples drawn from the same fixed distribution used to train the network. Is this estimate borne out in practice? You look back at your own experience from the homework exercise submitted last week. If necessary run the program again to gather any needed data.

4. (35 points) Consider a decision tree. Assume k attributes. Estimate m for PAC-larning of a decision tree.

5. (15 points) Given a set X, called the input space. Typically X will be a subset of R^m for some m. You are also given a concept class C, which consists of a family of subsets of X. Some examples of inputs and concept classes are given below:

(a) X = R, C = infinite open intervals, or the empty set

(b) X = R, C = R, 0, one open interval, or two disjoint infinite open intervals

(c) X = R², C = open half-spaces

Show that each of these is shattered respectively by

(d) Every 2-element set, no 3-element set

(e) Every 3-element set, no 4-element set

(f) Every non-collinear 3-element set, no 4-element set

Support your answer by a brief but complete explanation. You can use appropriate sketches to supplement your explanation.