Section 1.1 of Pattern Recognition and Machine Learning - Polynomial curve fitting

We begin by introducing a simple regression problem, which we shall use as a running example throughout this chapter to motivate a number of key concepts. Suppose we observe a real-valued input variable x and we wish to use this observation to predict the value of a real-valued target variable t. For the present purposes, it is instructive to consider an artificial example using synthetically generated data because we then know the precise process that generated the data for comparison against any learned model. The data for this example is generated from the function sin(2πx) with random noise included in the target values, as described in detail in Appendix A.

Now suppose that we are given a training set comprising N observations of x, written x = (x1,…,N)T, together with corresponding observations of the values of t, denoted t = (t1,…,t). Figure 1.2 shows a plot of a training set comprising N= 10 data points. The input data set x in Figure 1.2 was generated by choosing values of In, for n = 1,…, N, spaced uniformly in range [0, 1], and the target data set t was obtained by first computing the corresponding values of the function sin(2x) and then adding a small level of random noise having a Gaussian distribution (the Gaussian distribution is discussed in Section 1.2.4) to each such point in order to obtain the corresponding value tn. By generating data in this way, we are capturing a property of many real data sets, namely that they possess an underlying regularity, which we wish to learn, but that individual observations are corrupted by random noise. This noise might arise from intrinsically stochastic (i.e. random) processes such as radioactive decay but more typically is due to there being sources of variability that are themselves unobserved.

Our goal is to exploit this training set in order to make predictions of the value f of the target variable for some new value of the input variable. As we shall see later, this involves implicitly trying to discover the underlying function sin(2ππ). This is intrinsically a difficult problem as we have to generalize from a finite data set. Furthermore, the observed data are corrupted with noise, and so for a given there is uncertainty as to the appropriate value for f. Probability theory, discussed in Section 1.2, provides a framework for expressing such uncertainty in a precise and quantitative manner, and decision theory, discussed in Section 1.5, allows us to exploit this probabilistic representation in order to make predictions that are optimal according to appropriate criteria.

For the moment, however, we shall proceed rather informally and consider a simple approach based on curve fitting. In particular, we shall fit the data using a polynomial function of the form

31 person(s) visited this page until now.

Back