Machine Learning | Machine Learning using Probability & Statistics
96 videos • 2,137 views • by Python Kumar Learn Machine Learning using Probability & Statistics . Maximum Likelihood and Least Squares • Suppose that you are presented with a sequence of data points (X1T1), ..., (XnTn), and you are asked to find the “best fit” line passing through those points. • In order to answer this you need to know precisely how to tell whether one line is “fitter” than another • A common measure of fitness is the squarederror For a good discussion of Maximum likelihood estimators and least squares Maximum Likelihood and Least Squares y(x,w) is estimating the target t • Error/Loss/Cost/Objective function measures the squared error • Least Square Regression • Minimize L(w) wrt w. Maximum Likelihood and Least Squares • Now we approach curve fitting from a probabilistic perspective • We can express our uncertainty over the value of the target variable using a probability distribution • We assume, given the value of x, the corresponding value of t has a Gaussian distribution with a mean equal to the value y(x,w) β is the precision parameter (inverse variance) Maximum Likelihood and Least Squares Maximum Likelihood and Least Squares • We now use the training data {x, t} to determine the values of the unknown parameters w and β by maximum likelihood • Log Likelihood Maximum Likelihood and Least Squares • Log Likelihood • Maximize Log Likelihood wrt to w • Since last two terms, don’t depend on w, they can be omitted. • Also, scaling the log likelihood by a positive constant β/2 does not alter the location of the maximum with respect to w, so it can be ignored • Result: Maximize Maximum Likelihood and Least Squares • MLE • Maximize • Least Squares • Minimize • Therefore, maximizing likelihood is equivalent, so far as determining w is concerned, to minimizing the sum-of-squares error function • Significance: sum-of-squares error function arises as a consequence of maximizing likelihood under the assumption of a Gaussian noise distribution