L21 Intro ML
Uploaded by
Surya NarayanaL21 Intro ML
Uploaded by
Surya NarayanaMachine Learning - Introduction
Note: These slides have been borrowed and/or adapted from the Machine Learning courses offered by Pedro Domingos at the University of Washington: http://www.cs.washington.edu/education/courses/cse546/10wi Raymond Mooney at the University of Texas at Austin: http://www.cs.utexas.edu/~mooney/cs391L
A Few Quotes
A breakthrough in machine learning would be worth ten Microsofts (Bill Gates, Chairman, Microsoft) Machine learning is the next Internet
(Tony Tether, Director, DARPA)
Machine learning is the hot new thing
(John Hennessy, President, Stanford)
Web rankings today are mostly a matter of machine learning (Prabhakar Raghavan, Dir. Research, Yahoo) Machine learning is going to result in a real revolution
(Greg Papadopoulos, CTO, Sun)
Machine learning is todays discontinuity
(Jerry Yang, CEO, Yahoo)
So What Is Machine Learning?
Automating automation Getting computers to program themselves Writing software is the bottleneck Let the data do the work instead!
Traditional Programming Data Program Computer Output
Machine Learning Data Output Computer Program
Magic?
No, more like gardening Seeds = Algorithms Nutrients = Data Gardener = You Plants = Programs
What is Learning?
Herbert Simon: Learning is any process by which a system improves performance from experience.
Defining the Learning Task
Improve on task, T, with respect to performance metric, P, based on experience, E.
T: Playing chess P: Percentage of games won against an arbitrary opponent E: Playing practice games against itself T: Recognizing hand-written words P: Percentage of words correctly classified E: Database of human-labeled images of handwritten words T: Driving on four-lane highways using vision sensors P: Average distance traveled before a human-judged error E: A sequence of images and steering commands recorded while observing a human driver. T: Categorize email messages as spam or legitimate. P: Percentage of email messages correctly classified. E: Database of emails, some with human-given labels
Why Study Machine Learning?
Engineering Better Computing Systems
Develop systems that are too difficult/expensive to construct manually because they require specific detailed skills or knowledge tuned to a specific task (knowledge engineering bottleneck). Develop systems that can automatically adapt and customize themselves to individual users.
Personalized news or mail filter Personalized tutoring
Discover new knowledge from large databases (data mining).
Market basket analysis (e.g. diapers and beer) Medical text mining (e.g. migraines to calcium channel blockers to magnesium)
8
Why Study Machine Learning?
Cognitive Science
Computational studies of learning may help us understand learning in humans and other biological organisms. Power law of practice
log(perf. time) log(# training trials)
Why Study Machine Learning?
The Time is Ripe Many basic effective and efficient algorithms available. Large amounts of on-line data available. Large amounts of computational resources available.
10
Sample Applications
Web search Computational biology Finance E-commerce Space exploration Robotics Information extraction Social networks Debugging [Your favorite area]
ML in a Nutshell
Tens of thousands of machine learning algorithms Hundreds new every year Every machine learning algorithm has three components:
Representation Evaluation Optimization
Representation
Decision trees Sets of rules / Logic programs Instances Graphical models (Bayes/Markov nets) Neural networks Support vector machines Model ensembles Etc.
Evaluation
Accuracy Precision and recall Squared error Likelihood Posterior probability Cost / Utility Margin Entropy K-L divergence Etc.
Optimization
Combinatorial optimization
E.g.: Greedy search
Convex optimization
E.g.: Gradient descent
Constrained optimization
E.g.: Linear programming
Types of Learning
Supervised (inductive) learning
Training data includes desired outputs
Unsupervised learning
Training data does not include desired outputs
Semi-supervised learning
Training data includes a few desired outputs
Reinforcement learning
Rewards from sequence of actions
Inductive Learning
Given examples of a function (X, F(X)) Predict function F(X) for new examples X
Discrete F(X): Classification Continuous F(X): Regression F(X) = Probability(X): Probability estimation
ML in Practice
Understanding domain, prior knowledge, and goals Data integration, selection, cleaning, pre-processing, etc. Learning models Interpreting results Consolidating and deploying discovered knowledge Loop
Inductive Learning
