Machine Learning at Geeky Base
11 likes2,041 views
This document provides an overview of machine learning including definitions, types of learning problems, example algorithms, and best practices. It defines machine learning as a field that allows computers to learn without being explicitly programmed. The main types of learning are supervised, unsupervised, and reinforcement learning. Example algorithms covered are linear regression, k-nearest neighbors, naive Bayes, decision trees, support vector machines, and k-means clustering. The document stresses the importance of feature engineering, validation, regularization, and using more data to avoid overfitting.
More Related Content
Similar to Machine Learning at Geeky Base (20)
Machine Learning at Geeky Base
- 2. Kan Ouivirach Research & Development Engineer www.kanouivirach.com
- 3. Outline • What is Machine Learning? • Main Types of Learning • Model Validation, Selection, and Evaluation • Applied Machine Learning Process • Cautions
- 4. What is Machine Learning? http://www.bigdata-madesimple.com/
- 5. –Arthur Samuel (1959) “Field of study that gives computers the ability to learn without being explicitly programmed.”
- 6. –Tom Mitchell (1988) “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
- 7. Statistics vs. Data Mining vs. Machine Learning vs. …?
- 8. Programming vs. Machine Learning?
- 9. Programming? “Given a specification of a function f, implement f that meets the specification.” Machine Learning? “Given example (x, y) pairs, induce f such that y = f(x) for given pairs and generalizes well for unseen x” –Peter Norvig (2014)
- 10. Why is Machine Learning so hard? http://veronicaforand.com/
- 13. Applications of Machine Learning • Search Engines • Medical Diagnosis • Object Recognition • Stock Market Analysis • Credit Card Fraud Detection • Speech Recognition • etc.
- 14. Recommendation System on Amazon.com
- 15. Advertisement System on Facebook.com
- 16. Speech Recognition from Microsoft
- 18. Main Types of Learning • Supervised Learning • Unsupervised Learning • Reinforcement Learning
- 19. Supervised Learning y = f(x) Given x, y pairs, find a function f that will map new x to a proper y.
- 20. Supervised Learning Problems • Regression • Classification
- 21. Regression
- 22. Linear Regression y = wx + b
- 25. Perceptron Processor Input 0 Input 1 Output One or more inputs, a processor, and a single output
- 28. Probability Theory https://seisanshi.wordpress.com/tag/probability/
- 29. A2A1 A3 An Ck . . . P(Ck | A1, …, An) = P(Ck) * P(A1, …, An | Ck) / P(A1, …, An) P(Ck | A1, …, An) P(Ck) * Prod P(Ai | C) with independence assumption, we then have Naive Bayes
- 30. Naive Bayes No. Content Spam? 1 Party Yes 2 Sale Discount Yes 3 Party Sale Discount Yes 4 Python Party No 5 Python Programming No
- 31. Naive Bayes P(Spam | Party, Programming) = P(Spam) * P(Party | Spam) * P(Programming | Spam) P(NotSpam | Party, Programming) = P(NotSpam) * P(Party | NotSpam) * P(Programming | NotSpam) We want to find if “Party Programming” is spam or not? We need to know P(Spam), P(NotSpam) P(Party | Spam), P(Party | NotSpam) P(Programming | Spam), P(Programming | NotSpam)
- 32. Naive Bayes No. Content Spam? 1 Party Yes 2 Sale Discount Yes 3 Party Sale Discount Yes 4 Python Party No 5 Python Programming No P(Spam) = ? P(NotSpam) = ? P(Party | Spam) = ? P(Party | NotSpam) = ? P(Programming | Spam) = ? P(Programming | NotSpam) = ?
- 33. Naive Bayes No. Content Spam? 1 Party Yes 2 Sale Discount Yes 3 Party Sale Discount Yes 4 Python Party No 5 Python Programming No P(Spam) = 3/5 P(NotSpam) = 2/5 P(Party | Spam) = 2/3 P(Party | NotSpam) = 1/2 P(Programming | Spam) = 0 P(Programming | NotSpam) = 1/2
- 34. Naive Bayes P(Spam | Party, Programming) = 3/5 * 2/3 * 0 = 0 P(NotSpam | Party, Programming) = 2/5 * 1/2 * 1/2 = 0.1 P(NotSpam | Party, Programming) > P(Spam | Party, Programming) “Party Programming” is NOT a spam.
- 35. Decision Tree Outlook Humidity Wind Sunny Overcast Rain Yes High Normal Strong Weak No Yes No Yes Day Outlook Temp Humidity WInd Play D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Mild High Strong Yes D4 Rain Cool Normal Strong No Play tennis?
- 37. Support Vector Machines x y Current Coordinate System x z New Coordinate System “Kernel Trick”
- 38. Support Vector Machines http://www.mblondel.org/journal/2010/09/19/support-vector-machines-in-python/ 3 support vectors
- 39. Unsupervised Learning f(x) Given x, find a function f that gives a compact description of x.
- 40. Unsupervised Learning • k-Means Clustering • Hierarchical Clustering • Gaussian Mixture Models (GMMs)
- 42. Anomaly Detection http://modernfarmer.com/2013/11/farm-pop-idioms/
- 43. http://boxesandarrows.com/designing-screens-using-cores-and-paths/
- 44. Reinforcement Learning y = f(x) Given x and z, find a function f that generates y. z
- 45. Flappy Bird Hack using Reinforcement Learning http://sarvagyavaish.github.io/FlappyBirdRL/
- 47. Model Validation
- 48. I’ve got a perfect classifiers! https://500px.com/photo/65907417/like-a-frog-trapped-inside-a-coconut-shell-by-ellena-susanti
- 49. http://blog.csdn.net/love_tea_cat/article/details/25972921 Overfitting (High Variance) Normal fit Overfitting
- 50. http://blog.csdn.net/love_tea_cat/article/details/25972921 Underfitting (High Bias) Normal fit Underfitting
- 51. How to Avoid Overfitting and Underfitting • Using more data does NOT always help. • Recommend to • find a good number of features; • perform cross validation; • use regularization when overfitting is found.
- 52. Model Selection
- 53. Model Selection • Use cross validation to find the best parameters for the model.
- 54. Model Evaluation
- 55. Metrics • Accuracy • True Positive, False Positive, True Negative, False Negative • Precision and Recall • F1 Score • etc.
- 57. Applied Machine Learning Process http://machinelearningmastery.com/process-for-working-through-machine-learning-problems/
- 61. If two models fit the data equally well, choose the simpler one.
- 64. http://newventurist.com/ • Curse of dimensionality • Correlation does NOT imply causation. • Learn many models, not just ONE. • More data beats a cleaver algorithm. • Data alone are not enough. A Few Useful Things You Need to Know about Machine Learning, Pedro Domigos (2012) Some Cautions
- 65. — Feature engineering is the key. —
- 66. Example of Feature Engineering Width (m) Length (m) Cost (baht) 100 100 1,200,000 500 50 1,300,000 100 80 1,000,000 400 100 1,500,000 Are the data good to model the area’s cost? Size (m x m) Cost (baht) 100,000 1,200,000 25,000 1,300,000 8,000 1,000,000 400,00 1,500,000 Engineer features. They look better here.
- 67. Deep Learning at Microsoft’s Speech Group
- 68. Let’s get our hands dirty! https://github.com/zkan/intro-to-machine-learning