<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>1. Supervised learning — scikit-learn 0.21.3 documentation</title>
<!-- htmltitle is before nature.css - we use this hack to load bootstrap first -->
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" href="_static/css/bootstrap.min.css" media="screen" />
<link rel="stylesheet" href="_static/css/bootstrap-responsive.css"/>
<link rel="stylesheet" href="_static/nature.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/gallery.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '0.21.3',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/copybutton.js"></script>
<script type="text/javascript" src="_static/js/extra.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS_SVG"></script>
<link rel="shortcut icon" href="_static/favicon.ico"/>
<link rel="author" title="About these documents" href="about.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="1.1. Generalized Linear Models" href="modules/linear_model.html" />
<link rel="prev" title="User Guide" href="user_guide.html" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="_static/js/bootstrap.min.js" type="text/javascript"></script>
<script>
VERSION_SUBDIR = (function(groups) {
return groups ? groups[1] : null;
})(location.href.match(/^https?:\/\/scikit-learn.org\/([^\/]+)/));
</script>
<link rel="canonical" href="https://scikit-learn.org/stable/supervised_learning.html" />
<script type="text/javascript">
$("div.buttonNext, div.buttonPrevious").hover(
function () {
$(this).css('background-color', '#FF9C34');
},
function () {
$(this).css('background-color', '#A7D6E2');
}
);
function showMenu() {
var topNav = document.getElementById("scikit-navbar");
if (topNav.className === "navbar") {
topNav.className += " responsive";
} else {
topNav.className = "navbar";
}
};
</script>
</head>
<body>
<div class="header-wrapper">
<div class="header">
<p class="logo"><a href="index.html">
<img src="_static/scikit-learn-logo-small.png" alt="Logo"/>
</a>
</p><div class="navbar" id="scikit-navbar">
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="install.html">Installation</a></li>
<li class="btn-li"><div class="btn-group">
<a href="documentation.html">Documentation</a>
<a class="btn dropdown-toggle" data-toggle="dropdown">
<span class="caret"></span>
</a>
<ul class="dropdown-menu">
<li class="link-title">Scikit-learn <script>document.write(DOCUMENTATION_OPTIONS.VERSION + (VERSION_SUBDIR ? " (" + VERSION_SUBDIR + ")" : ""));</script></li>
<li><a href="tutorial/index.html">Tutorials</a></li>
<li><a href="user_guide.html">User guide</a></li>
<li><a href="modules/classes.html">API</a></li>
<li><a href="glossary.html">Glossary</a></li>
<li><a href="faq.html">FAQ</a></li>
<li><a href="developers/index.html">Development</a></li>
<li><a href="roadmap.html">Roadmap</a></li>
<li><a href="about.html">About us</a></li>
<li class="divider"></li>
<script>if (VERSION_SUBDIR != "stable") document.write('<li><a href="http://scikit-learn.org/stable/documentation.html">Stable version</a></li>')</script>
<script>if (VERSION_SUBDIR != "dev") document.write('<li><a href="http://scikit-learn.org/dev/documentation.html">Development version</a></li>')</script>
<li><a href="http://scikit-learn.org/dev/versions.html">All available versions</a></li>
<li><a href="_downloads/scikit-learn-docs.pdf">PDF documentation</a></li>
</ul>
</div>
</li>
<li><a href="auto_examples/index.html">Examples</a></li>
</ul>
<a href="javascript:void(0);" onclick="showMenu()">
<div class="nav-icon">
<div class="hamburger-line"></div>
<div class="hamburger-line"></div>
<div class="hamburger-line"></div>
</div>
</a>
<div class="search_form">
<div class="gcse-search" id="cse" style="width: 100%;"></div>
</div>
</div> <!-- end navbar --></div>
</div>
<!-- GitHub "fork me" ribbon -->
<a href="https://github.com/scikit-learn/scikit-learn">
<img class="fork-me"
style="position: absolute; top: 0; right: 0; border: 0;"
src="_static/img/forkme.png"
alt="Fork me on GitHub" />
</a>
<div class="content-wrapper">
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<div class="rel">
<div class="rellink">
<a href="user_guide.html"
accesskey="P">Previous
<br/>
<span class="smallrellink">
User Guide
</span>
<span class="hiddenrellink">
User Guide
</span>
</a>
</div>
<div class="spacer">
</div>
<div class="rellink">
<a href="modules/linear_model.html"
accesskey="N">Next
<br/>
<span class="smallrellink">
1.1. Generali...
</span>
<span class="hiddenrellink">
1.1. Generalized Linear Models
</span>
</a>
</div>
<!-- Ad a link to the 'up' page -->
<div class="spacer">
</div>
<div class="rellink">
<a href="user_guide.html">
Up
<br/>
<span class="smallrellink">
User Guide
</span>
<span class="hiddenrellink">
User Guide
</span>
</a>
</div>
</div>
<p class="doc-version"><b>scikit-learn v0.21.3</b><br/>
<a href="http://scikit-learn.org/dev/versions.html">Other versions</a></p>
<p class="citing">Please <b><a href="about.html#citing-scikit-learn" style="font-size: 110%;">cite us </a></b>if you use the software.</p>
<ul>
<li><a class="reference internal" href="#">1. Supervised learning</a></li>
</ul>
</div>
</div>
<input type="checkbox" id="nav-trigger" class="nav-trigger" checked />
<label for="nav-trigger"></label>
<div class="content">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<style type="text/css">
div.bodywrapper blockquote {
margin: 0 ;
}
div.toctree-wrapper ul {
margin-top: 0 ;
margin-bottom: 0 ;
padding-left: 10px ;
}
li.toctree-l1 {
padding: 0 0 0.5em 0 ;
list-style-type: none;
font-size: 150% ;
font-weight: bold;
}
li.toctree-l1 ul {
padding-left: 40px ;
}
li.toctree-l2 {
font-size: 70% ;
list-style-type: square;
font-weight: normal;
}
li.toctree-l3 {
font-size: 85% ;
list-style-type: circle;
font-weight: normal;
}
</style><div class="section" id="supervised-learning">
<span id="id1"></span><h1>1. Supervised learning<a class="headerlink" href="#supervised-learning" title="Permalink to this headline">¶</a></h1>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="modules/linear_model.html">1.1. Generalized Linear Models</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#ordinary-least-squares">1.1.1. Ordinary Least Squares</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#ordinary-least-squares-complexity">1.1.1.1. Ordinary Least Squares Complexity</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#ridge-regression">1.1.2. Ridge Regression</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#ridge-complexity">1.1.2.1. Ridge Complexity</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#setting-the-regularization-parameter-generalized-cross-validation">1.1.2.2. Setting the regularization parameter: generalized Cross-Validation</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#lasso">1.1.3. Lasso</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#setting-regularization-parameter">1.1.3.1. Setting regularization parameter</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/linear_model.html#using-cross-validation">1.1.3.1.1. Using cross-validation</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/linear_model.html#information-criteria-based-model-selection">1.1.3.1.2. Information-criteria based model selection</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/linear_model.html#comparison-with-the-regularization-parameter-of-svm">1.1.3.1.3. Comparison with the regularization parameter of SVM</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#multi-task-lasso">1.1.4. Multi-task Lasso</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#elastic-net">1.1.5. Elastic-Net</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#multi-task-elastic-net">1.1.6. Multi-task Elastic-Net</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#least-angle-regression">1.1.7. Least Angle Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#lars-lasso">1.1.8. LARS Lasso</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#mathematical-formulation">1.1.8.1. Mathematical formulation</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#orthogonal-matching-pursuit-omp">1.1.9. Orthogonal Matching Pursuit (OMP)</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#bayesian-regression">1.1.10. Bayesian Regression</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#bayesian-ridge-regression">1.1.10.1. Bayesian Ridge Regression</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#automatic-relevance-determination-ard">1.1.10.2. Automatic Relevance Determination - ARD</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#logistic-regression">1.1.11. Logistic regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#stochastic-gradient-descent-sgd">1.1.12. Stochastic Gradient Descent - SGD</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#perceptron">1.1.13. Perceptron</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#passive-aggressive-algorithms">1.1.14. Passive Aggressive Algorithms</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#robustness-regression-outliers-and-modeling-errors">1.1.15. Robustness regression: outliers and modeling errors</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#different-scenario-and-useful-concepts">1.1.15.1. Different scenario and useful concepts</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#ransac-random-sample-consensus">1.1.15.2. RANSAC: RANdom SAmple Consensus</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/linear_model.html#details-of-the-algorithm">1.1.15.2.1. Details of the algorithm</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#theil-sen-estimator-generalized-median-based-estimator">1.1.15.3. Theil-Sen estimator: generalized-median-based estimator</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/linear_model.html#theoretical-considerations">1.1.15.3.1. Theoretical considerations</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#huber-regression">1.1.15.4. Huber Regression</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/linear_model.html#notes">1.1.15.5. Notes</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/linear_model.html#polynomial-regression-extending-linear-models-with-basis-functions">1.1.16. Polynomial regression: extending linear models with basis functions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/lda_qda.html">1.2. Linear and Quadratic Discriminant Analysis</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/lda_qda.html#dimensionality-reduction-using-linear-discriminant-analysis">1.2.1. Dimensionality reduction using Linear Discriminant Analysis</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/lda_qda.html#mathematical-formulation-of-the-lda-and-qda-classifiers">1.2.2. Mathematical formulation of the LDA and QDA classifiers</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/lda_qda.html#mathematical-formulation-of-lda-dimensionality-reduction">1.2.3. Mathematical formulation of LDA dimensionality reduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/lda_qda.html#shrinkage">1.2.4. Shrinkage</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/lda_qda.html#estimation-algorithms">1.2.5. Estimation algorithms</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/kernel_ridge.html">1.3. Kernel ridge regression</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules/svm.html">1.4. Support Vector Machines</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#classification">1.4.1. Classification</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#multi-class-classification">1.4.1.1. Multi-class classification</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#scores-and-probabilities">1.4.1.2. Scores and probabilities</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#unbalanced-problems">1.4.1.3. Unbalanced problems</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#regression">1.4.2. Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#density-estimation-novelty-detection">1.4.3. Density estimation, novelty detection</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#complexity">1.4.4. Complexity</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#tips-on-practical-use">1.4.5. Tips on Practical Use</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#kernel-functions">1.4.6. Kernel functions</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#custom-kernels">1.4.6.1. Custom Kernels</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/svm.html#using-python-functions-as-kernels">1.4.6.1.1. Using Python functions as kernels</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/svm.html#using-the-gram-matrix">1.4.6.1.2. Using the Gram matrix</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/svm.html#parameters-of-the-rbf-kernel">1.4.6.1.3. Parameters of the RBF Kernel</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#mathematical-formulation">1.4.7. Mathematical formulation</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#svc">1.4.7.1. SVC</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#nusvc">1.4.7.2. NuSVC</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/svm.html#svr">1.4.7.3. SVR</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/svm.html#implementation-details">1.4.8. Implementation details</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/sgd.html">1.5. Stochastic Gradient Descent</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#classification">1.5.1. Classification</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#regression">1.5.2. Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#stochastic-gradient-descent-for-sparse-data">1.5.3. Stochastic Gradient Descent for sparse data</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#complexity">1.5.4. Complexity</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#stopping-criterion">1.5.5. Stopping criterion</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#tips-on-practical-use">1.5.6. Tips on Practical Use</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#mathematical-formulation">1.5.7. Mathematical formulation</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/sgd.html#id1">1.5.7.1. SGD</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/sgd.html#implementation-details">1.5.8. Implementation details</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/neighbors.html">1.6. Nearest Neighbors</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#unsupervised-nearest-neighbors">1.6.1. Unsupervised Nearest Neighbors</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#finding-the-nearest-neighbors">1.6.1.1. Finding the Nearest Neighbors</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#kdtree-and-balltree-classes">1.6.1.2. KDTree and BallTree Classes</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#nearest-neighbors-classification">1.6.2. Nearest Neighbors Classification</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#nearest-neighbors-regression">1.6.3. Nearest Neighbors Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#nearest-neighbor-algorithms">1.6.4. Nearest Neighbor Algorithms</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#brute-force">1.6.4.1. Brute Force</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#k-d-tree">1.6.4.2. K-D Tree</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#ball-tree">1.6.4.3. Ball Tree</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#choice-of-nearest-neighbors-algorithm">1.6.4.4. Choice of Nearest Neighbors Algorithm</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#effect-of-leaf-size">1.6.4.5. Effect of <code class="docutils literal"><span class="pre">leaf_size</span></code></a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#nearest-centroid-classifier">1.6.5. Nearest Centroid Classifier</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#nearest-shrunken-centroid">1.6.5.1. Nearest Shrunken Centroid</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/neighbors.html#neighborhood-components-analysis">1.6.6. Neighborhood Components Analysis</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#id4">1.6.6.1. Classification</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#dimensionality-reduction">1.6.6.2. Dimensionality reduction</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#mathematical-formulation">1.6.6.3. Mathematical formulation</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/neighbors.html#mahalanobis-distance">1.6.6.3.1. Mahalanobis distance</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#implementation">1.6.6.4. Implementation</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/neighbors.html#complexity">1.6.6.5. Complexity</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/neighbors.html#training">1.6.6.5.1. Training</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/neighbors.html#transform">1.6.6.5.2. Transform</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/gaussian_process.html">1.7. Gaussian Processes</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/gaussian_process.html#gaussian-process-regression-gpr">1.7.1. Gaussian Process Regression (GPR)</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/gaussian_process.html#gpr-examples">1.7.2. GPR examples</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#gpr-with-noise-level-estimation">1.7.2.1. GPR with noise-level estimation</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#comparison-of-gpr-and-kernel-ridge-regression">1.7.2.2. Comparison of GPR and Kernel Ridge Regression</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#gpr-on-mauna-loa-co2-data">1.7.2.3. GPR on Mauna Loa CO2 data</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/gaussian_process.html#gaussian-process-classification-gpc">1.7.3. Gaussian Process Classification (GPC)</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/gaussian_process.html#gpc-examples">1.7.4. GPC examples</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#probabilistic-predictions-with-gpc">1.7.4.1. Probabilistic predictions with GPC</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#illustration-of-gpc-on-the-xor-dataset">1.7.4.2. Illustration of GPC on the XOR dataset</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#gaussian-process-classification-gpc-on-iris-dataset">1.7.4.3. Gaussian process classification (GPC) on iris dataset</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/gaussian_process.html#kernels-for-gaussian-processes">1.7.5. Kernels for Gaussian Processes</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#gaussian-process-kernel-api">1.7.5.1. Gaussian Process Kernel API</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#basic-kernels">1.7.5.2. Basic kernels</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#kernel-operators">1.7.5.3. Kernel operators</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#radial-basis-function-rbf-kernel">1.7.5.4. Radial-basis function (RBF) kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#matern-kernel">1.7.5.5. Matérn kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#rational-quadratic-kernel">1.7.5.6. Rational quadratic kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#exp-sine-squared-kernel">1.7.5.7. Exp-Sine-Squared kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#dot-product-kernel">1.7.5.8. Dot-Product kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/gaussian_process.html#references">1.7.5.9. References</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/cross_decomposition.html">1.8. Cross decomposition</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules/naive_bayes.html">1.9. Naive Bayes</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/naive_bayes.html#gaussian-naive-bayes">1.9.1. Gaussian Naive Bayes</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/naive_bayes.html#multinomial-naive-bayes">1.9.2. Multinomial Naive Bayes</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/naive_bayes.html#complement-naive-bayes">1.9.3. Complement Naive Bayes</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/naive_bayes.html#bernoulli-naive-bayes">1.9.4. Bernoulli Naive Bayes</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/naive_bayes.html#out-of-core-naive-bayes-model-fitting">1.9.5. Out-of-core naive Bayes model fitting</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/tree.html">1.10. Decision Trees</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#classification">1.10.1. Classification</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#regression">1.10.2. Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#multi-output-problems">1.10.3. Multi-output problems</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#complexity">1.10.4. Complexity</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#tips-on-practical-use">1.10.5. Tips on practical use</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#tree-algorithms-id3-c4-5-c5-0-and-cart">1.10.6. Tree algorithms: ID3, C4.5, C5.0 and CART</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/tree.html#mathematical-formulation">1.10.7. Mathematical formulation</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/tree.html#classification-criteria">1.10.7.1. Classification criteria</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/tree.html#regression-criteria">1.10.7.2. Regression criteria</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/ensemble.html">1.11. Ensemble methods</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#bagging-meta-estimator">1.11.1. Bagging meta-estimator</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#forests-of-randomized-trees">1.11.2. Forests of randomized trees</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#random-forests">1.11.2.1. Random Forests</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#extremely-randomized-trees">1.11.2.2. Extremely Randomized Trees</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#parameters">1.11.2.3. Parameters</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#parallelization">1.11.2.4. Parallelization</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#feature-importance-evaluation">1.11.2.5. Feature importance evaluation</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#totally-random-trees-embedding">1.11.2.6. Totally Random Trees Embedding</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#adaboost">1.11.3. AdaBoost</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#usage">1.11.3.1. Usage</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#gradient-tree-boosting">1.11.4. Gradient Tree Boosting</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#classification">1.11.4.1. Classification</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#regression">1.11.4.2. Regression</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#fitting-additional-weak-learners">1.11.4.3. Fitting additional weak-learners</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#controlling-the-tree-size">1.11.4.4. Controlling the tree size</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#mathematical-formulation">1.11.4.5. Mathematical formulation</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#loss-functions">1.11.4.5.1. Loss Functions</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#regularization">1.11.4.6. Regularization</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#shrinkage">1.11.4.6.1. Shrinkage</a></li>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#subsampling">1.11.4.6.2. Subsampling</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#interpretation">1.11.4.7. Interpretation</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#feature-importance">1.11.4.7.1. Feature importance</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#voting-classifier">1.11.5. Voting Classifier</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#majority-class-labels-majority-hard-voting">1.11.5.1. Majority Class Labels (Majority/Hard Voting)</a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#id21">1.11.5.1.1. Usage</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#weighted-average-probabilities-soft-voting">1.11.5.2. Weighted Average Probabilities (Soft Voting)</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/ensemble.html#using-the-votingclassifier-with-gridsearchcv">1.11.5.3. Using the <code class="docutils literal"><span class="pre">VotingClassifier</span></code> with <code class="docutils literal"><span class="pre">GridSearchCV</span></code></a><ul>
<li class="toctree-l4"><a class="reference internal" href="modules/ensemble.html#id22">1.11.5.3.1. Usage</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/ensemble.html#voting-regressor">1.11.6. Voting Regressor</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/multiclass.html">1.12. Multiclass and multilabel algorithms</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#multilabel-classification-format">1.12.1. Multilabel classification format</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#one-vs-the-rest">1.12.2. One-Vs-The-Rest</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/multiclass.html#multiclass-learning">1.12.2.1. Multiclass learning</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/multiclass.html#multilabel-learning">1.12.2.2. Multilabel learning</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#one-vs-one">1.12.3. One-Vs-One</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/multiclass.html#id1">1.12.3.1. Multiclass learning</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#error-correcting-output-codes">1.12.4. Error-Correcting Output-Codes</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/multiclass.html#id3">1.12.4.1. Multiclass learning</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#multioutput-regression">1.12.5. Multioutput regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#multioutput-classification">1.12.6. Multioutput classification</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#classifier-chain">1.12.7. Classifier Chain</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/multiclass.html#regressor-chain">1.12.8. Regressor Chain</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/feature_selection.html">1.13. Feature selection</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/feature_selection.html#removing-features-with-low-variance">1.13.1. Removing features with low variance</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/feature_selection.html#univariate-feature-selection">1.13.2. Univariate feature selection</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/feature_selection.html#recursive-feature-elimination">1.13.3. Recursive feature elimination</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/feature_selection.html#feature-selection-using-selectfrommodel">1.13.4. Feature selection using SelectFromModel</a><ul>
<li class="toctree-l3"><a class="reference internal" href="modules/feature_selection.html#l1-based-feature-selection">1.13.4.1. L1-based feature selection</a></li>
<li class="toctree-l3"><a class="reference internal" href="modules/feature_selection.html#tree-based-feature-selection">1.13.4.2. Tree-based feature selection</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="modules/feature_selection.html#feature-selection-as-part-of-a-pipeline">1.13.5. Feature selection as part of a pipeline</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/label_propagation.html">1.14. Semi-Supervised</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/label_propagation.html#label-propagation">1.14.1. Label Propagation</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="modules/isotonic.html">1.15. Isotonic regression</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules/calibration.html">1.16. Probability calibration</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules/neural_networks_supervised.html">1.17. Neural network models (supervised)</a><ul>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#multi-layer-perceptron">1.17.1. Multi-layer Perceptron</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#classification">1.17.2. Classification</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#regression">1.17.3. Regression</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#regularization">1.17.4. Regularization</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#algorithms">1.17.5. Algorithms</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#complexity">1.17.6. Complexity</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#mathematical-formulation">1.17.7. Mathematical formulation</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#tips-on-practical-use">1.17.8. Tips on Practical Use</a></li>
<li class="toctree-l2"><a class="reference internal" href="modules/neural_networks_supervised.html#more-control-with-warm-start">1.17.9. More control with warm_start</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
<div class="footer">
© 2007 - 2019, scikit-learn developers (BSD License).
<a href="_sources/supervised_learning.rst.txt" rel="nofollow">Show this page source</a>
</div>
<div class="rel">
<div class="buttonPrevious">
<a href="user_guide.html">Previous
</a>
</div>
<div class="buttonNext">
<a href="modules/linear_model.html">Next
</a>
</div>
</div>
<script>
window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
ga('create', 'UA-22606712-2', 'auto');
ga('set', 'anonymizeIp', true);
ga('send', 'pageview');
</script>
<script async src='https://www.google-analytics.com/analytics.js'></script>
<script>
(function() {
var cx = '016639176250731907682:tjtqbvtvij0';
var gcse = document.createElement('script'); gcse.type = 'text/javascript'; gcse.async = true;
gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(gcse, s);
})();
</script>
<script src="https://scikit-learn.org/versionwarning.js"></script>
<script src="https://scikit-learn.org/versionwarning.js"></script>
</body>
</html>