Pushing the docs to dev/ for branch: main, commit ce8f23df3de9659efe146308b0639f5bc681b244

Author: sklearn-ci

dev/_downloads/02192f99342d6d1323161978b6c80bfc/plot_ols_ridge.zip

@@ -40,7 +40,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Define the classifiers\n\nHere we define two different classifiers. The goal is to visually compare their\nstatistical performance across thresholds using the ROC and DET curves. There\nis no particular reason why these classifiers are chosen other classifiers\navailable in scikit-learn.\n\n"
+        "## Define the classifiers\n\nHere we define two different classifiers. The goal is to visually compare their\nstatistical performance across thresholds using the ROC and DET curves.\n\n"
       ]
     },
     {
@@ -51,14 +51,14 @@
       },
       "outputs": [],
       "source": [
-        "from sklearn.ensemble import RandomForestClassifier\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.svm import LinearSVC\n\nclassifiers = {\n    \"Linear SVM\": make_pipeline(StandardScaler(), LinearSVC(C=0.025)),\n    \"Random Forest\": RandomForestClassifier(\n        max_depth=5, n_estimators=10, max_features=1\n    ),\n}"
+        "from sklearn.dummy import DummyClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.svm import LinearSVC\n\nclassifiers = {\n    \"Linear SVM\": make_pipeline(StandardScaler(), LinearSVC(C=0.025)),\n    \"Random Forest\": RandomForestClassifier(\n        max_depth=5, n_estimators=10, max_features=1, random_state=0\n    ),\n    \"Non-informative baseline\": DummyClassifier(),\n}"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Plot ROC and DET curves\n\nDET curves are commonly plotted in normal deviate scale. To achieve this the\nDET display transforms the error rates as returned by the\n:func:`~sklearn.metrics.det_curve` and the axis scale using\n`scipy.stats.norm`.\n\n"
+        "## Compare ROC and DET curves\n\nDET curves are commonly plotted in normal deviate scale. To achieve this the\nDET display transforms the error rates as returned by the\n:func:`~sklearn.metrics.det_curve` and the axis scale using\n`scipy.stats.norm`.\n\n"
       ]
     },
     {
@@ -69,14 +69,14 @@
       },
       "outputs": [],
       "source": [
-        "import matplotlib.pyplot as plt\n\nfrom sklearn.metrics import DetCurveDisplay, RocCurveDisplay\n\nfig, [ax_roc, ax_det] = plt.subplots(1, 2, figsize=(11, 5))\n\nfor name, clf in classifiers.items():\n    clf.fit(X_train, y_train)\n\n    RocCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_roc, name=name)\n    DetCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_det, name=name)\n\nax_roc.set_title(\"Receiver Operating Characteristic (ROC) curves\")\nax_det.set_title(\"Detection Error Tradeoff (DET) curves\")\n\nax_roc.grid(linestyle=\"--\")\nax_det.grid(linestyle=\"--\")\n\nplt.legend()\nplt.show()"
+        "import matplotlib.pyplot as plt\n\nfrom sklearn.dummy import DummyClassifier\nfrom sklearn.metrics import DetCurveDisplay, RocCurveDisplay\n\nfig, [ax_roc, ax_det] = plt.subplots(1, 2, figsize=(11, 5))\n\nax_roc.set_title(\"Receiver Operating Characteristic (ROC) curves\")\nax_det.set_title(\"Detection Error Tradeoff (DET) curves\")\n\nax_roc.grid(linestyle=\"--\")\nax_det.grid(linestyle=\"--\")\n\nfor name, clf in classifiers.items():\n    (color, linestyle) = (\n        (\"black\", \"--\") if name == \"Non-informative baseline\" else (None, None)\n    )\n    clf.fit(X_train, y_train)\n    RocCurveDisplay.from_estimator(\n        clf, X_test, y_test, ax=ax_roc, name=name, color=color, linestyle=linestyle\n    )\n    DetCurveDisplay.from_estimator(\n        clf, X_test, y_test, ax=ax_det, name=name, color=color, linestyle=linestyle\n    )\n\nplt.legend()\nplt.show()"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Notice that it is easier to visually assess the overall performance of\ndifferent classification algorithms using DET curves than using ROC curves. As\nROC curves are plot in a linear scale, different classifiers usually appear\nsimilar for a large part of the plot and differ the most in the top left\ncorner of the graph. On the other hand, because DET curves represent straight\nlines in normal deviate scale, they tend to be distinguishable as a whole and\nthe area of interest spans a large part of the plot.\n\nDET curves give direct feedback of the detection error tradeoff to aid in\noperating point analysis. The user can then decide the FNR they are willing to\naccept at the expense of the FPR (or vice-versa).\n\n"
+        "Notice that it is easier to visually assess the overall performance of\ndifferent classification algorithms using DET curves than using ROC curves. As\nROC curves are plot in a linear scale, different classifiers usually appear\nsimilar for a large part of the plot and differ the most in the top left\ncorner of the graph. On the other hand, because DET curves represent straight\nlines in normal deviate scale, they tend to be distinguishable as a whole and\nthe area of interest spans a large part of the plot.\n\nDET curves give direct feedback of the detection error tradeoff to aid in\noperating point analysis. The user can then decide the FNR they are willing to\naccept at the expense of the FPR (or vice-versa).\n\n## Non-informative classifier baseline for the ROC and DET curves\n\nThe diagonal black-dotted lines in the plots above correspond to a\n:class:`~sklearn.dummy.DummyClassifier` using the default \"prior\" strategy, to\nserve as baseline for comparison with other classifiers. This classifier makes\nconstant predictions, independent of the input features in `X`, making it a\nnon-informative classifier.\n\nTo further understand the non-informative baseline of the ROC and DET curves,\nwe recall the following mathematical definitions:\n\n$\\text{FPR} = \\frac{\\text{FP}}{\\text{FP} + \\text{TN}}$\n\n$\\text{FNR} = \\frac{\\text{FN}}{\\text{TP} + \\text{FN}}$\n\n$\\text{TPR} = \\frac{\\text{TP}}{\\text{TP} + \\text{FN}}$\n\nA classifier that always predict the positive class would have no true\nnegatives nor false negatives, giving $\\text{FPR} = \\text{TPR} = 1$ and\n$\\text{FNR} = 0$, i.e.:\n\n- a single point in the upper right corner of the ROC plane,\n- a single point in the lower right corner of the DET plane.\n\nSimilarly, a classifier that always predict the negative class would have no\ntrue positives nor false positives, thus $\\text{FPR} = \\text{TPR} = 0$\nand $\\text{FNR} = 1$, i.e.:\n\n- a single point in the lower left corner of the ROC plane,\n- a single point in the upper left corner of the DET plane.\n\n"
       ]
     }
   ],

dev/_downloads/02fe21a1f5d14bd1ebbe34aa905d9bf6/plot_multilabel.zip

@@ -60,24 +60,24 @@
 # ----------------------
 #
 # Here we define two different classifiers. The goal is to visually compare their
-# statistical performance across thresholds using the ROC and DET curves. There
-# is no particular reason why these classifiers are chosen other classifiers
-# available in scikit-learn.
+# statistical performance across thresholds using the ROC and DET curves.
 
+from sklearn.dummy import DummyClassifier
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.pipeline import make_pipeline
 from sklearn.svm import LinearSVC
 
 classifiers = {
     "Linear SVM": make_pipeline(StandardScaler(), LinearSVC(C=0.025)),
     "Random Forest": RandomForestClassifier(
-        max_depth=5, n_estimators=10, max_features=1
+        max_depth=5, n_estimators=10, max_features=1, random_state=0
     ),
+    "Non-informative baseline": DummyClassifier(),
 }
 
 # %%
-# Plot ROC and DET curves
-# -----------------------
+# Compare ROC and DET curves
+# --------------------------
 #
 # DET curves are commonly plotted in normal deviate scale. To achieve this the
 # DET display transforms the error rates as returned by the
@@ -86,22 +86,29 @@
 
 import matplotlib.pyplot as plt
 
+from sklearn.dummy import DummyClassifier
 from sklearn.metrics import DetCurveDisplay, RocCurveDisplay
 
 fig, [ax_roc, ax_det] = plt.subplots(1, 2, figsize=(11, 5))
 
-for name, clf in classifiers.items():
-    clf.fit(X_train, y_train)
-
-    RocCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_roc, name=name)
-    DetCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_det, name=name)
-
 ax_roc.set_title("Receiver Operating Characteristic (ROC) curves")
 ax_det.set_title("Detection Error Tradeoff (DET) curves")
 
 ax_roc.grid(linestyle="--")
 ax_det.grid(linestyle="--")
 
+for name, clf in classifiers.items():
+    (color, linestyle) = (
+        ("black", "--") if name == "Non-informative baseline" else (None, None)
+    )
+    clf.fit(X_train, y_train)
+    RocCurveDisplay.from_estimator(
+        clf, X_test, y_test, ax=ax_roc, name=name, color=color, linestyle=linestyle
+    )
+    DetCurveDisplay.from_estimator(
+        clf, X_test, y_test, ax=ax_det, name=name, color=color, linestyle=linestyle
+    )
+
 plt.legend()
 plt.show()
 
@@ -117,3 +124,35 @@
 # DET curves give direct feedback of the detection error tradeoff to aid in
 # operating point analysis. The user can then decide the FNR they are willing to
 # accept at the expense of the FPR (or vice-versa).
+#
+# Non-informative classifier baseline for the ROC and DET curves
+# --------------------------------------------------------------
+#
+# The diagonal black-dotted lines in the plots above correspond to a
+# :class:`~sklearn.dummy.DummyClassifier` using the default "prior" strategy, to
+# serve as baseline for comparison with other classifiers. This classifier makes
+# constant predictions, independent of the input features in `X`, making it a
+# non-informative classifier.
+#
+# To further understand the non-informative baseline of the ROC and DET curves,
+# we recall the following mathematical definitions:
+#
+# :math:`\text{FPR} = \frac{\text{FP}}{\text{FP} + \text{TN}}`
+#
+# :math:`\text{FNR} = \frac{\text{FN}}{\text{TP} + \text{FN}}`
+#
+# :math:`\text{TPR} = \frac{\text{TP}}{\text{TP} + \text{FN}}`
+#
+# A classifier that always predict the positive class would have no true
+# negatives nor false negatives, giving :math:`\text{FPR} = \text{TPR} = 1` and
+# :math:`\text{FNR} = 0`, i.e.:
+#
+# - a single point in the upper right corner of the ROC plane,
+# - a single point in the lower right corner of the DET plane.
+#
+# Similarly, a classifier that always predict the negative class would have no
+# true positives nor false positives, thus :math:`\text{FPR} = \text{TPR} = 0`
+# and :math:`\text{FNR} = 1`, i.e.:
+#
+# - a single point in the lower left corner of the ROC plane,
+# - a single point in the upper left corner of the DET plane.