深度学习中的inductive bias解释

phy12321

于 2021-09-19 15:49:04 发布

阅读量4.3k

点赞数 5

CC 4.0 BY-SA版权

文章标签：深度学习机器学习

本文链接：https://blog.csdn.net/phy12321/article/details/120381214

本文介绍了机器学习中的一个重要概念——归纳偏好（inductive bias），即算法在学习过程中的假设偏好。归纳偏好反映了人类对世界的先验知识，并通过不同算法的具体实现展现出来。文章列举了线性回归、感知器、神经网络等常见算法的归纳偏好。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

机器学习算法在学习过程中对某种假设（hypothesis）的偏好，称为“归纳偏好”（inductive bias），或简称为“偏好”。

所谓的inductive bias，指的是人类对世界的先验知识，对应在网络中就是网络结构。

下面是一些inductive bias的例子：

Algorithm | Inductive Bias

Linear Regression | The relationship between the attributes x and the output y is linear. The goal is to minimize the sum of squared errors.

Single-Unit Perceptron | Each input votes independently toward the final classification (interactions between inputs are not possible).

Neural Networks with Backpropagation | Smooth interpolation between data points.

K-Nearest Neighbors | The classification of an instance x will be most similar to the classification of other instances that are nearby in Euclidean distance.

Support Vector Machines | Distinct classes tend to be separated by wide margins.

Naive Bayes | Each input depends only on the output class or label; the inputs are independent from each other.