Scribd
Upload
23 views

Ensemble Learning Helps Improve Machine Learning Results by Combining Several Models

Bagging is an ensemble learning technique that improves machine learning performance and accuracy. It works by combining multiple models trained on random subsets of a dataset to reduce variance. Bootstrapping randomly samples the dataset with replacement to create these subsets. The predictions from all the models are then aggregated to produce the final prediction. This eliminates variance and reduces overfitting compared to a single model. While computationally expensive, bagging improves stability and predictive performance.

Uploaded by

Eknath Dake
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
23 views

Ensemble Learning Helps Improve Machine Learning Results by Combining Several Models

Bagging is an ensemble learning technique that improves machine learning performance and accuracy. It works by combining multiple models trained on random subsets of a dataset to reduce variance. Bootstrapping randomly samples the dataset with replacement to create these subsets. The predictions from all the models are then aggregated to produce the final prediction. This eliminates variance and reduces overfitting compared to a single model. While computationally expensive, bagging improves stability and predictive performance.

Uploaded by

Eknath Dake
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Ensemble learning helps improve machine learning results by

combining several models. This approach allows the


production of better predictive performance compared to a
single model. Basic idea is to learn a set of classifiers
(experts) and to allow them to vote.
……………………………………………………………………………………………

Bagging
Bagging, also known as Bootstrap aggregating, is an ensemble learning
technique that helps to improve the performance and accuracy of
machine learning algorithms. It is used to deal with bias-variance trade-
offs and reduces the variance of a prediction model. Bagging avoids
overfitting of data and is used for both regression and classification
models, specifically for decision tree algorithms.

Bootstrapping is a sampling technique where samples are


derived from the whole population (set) using the replacement
procedure. The sampling with replacement method helps make the
selection procedure randomized. The base learning algorithm is run on
the samples to complete the procedure.

Aggregation in bagging is done to incorporate all possible outcomes of


the prediction and randomize the outcome. Without aggregation,
predictions will not be accurate because all outcomes are not put into
consideration. Therefore, the aggregation is based on the probability
bootstrapping procedures or on the basis of all outcomes of the
predictive models.

Bagging is advantageous since weak base learners are combined to


form a single strong learner that is more stable than single learners. It
also eliminates any variance, thereby reducing the overfitting of
models. One limitation of bagging is that it is computationally
expensive. Thus, it can lead to more bias in models when the proper
procedure of bagging is ignored.

…………………………………………………………………………………………………………..

1. Boosting

Boosting takes many forms, including gradient boosting, Adaptive


Boosting (AdaBoost), and XGBoost (Extreme Gradient
Boosting). AdaBoost uses weak learners in the form of decision trees,
which mostly include one split that is popularly known as decision
stumps. AdaBoost’s main decision stump comprises observations
carrying similar weights.

Gradient boosting adds predictors sequentially to the ensemble, where


preceding predictors correct their successors, thereby increasing the
model’s accuracy. New predictors are fit to counter the effects of errors
in the previous predictors. The gradient of descent helps the gradient
booster identify problems in learners’ predictions and counter them
accordingly.

XGBoost makes use of decision trees with boosted gradient, providing


improved speed and performance. It relies heavily on the
computational speed and the performance of the target model. Model
training should follow a sequence, thus making the implementation of
gradient boosted machines slow.

…………………………………………………………………………………………………………………………………………………………

You might also like