http://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning

These are different approaches to improve the performance of your model (so-called meta-algorithms):

Short examples of each:

These are different approaches to improve the performance of your model (so-called meta-algorithms):

**Bagging**(stands for**B**ootstrap**Agg**regation) is the way decrease the variance of your prediction by generating additional data for training from your original dataset using combinations with repetitions to produce multisets of the same cardinality/size as your original data. By increasing the size of your training set you can't improve the model predictive force, but just decrease the variance, narrowly tuning the prediction to expected outcome.**Boosting**is a an approach to calculate the output using several different models and then average the result using a weighted average approach. By combining the advantages and pitfalls of these approaches by varying your weighting formula you can come up with a good predictive force for a wider range of input data, using different narrowly tuned models.**Stacking**is a similar to boosting: you also apply several models to you original data. The difference here is, however, that you don't have just an empirical formula for your weight function, rather you introduce a meta-level and use another model/approach to estimate the input together with outputs of every model to estimate the weights or, in other words, to determine what models perform well and what badly given these input data.

*stacking*as a sort of more advances*boosting*, however, the difficulty of finding a good approach for your meta-level makes it difficult to apply this approach in practice.Short examples of each:

*Bagging*: Ozone data.*Boosting*: is used to improve optical character recognition (OCR) accuracy.*Stacking*: is used in K-fold cross validation algorithms.

## No comments:

## Post a Comment