top of page

Ensembles: How Models Vote for a Final Decision

Writer's picture: Tian Khean NgTian Khean Ng

What are Ensembles

Ensemble methods in predictive modeling are like gathering the opinions of multiple experts to make a final decision. Instead of relying on just one model to predict future market trends or ETF performance, ensemble techniques combine several different models to improve accuracy and reduce the risk of errors. By using a group of models, ensembles "vote" on the best possible outcome, leading to more reliable and balanced predictions. In regression models, ensemble methods typically use averaging to arrive at a final prediction. The final prediction is the average of all the individual predictions from these models.


Similarly, in boosting techniques, models are built sequentially, each improving upon the errors of the previous ones, and their predictions are also averaged to yield a final result. By combining multiple models, ensemble methods smooth out any individual errors, leading to more accurate and reliable predictions in tasks such as forecasting market trends or predicting asset prices. These models may include decision trees, linear regressions, or other statistical approaches. This method helps smooth out any weaknesses or biases in individual models, producing results that are typically more robust.


What is Boosting

Boosting is a process that focuses on improving model accuracy by paying close attention to the errors made in previous predictions. In simple terms, boosting builds models in a step-by-step manner, where each new model corrects the mistakes of the earlier ones. It works by giving more weight to the data points that were predicted incorrectly in previous rounds, so the model learns from its errors and improves over time. This approach helps create stronger, more refined predictions, and boosting is often used when an investor wants a model that is highly focused on accuracy. For example, predicting how an ETF might react to sudden market shifts. Our models use Gradient Boosting. Gradient Boosting improves model performance by optimizing a loss function, focusing on the difference (or gradient) between the actual and predicted values. Each new model is trained to reduce this error, leading to improved accuracy over time. to other methods.


What is Bagging

Bagging, short for Bootstrap Aggregating, takes a different approach. Instead of building models one after another, bagging works by training several models independently using different subsets of the data. Each model is trained separately, and then their predictions are averaged or "voted" on to reach a result. The idea is that by using multiple models trained on different samples of data, the overall prediction becomes more stable and less sensitive to fluctuations in the data. Bagging is particularly useful when you want to reduce the chance of overfitting, which happens when a model becomes too focused on the specific patterns in the training data and doesn't generalize well to new data.


Benefits of Ensembles

Both boosting and bagging help improve model performance by reducing errors and making predictions more stable. Boosted ensembles, which correct errors over multiple steps, are highly accurate but can be more complex. Bagged ensembles, by averaging predictions across different models, provide more stable and generalizable forecasts. Together, these methods make ensemble modeling a powerful tool for predicting ETF performance, giving investors more reliable insights and reducing the risk of relying on a single, possibly flawed, model. By combining strengths from different models, these techniques allow for more informed and confident investment decisions.


This can be seen in the Table below:


Using Median Average Deviation (MAD) as the metric for evaluating model performance [the smaller the better], Boosting tops the list at 0.2617. Random Trees, which is an Ensemble of Decision Trees has almost the same performance at 0.2695. Bagging ranks 3rd and a single Decision Tree has the worst performance at 0.3803.


0 views0 comments

Recent Posts

See All

Σχόλια


bottom of page