MATLAB: How to make a decision stump using a decision tree

adaboostdecision stumpdecision treefitctreemachine learningmaxnumsplitspruneprunecriterionsplit criteriasplitcriterionStatistics and Machine Learning Toolbox

Right now, I am using fitctree and pruning the tree to level one, but beacuse the 'SplitCriterion' is 'gdi' and the 'PruneCriterion' is 'error', I can not find the best decision stump to minimize the total error. I know there is an option 'MaxNumSplits' for growing a decision tree, but this option was added on 2015 and I am using 2014 version of Matlab. So, I need to build a decision stump which minimizes the error with respects to the given weights, and I prefer to use fitctree instead of writing that from scratch. Any suggestion ?!
Here is an example that explain my problem more clearly. The correct one must look like this :
I have a full tree like the following tree and I am pruning that until (max-1) level.
Since it is not a decision stump yet, I am pruning that one level more. but it does not the same as the correct one.
And it is the code that I am using:
MinLeafSize = 1;
MinParentSize = 2;
NumVariablesToSample = 'all';
ScoreTransform = 'none';
PruneCriterion= 'error' ;
SplitCriterion = 'gdi';
Weights = Pt; % Train the weak learner by weights Pt
tree = fitctree(X,Y, 'MinLeafSize',MinLeafSize, 'MinParentSize', MinParentSize, 'NumVariablesToSample', NumVariablesToSample, ...
'PruneCriterion',PruneCriterion, 'SplitCriterion', SplitCriterion, 'Weights', Weights, 'ScoreTransform', ScoreTransform);
prune_tree = prune(tree, 'Level', max(tree.PruneList)-1); % prune tree to have decision stump
% if the prune tree still has more than one decision node (three inner nodes) use the max(tree.PruneList) to reduce it to just one node
if length(prune_tree.NodeSize) > 3
prune_tree = prune(tree, 'Level', max(tree.PruneList));
P.S. I would like to implement AdaBoostM1 by using Decision Stumps as the weak learner. I know that there is a built in function 'fitensemble' for making an ensemble classifier using AdaBoostM1 algorithm, but I would like to implement it myself.

Best Answer

  • Use