I'm implementing C4.5 and in my calculations im getting (for some examples) negative values for information gain. I read Why am I getting a negative information gain, but my issiue seeams to be diffrent. I putt my calculation to excel and i get the same results as below:
What am i doing wrong?
I tried calculate it again, and also i get negative value as is on image below: Newest calculations with data set 80 is split value, so i get 11 <=80 and 3objects > 80
Are you multiplying your result for entropy by
-1?$$ H(X) = -\sum_{i=1}^n {\mathrm{P}(x_i) \log_b \mathrm{P}(x_i)} $$
Ugh... having trouble with mathjax, go here for definition