Predictive Analytics vs. Domain Knowledge

[pwf]

In the genre competition, the top performers have a substantial lead over the pack. Several participants have clearly discovered a trick or two that has not been discovered by the other participants. I am curious whether this advantage depends on superior knowledge of predictive analytic techniques or superior domain knowledge in respect to the variables that are used to quantify musical passages.

[domcastro]

Hi
Predictive analytics
I've just topped the leaderboard and I can guarantee you that I have no domain knowledge whatsoever - I haven't even read up on it. This is an interesting dataset and even though I now have 82% on preliminary (1/3) - I believe this will get much lower for the final testset (2/3).

[cdasilva]

i'm a beginner at data mining but i've been trying hard on this, doing several experiments, trying meta learning, etc but can't overcome the 0.76 barrier :S
maybe i'm missing some pre-processing.. what can be done ? I've tried normalizing data, PCA, but nothing great comes up ..

@dom: after the competition ends can you share you knowledge with us?

good luck!

[domcastro]

Hi
Yes I will share. However, I'm not too confident that I will actually win the competition. I made something up that I had never tried before and therefore I have no idea about how much the model has overfit. I am happy that my method has potential and I'm now going to try it on more of a balanced benchmark training / test set.

An email from Kaggle came the other day saying that it is often the case that the leader of the preliminary contest is not the winner of the final contest because of overfitting. This is my concern. I tried something novel - but of course this means I've got no idea how it actually performs. So I'm more seeing it as this competition has possibly helped me discover something worth investigating rather than actually winning the competition. So don't be surprised if I end up all the way down the board!!! lol!

[cdasilva]

thinking outside the box, that's what this is all about!
thanks for your insight and good luck!