Page 1 of 1

Will the full test set be posted?

PostPosted: Tue Mar 22, 2011 11:21 pm
by JRM
As of 22 March, I was unable to find it.

It would be instructive to attempt to replicate the performance of other data miners, but model evaluation via the online premliminy testset is no longer available.

Thanks!
John McDowell

Re: Will the full test set be posted?

PostPosted: Wed Mar 23, 2011 6:30 pm
by Marcin
Hi John,
All contest files are available now in this folder. In particular, you will find there ground truth decisions for Genres and Instruments test data. Good luck!

Marcin

Re: Will the full test set be posted?

PostPosted: Wed Mar 23, 2011 6:45 pm
by JRM
Ah, perfect-- thanks for the pointer, and for helping run such a great contest. I know many of us learned a great deal from participating.

Re: Will the full test set be posted?

PostPosted: Thu Mar 24, 2011 3:23 am
by wahoo
Is it possible to get the artist -> song -> segment mapping for the genre task?

Thank you,
Brian

Re: Will the full test set be posted?

PostPosted: Thu Mar 24, 2011 7:26 pm
by Marcin
Brian, we'll check what can be done.
-m

Re: Will the full test set be posted?

PostPosted: Fri Mar 25, 2011 3:39 pm
by abusche
Thanks for providing the data also from my side ...

... some further question: Is it also possible to post the instance-ids that have been used to post scores for the preliminary leader board?

Thanks & Best,
André

Re: Will the full test set be posted?

PostPosted: Thu Mar 31, 2011 3:36 pm
by Marcin
wahoo wrote:Is it possible to get the artist -> song -> segment mapping for the genre task?

Brian, Andre,
We've uploaded now the original dataset from which Training/Test parts were generated:

genresRaw.csv

You will find there the artist-song-segment mapping. It doesn't include the training/test split, but if you match records between challenge files and this file, you should be able to recover this relation.
As to the prelim/final split, it's a bit more tricky. This split is encoded in the evaluation procedure - look at the source code: final & prelim - and it depends on a random seed, passed to the function as a parameter (seed). In this particular case, the seed was 1018072252. If you have a hacker's soul, you can try and execute the code to see what records go where. ;)

Best
Marcin