Will the full test set be posted?

Questions, answers, discussions related to ISMIS 2011 Contest

Will the full test set be posted?

Postby JRM » Tue Mar 22, 2011 11:21 pm

As of 22 March, I was unable to find it.

It would be instructive to attempt to replicate the performance of other data miners, but model evaluation via the online premliminy testset is no longer available.

Thanks!
John McDowell
JRM
 
Posts: 4
Joined: Mon Mar 21, 2011 9:13 pm

Re: Will the full test set be posted?

Postby Marcin » Wed Mar 23, 2011 6:30 pm

Hi John,
All contest files are available now in this folder. In particular, you will find there ground truth decisions for Genres and Instruments test data. Good luck!

Marcin
Marcin
 
Posts: 115
Joined: Fri Oct 09, 2009 6:45 pm

Re: Will the full test set be posted?

Postby JRM » Wed Mar 23, 2011 6:45 pm

Ah, perfect-- thanks for the pointer, and for helping run such a great contest. I know many of us learned a great deal from participating.
JRM
 
Posts: 4
Joined: Mon Mar 21, 2011 9:13 pm

Re: Will the full test set be posted?

Postby wahoo » Thu Mar 24, 2011 3:23 am

Is it possible to get the artist -> song -> segment mapping for the genre task?

Thank you,
Brian
wahoo
 
Posts: 3
Joined: Thu Jan 07, 2010 1:15 am

Re: Will the full test set be posted?

Postby Marcin » Thu Mar 24, 2011 7:26 pm

Brian, we'll check what can be done.
-m
Marcin
 
Posts: 115
Joined: Fri Oct 09, 2009 6:45 pm

Re: Will the full test set be posted?

Postby abusche » Fri Mar 25, 2011 3:39 pm

Thanks for providing the data also from my side ...

... some further question: Is it also possible to post the instance-ids that have been used to post scores for the preliminary leader board?

Thanks & Best,
André
abusche
 
Posts: 2
Joined: Tue Jan 25, 2011 2:25 pm

Re: Will the full test set be posted?

Postby Marcin » Thu Mar 31, 2011 3:36 pm

wahoo wrote:Is it possible to get the artist -> song -> segment mapping for the genre task?

Brian, Andre,
We've uploaded now the original dataset from which Training/Test parts were generated:

genresRaw.csv

You will find there the artist-song-segment mapping. It doesn't include the training/test split, but if you match records between challenge files and this file, you should be able to recover this relation.
As to the prelim/final split, it's a bit more tricky. This split is encoded in the evaluation procedure - look at the source code: final & prelim - and it depends on a random seed, passed to the function as a parameter (seed). In this particular case, the seed was 1018072252. If you have a hacker's soul, you can try and execute the code to see what records go where. ;)

Best
Marcin
Marcin
 
Posts: 115
Joined: Fri Oct 09, 2009 6:45 pm


Return to ISMIS 2011 Contest: Music Information Retrieval

Who is online

Users browsing this forum: No registered users and 1 guest

cron