TunedIT Challenges is a unique platform for organization of on-line data mining competitions like Netflix Prize. As a lecturer, instructor or teacher you may launch a competition for your student group, to make assignment task more attractive and valuable. Thanks to live leaderboard and automated evaluation of solutions, students get instant feedback about their preliminary results, may compare them with others' and improve their solutions, actively searching for the best algorithm and learning from one's own mistakes.
Every course that deals with intelligent data analysis may benefit, like: Artificial Intelligence, Machine Learning, Data Mining, Knowledge Discovery, Pattern Recognition, Computer Vision, Computational Biology and others. Configuring and running the challenge can be done by yourself, on the web site. Student challenges are free for everyone. See the list of current competitions.
Launching a Challenge - Tutorial
1. Create an empty challenge
On New Challenge form, choose title and name.
Click Create and you'll be redirected to the new challenge web page.
The challenge will not show up publicly on the list of challenges until you explicitly publish it.
2. Upload datasets
Go to Evaluation Settings tab and choose datasets from your local disk that should be uploaded to the challenge folder and used in evaluation of solutions. Challenge folder is the single place in Repository where all challenge resources are kept. It's accessible through Repository pages, so you can easily browse its contents, upload additional files that might be helpful for participants, create subfolders, add file descriptions.
3. Configure evaluation of solutions
TunedIT has very flexible testing mechanism. Evaluation methods are implemented as pluggable Java class, called evaluation procedures. You may use any procedure you want, either chosen from examples or implemented by your own.
4. Write overview and task description
There are two places where you can put information for participants: Overview and Task pages. Descriptions are written in wiki text. You can use formatting and insert links to other pages.
5. Check & Publish
Go to Checklist page to view all the settings and check if they're properly configured. If you wish, you may perform dry-run tests to see if everything works as expected. Then, press Publish to make the challenge visible on Challenges page.
6. Accept solutions
The challenge will be opened automatically when the start date passes. You only have to start TunedTester application, which will be automatically downloading and evaluating new solutions submitted by participants. Preliminary results will show up on Leaderboard immediately after calculation.
7. Announce final results
The challenge will be closed automatically when the end date passes, and final results will be published on Leaderboard. You may wish to write summary of the challenge on Overview page.