Organizers: Please confirm that you achieved 79% accuracy

[sanchom]

Organizers: Can you confirm that you achieved 79% classification accuracy on the exact dataset that we're working with (training on "trainFull" and testing on "final")?

[iampat]

I am also interested to know if that is true or not.

[johnray]

I believe the 79% comes from this report

Fermion Report
Addendum

However, in the report the group compared the 3 different samples in pairs. So the algorithms they used only tried to figure out which one of two samples matched. So the comparisons were for A vs B, A vs C and B vs C. Also, the 79% is from the best pair. It may also be worth noting that Fermion had 1500 samples like the contest but used 1200 samples for training and then tested the remaining 300 samples.

If my math is right you could run all 3 Fermion algorithms against each sample and get them to "vote" on which of the 3 samples it is likely to be. In this case you get about 47% accuracy. So I think the actual accuracy achieved is 47% not the 79%.

[sanchom]

I don't want to speculate. I want the organizers to confirm whether or not they achieved 79% accuracy on the dataset they provided us, and if not, tell us what accuracy they _have_ achieved on this competition dataset.

[franklabella]

Organizers: Can you confirm that you achieved 79% classification accuracy on the exact dataset that we're working with (training on "trainFull" and testing on "final")?

We posted one of the reports on the classification results of the National Research Council group. I misstated the accuracy in the sense that it was not truly an accuracy of identifying three substances, but they compared two pairs of substances.
The contest data was obtained separately, although the hardware and experimental protocols are the same.

Frank LaBella,
The Organizers