Optimized Design of Scoring Criteria for Large-Scale Innovation Competitions

Abstract

The reasonable and reliable evaluation score calculation scheme wins the top priority in large-scale innovation-competitions currently. Due to the disparity among individual experts, and the existing standard score calculation model merely based on the experts' own circumstance, it cannot fully reflect the comprehensive level of players, resulting in a certain degree of error in the evaluation results. From the perspective of individual and decision-making of a group, this paper improves existing models and introduces the concept of modified scores. To verify the feasibility of the model, four schemes were designed based on the analysis of data distribution. Consistency and difference factors were applied to compare. Come to a conclusion: scheme adopted the new standard score calculation model ranked the top, indicating that the new calculation model is more reliable. Next, considering that the volatile correction factor affected by data, there is a possibility of some poor actual results with high correction scores. To rectify this case, a power exponent is added to the correction factor and a sign function is adopted to specify the positive or negative of correction scores. To verify the feasibility of the model, four sets of controlled experiments were designed with the introduction of two factors: power exponent and reconsideration bonus, as well as the ranking consistency test based on the condition that the award order of the expert agreement was consistent. In the end, it was found that the scheme that introduced both power exponent and reconsideration bonus points had a sorting consistency rate of 74%, which increased by 30% compared to the original model, indicating the rationality of the modification. The expert evaluation standard score calculation model established in this paper comprehensively considers individual and group decision-making, and provides reasonable correction for the differential scoring between experts. At the same time, this scheme provides a reference basis for further optimizing large-scale innovation competition plans in the future.