Optimized Design of Scoring Criteria for Large-Scale Innovation Competitions

Authors

  • Rui Fang
  • Bo Zhang
  • Hao Wang
  • Weihan Chen
  • Xiaoyin Wang

DOI:

https://doi.org/10.54097/h3w23z80

Keywords:

Standard score calculation, consistency test, group decision-making, large-scale innovation-competitions.

Abstract

The reasonable and reliable evaluation score calculation scheme wins the top priority in large-scale innovation-competitions currently. Due to the disparity among individual experts, and the existing standard score calculation model merely based on the experts' own circumstance, it cannot fully reflect the comprehensive level of players, resulting in a certain degree of error in the evaluation results. From the perspective of individual and decision-making of a group, this paper improves existing models and introduces the concept of modified scores. To verify the feasibility of the model, four schemes were designed based on the analysis of data distribution. Consistency and difference factors were applied to compare. Come to a conclusion: scheme adopted the new standard score calculation model ranked the top, indicating that the new calculation model is more reliable. Next, considering that the volatile correction factor affected by data, there is a possibility of some poor actual results with high correction scores. To rectify this case, a power exponent is added to the correction factor and a sign function is adopted to specify the positive or negative of correction scores. To verify the feasibility of the model, four sets of controlled experiments were designed with the introduction of two factors: power exponent and reconsideration bonus, as well as the ranking consistency test based on the condition that the award order of the expert agreement was consistent. In the end, it was found that the scheme that introduced both power exponent and reconsideration bonus points had a sorting consistency rate of 74%, which increased by 30% compared to the original model, indicating the rationality of the modification. The expert evaluation standard score calculation model established in this paper comprehensively considers individual and group decision-making, and provides reasonable correction for the differential scoring between experts. At the same time, this scheme provides a reference basis for further optimizing large-scale innovation competition plans in the future.

Downloads

Download data is not yet available.

References

Wang Xiulian. "College Students Innovative Method Competition and Innovative Talent Training". Science and Technology Entrepreneurship Monthly, 2023, 36(08):181-185.

LV Shulong, Liang Feibao, Liu Wenli. "Evaluation model of judges' scores". Journal of Fuzhou University (Natural Science Edition),2010,38(03):358-362.

Yuan Jixue. "Research on Group decision Mechanism in Expert Subjective rating Competition". China Soft Science, 2009 (02): 173-176+192.

Liang Wei. "Comprehensive Quality evaluation of online judges based on projection pursuit Model". Statistics and Decision, 2017(23):60-63.

Guo Dongwei, Ding Genhong, Liu Wei. "Optimization model of grading and ranking in essay competition". Mathematics in Practice and Understanding,2019,49(08):258-263.

Chen Xing, Fang Ling, WU Songlin et al. "Collaborative modified prediction algorithm for essay score synthesis in competition". Mathematics in Practice and Understanding, 2019, 49(15): 16-23.

2023 "Huawei Cup" 20th China Graduate Student Mathematical Contest in Modeling C data, China graduate student innovation practice series competition management platform,2023.11.20.https://cpipc.acge.org.cn/cw/hp/4.

Pearson, Karl. "Notes on the history of correlation", Biometrika. 13 (1920): 25-45.

M. G. Kendall, "A new measure of rank correlation ", Biometrika, Volume 30, Issue 1-2, June 1938, Pages 81-93.

Downloads

Published

20-01-2024

Issue

Section

Articles

How to Cite

Optimized Design of Scoring Criteria for Large-Scale Innovation Competitions. (2024). Academic Journal of Science and Technology, 9(1), 50-58. https://doi.org/10.54097/h3w23z80

Similar Articles

1-10 of 318

You may also start an advanced similarity search for this article.