The popularity of machine learning is steadily growing in biomedical sciences facilitated by the exponential growth in technologies enabling high-throughput data acquisition. The complexity of biomedical data mandates specialized algorithm development. Independent benchmarking and open distribution of developed algorithms are key to safeguard against developer-bias, poor generalizability, and low reusability due to limited documentation. Crowdsourced competitions or Challenges are a compelling avenue for such independent benchmarking.
While independent benchmarking is essential for the field's advancement, organizing and running effective challenges with biomedical data presents a considerable task that requires significant commitment from thought leaders and sponsors, data contributors, domain experts, technical experts, and others. To improve the effectiveness of challenges, it is essential to examine past challenges to identify key features that correlate with high participant-engagement in addition to development of novel algorithms. Identifying such features will be key in replicating highly productive challenges and increasing their value to the field of computational biology.