Data-Challenges: More than a competition

AI is everywhere - in all areas, from medicine to finance to ecology, the use of artificial intelligence bears innovative solutions and raises many questions. From April 21 to 22, 2021, the "Big-DATA.AI Summit" of the German digital association Bitkom addressed real industrial implementation cases as well as ethical and societal challenges of human-machine interaction.

Among the more than 120 international speakers, was Helmholtz scientist Prof. Dr. Lena Meier-Hein from the German Cancer Research Center (DKFZ). There, she is head of the Department of Computer Assisted Medical Interventions and works on biomedical image analysis - her research focuses in particular on surgical data science, computational biophotonics and the validation of machine learning algorithms. Her presentation "Comparative validation of AI algorithms: common pitfalls and new concepts" was attended by more than 200 congress participants and focused on biomedical challenges and their contribution to effective algorithm development based on comparability.

Every year hundreds of new algorithms are published in the field of biomedical image analysis. For a long time, however, the validation and evaluation of new methods was based on the authors' personal datasets, which made a fair and direct comparison of solutions impossible. Common research practice has since changed - for example, hosting international challenges has made it possible to compare algorithms on publicly released datasets. After more than 15 years of biomedical challenges, it was time to take a close look at the current practice regarding the design and organization of biomedical image analysis challenges.

Meier-Hein's talk offers new insights on various topics related to biomedical image analysis challenges, from the choice of metrics and ranking schemes to the pitfalls that can arise when organizing or participating in a challenge.

You can watch a record of her talk here.