An international research team with participants from several universities including the Technical University of Munich has proposed a standardized registry for artificial intelligence (AI) work in biomedicine to improve the reproducibility of results and create trust in the use of AI algorithms in biomedical research and, in the future, in everyday clinical practice. The scientists presented their proposal in the scientific journal “Nature Methods”.
In the last decades, new technologies have made it possible to develop a wide variety of systems that can generate huge amounts of biomedical data, for example in cancer research. At the same time, completely new possibilities have developed for examining and evaluating this data using artificial intelligence methods. AI algorithms in intensive care units, e.g., can predict circulatory failure at an early stage based on large amounts of data from several monitoring systems by processing a lot of complex information from different sources at the same time, which is far beyond human capabilities.
This great potential of AI systems leads to an unmanageable number of biomedical AI applications. Unfortunately, the corresponding reports and publications do not always adhere to best practices or provide only incomplete information about the algorithms used or the origin of the data. This makes assessment and comprehensive comparisons of AI models difficult. The decisions of AIs are not always comprehensible to humans and results are seldomly fully reproducible. This situation is untenable, especially in clinical research, where trust in AI models and transparent research reports are crucial to increase the acceptance of AI algorithms and to develop improved AI methods for basic biomedical research.
AIMe – A standard for artificial intelligence in biomedicine
To address this problem, an international research team including Prof. Dominik Grimm, Markus List, Ph.D. and Josch Pauling, Ph.D., has proposed the AIMe registry for artificial intelligence in biomedical research, a community-driven registry that enables users of new biomedical AI to create easily accessible, searchable and citable reports that can be studied and reviewed by the scientific community.
Freely accessible registry
The freely accessible registry is available at https://aime-registry.org and consists of a user-friendly web service that guides users through the AIMe standard and enables them to generate complete and standardised reports on the AI models used. A unique AIMe identifier is automatically created, which ensures that the report remains persistent and can be specified in publications. Hence, authors don’t have to cope with the time-consuming description of all facets of the AI used in articles for scientific journals and simply refer to the report in the AIMe registry.
Since the registry is designed as a web platform maintained by the scientific community, every user can ask questions about existing reports, make comments or suggest improvements. This feedback from the community will also be included in the annual update of the AIMe standard, and interested researchers can join the AIMe Steering Committee to become more involved in the further standardisation of biomedical AI.
„The AIMe registry implements a minimal information standard that aims to ensure reproducibility as well as comparability of results but without being too restrictive which could hinder the development of novel and innovative AI solutions. Furthermore, by registering your AI solution in the AIMe database it is not only evaluated using the AIMe standard but it also becomes easily findable and accessible. Therefore, AIMe is now an important and necessary step to fuel the early standardization of future AI in biomedicine” reports Josch Pauling, Ph.D., Head of the junior research group ‘LipiTUM’ at the Chair of Experimental Bioinformatics of the TUM School of Life Sciences.
„For a successful clinical application of AI, reproducibility is of utmost importance. Yet it is often difficult to reproduce results when important methodological details are missing. To close this gap, reporting standards such as AIMe are urgently needed. What is more, AIMe does not only offer a standard but comes with a registry that makes report generation, citation and lookup of existing AI methods a breeze” reports Markus List, Ph.D., Head of research group ‘Big Data in Biomedicine’ at the Chair of Experimental Bioinformatics of the TUM School of Life Sciences.
„The AIMe-Registry serves not only as a repository for standardized reports of AI-Models utilized in biomedical publications but it should also be regarded as a best practice protocol of how AI-models and their often neglected but yet critical specifics should be described in scientific publications“ says Dr. Dominik Grimm, Professor for Bioinformatics at the TUM Campus Straubing of Biotechnology und Sustainability.
Original publication
Matschinske, N. Alcaraz, A. Benis, M. Golebiewski, D. G. Grimm, L. Heumos, T. Kacprowski, O. Lazareva, M. List, Z. Louadi, J. K. Pauling, N. Pfeifer, R. Röttger, V. Schwämmle, G. Sturm, A. Traverso, K. van Steen, M. V. de Freitas, G. C. V. Silva, L. Wee, N. K. Wenke, M. Zanin, O. Zolotareva, J. Baumbach, and D. B. Blumenthal: The AIMe registry for artificial intelligence in biomedical research. Nature Methods (2021). DOI: 10.1038/s41592-021-01241-0. https://www.nature.com/articles/s41592-021-01241-0
Website: https://aime-registry.org
Contact
Prof Dominik Grimm
HSWT at TUM Campus Straubing
Professorship of Bioinformatics
Technical University of Munich
E-Mail: dominik.grimm@tum.de