AI with swarm intelligence
A new technology for the collaborative analysis of Big Data
Medical research data is a real treasure trove. It can play a crucial role in developing personalized therapies that are tailored to each individual more precisely than conventional treatments. Big data has therefore long been considered the key to better treatment options in science and medicine, which is why it is important to make this data available to researchers.
Such data sharing is naturally subject to the highest data protection requirements and is sometimes technically complicated or even impossible due to the sheer volume of data. As a result, many studies in medical research are limited to local data utilization. The research area of systems medicine is meeting this challenge by using a technology developed by the IT company Hewlett Packard Enterprise (HPE) known as swarm learning.
Swarm learning combines a special form of information exchange over different sites or nodes of a network with methods taken from “machine learning,” a branch of artificial intelligence (AI). The key components of machine learning are algorithms that are trained on data to detect regularities in it – and consequently acquire the ability to recognize the learned patterns in other data as well.
Through this approach, swarm learning is opening up new opportunities for collaboration in medical research. The key is that all cooperating partners can learn from each other without having to share personal patient data. As a result, swarm learning keeps all the research data on site, exchanging only the algorithms and parameters; in other words, the empirical values. In addition to saving resources, swarm learning automatically meets data privacy requirements. The data remains on site, duplicates that distort results are avoided, and only the steadily improving calculation rules are shared among the partners.
The backbone of swarm learning is the “blockchain.” Swarm learning happens on a collaborative basis using rules that all partners have agreed on in advance. This set of rules is recorded in a blockchain. As a result, all members of the swarm are equal. There is no central power that controls what happens and what the results are. The AI algorithms thus learn locally; namely, on the basis of the data available at each site or network node. The learning outcomes, a set of parameters, are collected through the blockchain and merged by the system. The optimized parameters are then passed on to all parties involved. This process repeats itself several times, gradually improving the algorithms' ability to recognize patterns, specifically at each node of the network.
Publications
Warnat-Herresthal et al. Swarm Learning for decentralized and confidential clinical machine learning. Nature (2021). DOI: 10.1038/s41586-021-03583-3
Further information
Press release: With “Swarming AI” Against COVID and Other Diseases (01/18/2024)
Press release: AI with Swarm Intelligence (05/26/2021)
Project head
