Data efficiency: Laurens Engwegen
Focused on addressing the training data bottleneck caused by the high dimensionality and variability in the biological system, we will investigate vision transformer architectures, which can dissociate the location of visual features and their functional role. In combination with biological priors expressed as homomorphic networks, the architecture should be able to identify the hidden relationship between features and generalize to unseen probes/subjects. We want to study the system’s scalability further by augmenting training data with simulated data and employing sim2real transfer methodologies in order to generalize to real-world samples. This project will combine the expertise of the Böhmer, Brinks, and Carroll labs in extracting and interpreting neural data.
Computational efficiency: Alejandro Castañeda
Focused on making the necessary AI more time and energy efficient, this project will make use of the experience of the Tömen and Brinks labs in relating AI models to neural computation and the experience of the Grußmayer lab in efficient super-resolution imaging algorithms. In order to develop more efficient algorithms to process imaging data, we will employ biologically-inspired memory and power-efficient neural network models including binary networks and deep spiking neural networks. To avoid redundant information processing by the models, we will incorporate domain-specific knowledge into the developed architectures and the objective function, taking into account temporal consistency in the longitudinal analysis of 4D data instead of frame-by-frame analysis, and physical knowledge of the system under analysis.
4D reinforcement learning for cancer treatment: Mikolaj Stryja
Focused on developing AI methods for cancer treatment optimization, this work package will combine the radiation therapy (RT) and AI experience of the Perkó lab, control learning expertise of the Böhmer lab, and the experience of the Carroll in in vivo imaging and tracking tissue development and radiation response. First we will develop methods to build a patient-specific virtual radiotherapy (RT) environment predicting the impact of radiation treatments. Second, we will study reinforcement learning methods interacting with the virtual RT environment to derive optimal treatment plans and enable real-time irradiation adaptation. Due to the multi-agent nature of the radiotherapy control problem, we will decompose control into independent agent policies, where agents will decide themselves when to act and how long, drawing on the experience of Böhmer lab in reconciling miscoordination in multi-agent models.
Project 4 - Smart microscopy for in vivo super-resolution: Miyase Tekpinar
Complementary to WP3, here we will focus on incorporating feedback in decision-making routines about patterned illumination for smart microscopy. We will make use of the super-resolution expertise of the Grußmayer lab, the radiation physics experience of the Perkó lab and the machine vision experience of the Tömen lab. This would involve using the inductive priors and domain-specific knowledge of the Grußmayer lab to improve deep learning algorithms for bio-imaging using state-of-the-art deep networks (CNNs and transformer models) for image and video understanding in order to predict, interpolate and extrapolate biologically-relevant time series from imaging data in a data-efficient way. It will allow information-enriched longitudinal super-resolution imaging by minimizing photodamage through smart adaptive choice of microscopy modality with optimal imaging parameters (adaptation of imaging light dose, speed & resolution, AI enhanced vs true super-resolution, structural & functional imaging).